Solaris - 1

The following patch is needed for mariadb-10.1.22 and later on Solaris 11.3:

Update 29.04.2017: A more complete description and solution is presented at:
https://jira.mariadb.org/browse/MDEV-12310

diff -ur mariadb-10.1.22~/mysys/mysys_priv.h mariadb-10.1.22/mysys/mysys_priv.h
--- mariadb-10.1.22~/mysys/mysys_priv.h 2017-03-11 20:09:10.000000000 +0000
+++ mariadb-10.1.22/mysys/mysys_priv.h 2017-04-28 15:48:49.811392259 +0000
@@ -108,7 +108,7 @@

void my_error_unregister_all(void);

-#if !defined(O_PATH) && defined(O_EXEC) / FreeBSD /
+#if !defined(O_PATH) && defined(O_EXEC) && !defined(__sun__) / FreeBSD /
#define O_PATH O_EXEC
#endif

Otherwise mariadb would abort on startup with:

ERROR: 1030 Got error 8 "Exec format error" from storage engine MyISAM


When I migrated from apache with mod_php to nginx with php-fpm a few days ago, php dumped core on me if it was stopped immediately after having been started. This is due to https://bugs.php.net/bug.php?id=66694. The same bug is also filed under #68349.

Basically, port_getn claims to have events ready to be processed, when in reality, there are none. There are quite a few patches for this problem floating around, all of which are too complicated for my taste.

The following very small patch worked fine for me:

diff -ur php-7.0.13~/sapi/fpm/fpm/events/port.c php-7.0.13/sapi/fpm/fpm/events/port.c
--- php-7.0.13~/sapi/fpm/fpm/events/port.c 2016-11-08 16:07:40.000000000 +0000
+++ php-7.0.13/sapi/fpm/fpm/events/port.c 2016-11-12 12:47:24.964771094 +0000
@@ -138,7 +138,7 @@
for (i = 0; i < nget; i++) {

/ do we have a ptr to the event ? /
- if (!events[i].portev_user) {
+ if (events[i].portev_source == 0 || !events[i].portev_user) {
continue;
}

The idea is to check whether a proper event source value is set for any events returned by port_getn.

Apply this patch for openssl 1.0.2e to store a proper library run-path for libcrypto.so in libssl.so:

diff -ur openssl-1.0.2e~/Makefile.org openssl-1.0.2e/Makefile.org
--- openssl-1.0.2e~/Makefile.org 2015-12-03 15:04:23.000000000 +0100
+++ openssl-1.0.2e/Makefile.org 2015-12-31 19:26:56.574207607 +0100
@@ -350,7 +350,7 @@
build-shared: do_$(SHLIB_TARGET) link-shared

do_$(SHLIB_TARGET):
- @ set -e; libs='-L. $(SHLIBDEPS)'; for i in $(SHLIBDIRS); do \
+ @ set -e; libs='-L. -R$(INSTALL_PREFIX)$(INSTALLTOP)/$(LIBDIR) $(SHLIBDEPS)'; for i in $(SHLIBDIRS); do \
if [ "$$i" = "ssl" -a -n "$(LIBKRB5)" ]; then \
libs="$(LIBKRB5) $$libs"; \
fi; \
@@ -372,7 +372,7 @@
echo 'Description: OpenSSL cryptography library'; \
echo 'Version: '$(VERSION); \
echo 'Requires: '; \
- echo 'Libs: -L$${libdir} -lcrypto'; \
+ echo 'Libs: -L$${libdir} -R$${libdir} -lcrypto'; \
echo 'Libs.private: $(EX_LIBS)'; \
echo 'Cflags: -I$${includedir} $(KRB5_INCLUDES)' ) > libcrypto.pc

@@ -386,7 +386,7 @@
echo 'Description: Secure Sockets Layer and cryptography libraries'; \
echo 'Version: '$(VERSION); \
echo 'Requires.private: libcrypto'; \
- echo 'Libs: -L$${libdir} -lssl'; \
+ echo 'Libs: -L$${libdir} -R$${libdir} -lssl'; \
echo 'Libs.private: $(EX_LIBS)'; \
echo 'Cflags: -I$${includedir} $(KRB5_INCLUDES)' ) > libssl.pc



Apply this patch to gcc 4.9.3 to store proper library run-paths for libgcc_s.so in libgfortran, libgomp, libitm and libstdc++:

gcc-4.9.3.diff

gcc-5.3.0.diff

Apply this patch to compile kamailio 4.3.4 on Solaris 11 with full IP V6 support:

kamailio-4.3.4.diff

I configured in.ndpd (svc:/network/routing/ndp:default) to generate privacy-enhanced ipv6 addresses (RFC 4941) by putting "ifdefault TmpAddrsEnabled true" into /etc/inet/ndpd.conf. The auto-configured address derived from the interface's MAC address continues to exist. To my surprise, the system continues to use this (non-privacy-enhanced) address by default!

I found no other way to make solaris use the temporary, privacy-enhanced address than to patch in.ndpd as follows:

--- in.ndpd.orig.dis 2015-11-07 10:56:48.776716882 +0100
+++ in.ndpd.dis 2015-11-07 12:19:59.009907616 +0100
@@ -1,4 +1,4 @@
-disassembly for in.ndpd.orig
+disassembly for in.ndpd

section .plt
@@ -14442,7 +14442,7 @@
prefix_update_k+0x157: 33 d2 xorl %edx,%edx
prefix_update_k+0x159: 83 c8 00 orl $0x0,%eax
prefix_update_k+0x15c: 74 05 je +0x5 <prefix_update_k+0x163>
- prefix_update_k+0x15e: ba 08 00 00 00 movl $0x8,%edx
+ prefix_update_k+0x15e: ba 0c 00 00 00 movl $0xc,%edx
prefix_update_k+0x163: 83 ec 0c subl $0xc,%esp
prefix_update_k+0x166: 6a 00 pushl $0x0
prefix_update_k+0x168: 6a 00 pushl $0x0

This adds the "PREFERRED" flag to the "TEMPORARY" address generated by in.ndpd:

net0: flags=120002000841<UP,RUNNING,MULTICAST,IPv6,PHYSRUNNING> mtu 1492 index 2
inet6 fe80::219:99ff:****:84d5/10
net0:1: flags=120002080841<UP,RUNNING,MULTICAST,ADDRCONF,IPv6,PHYSRUNNING> mtu 1492 index 2
inet6 2003:48:2937:5000:219:99ff:****:84d5/64
net0:2: flags=120c02080841<UP,RUNNING,MULTICAST,ADDRCONF,IPv6,PREFERRED,TEMPORARY,PHYSRUNNING> mtu 1492 index 2
inet6 2003:48:2937:5000:70b7:8159:937c:f526/64

If you know a better way to make the privacy-enhanced address the default one, please leave a comment!

For your convenience, I have prepared patched versions (with an "ashift" setting of 12) of the original zpool program for Solaris 11 11/11 and Solaris 10 8/11 (s10u10).

It has been produced by binary patching, following the idea presented here and here.

This is how to permanently change your hostname in Solaris 11:

root@vbox3:/tmp# svccfg <<EOM
select /system/identity:node
#listprop config
setprop config/nodename="newhost"
setprop config/loopback="newhost"
refresh
EOM

And this is how to permanently change your static IP-Address in Solaris 11, if you are not using NWAM:

root@vbox3:/tmp# svcs svc:/network/physical:nwam
STATE STIME FMRI
disabled Sep_22 svc:/network/physical:nwam
root@vbox3:/tmp# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.3.17/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::a00:27ff:febd:7496/10
root@vbox3:/tmp# ipadm delete-addr net0/v4
root@vbox3:/tmp# ipadm create-addr -T static -a 192.168.3.18/24 net0/v4
root@vbox3:/tmp# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 192.168.3.18/24
lo0/v6 static ok ::1/128
net0/v6 addrconf ok fe80::a00:27ff:febd:7496/10
root@vbox3:/tmp# exit


Solaris 11 has arrived for "early adopters", and I am one of those. As far as I can see, nonthing has changed on the "ashift" front. Pools created on a WD EARS disk by the standard zpool program still have an "ashift" value of 9.

So, here comes the patch for "zpool" for Solaris 11 EA (snv_173):

--- zpool_snv173.save.dis 2011-09-17 14:30:53.911407633 +0200
+++ zpool_snv173.dis 2011-09-17 15:14:23.263512388 +0200
@@ -13444,13 +13444,12 @@
805e72c: ff b5 4c fb ff ff pushl -0x4b4(%ebp)
805e732: e8 c5 5d ff ff call 80544fc
805e737: 83 c4 10 add $0x10,%esp
- 805e73a: 85 c0 test %eax,%eax
- 805e73c: 74 1a je 805e758
- 805e73e: 83 ec 04 sub $0x4,%esp
- 805e741: 68 0b 02 00 00 push $0x20b
- 805e746: 68 4c 5f 06 08 push $0x8065f4c
- 805e74b: 68 d0 5c 06 08 push $0x8065cd0
- 805e750: e8 17 5a ff ff call 805416c <__assert@plt>
+ 805e73a: 68 00 00 00 00 push $0x0
+ 805e73f: 68 0c 00 00 00 push $0xc
+ 805e744: 68 d0 5c 06 08 push $0x8065cd0
+ 805e749: ff b5 4c fb ff ff pushl -0x4b4(%ebp)
+ 805e74f: e8 a8 5d ff ff call 80544fc
+ 805e754: 90 nop
805e755: 83 c4 10 add $0x10,%esp
805e758: 83 ec 08 sub $0x8,%esp
805e75b: 68 44 5e 06 08 push $0x8065e44


It is a straightforward adaptation of the same patch for NexentaStor 3.1.0. You can find the binary here. If you want, you can check its (and my) integrity by doing a "cmp -lc zpool zpool-s11ea".

I am currently experimenting with the Community Edition of NexentaStor. If I am not mistaken, NexentaStor does not have special provisions for disks with a logical sector size of 512 Bytes and a physical sector size of 4096 Bytes (like e.g. the WDXXEARS).

I am a bit fed up chasing down the appropriate source for the "zpool" utility, inserting one line (to set "ashift" to 12) and then recompiling it.

This time (for NexentaStor 3.1.0) I decided to patch the binary. Thus, I do not need to care from which source exactly they produced the "zpool" utility.

It goes like this:

--- zpool_nxs310.save.dis 2011-07-30 13:47:58.675595763 +0200
+++ zpool_nxs310.dis 2011-07-30 15:44:50.378893898 +0200
@@ -12667,13 +12667,12 @@
805d901: ff b5 5c fb ff ff pushl -0x4a4(%ebp)
805d907: e8 28 69 ff ff call 8054234 <nvlist_add_uint64@plt>
805d90c: 83 c4 10 add $0x10,%esp
- 805d90f: 85 c0 test %eax,%eax
- 805d911: 74 1a je 805d92d <make_leaf_vdev+0x265>
- 805d913: 83 ec 04 sub $0x4,%esp
- 805d916: 68 d9 01 00 00 push $0x1d9
- 805d91b: 68 00 5f 06 08 push $0x8065f00
- 805d920: 68 10 5f 06 08 push $0x8065f10
- 805d925: e8 8a 66 ff ff call 8053fb4 <__assert@plt>
+ 805d90f: 68 00 00 00 00 push $0x0
+ 805d914: 68 0c 00 00 00 push $0xc
+ 805d919: 68 10 5f 06 08 push $0x8065f10
+ 805d91e: ff b5 5c fb ff ff pushl -0x4a4(%ebp)
+ 805d924: e8 0b 69 ff ff call 8054234 <nvlist_add_uint64@plt>
+ 805d929: 90 nop
805d92a: 83 c4 10 add $0x10,%esp
805d92d: 83 ec 08 sub $0x8,%esp
805d930: 68 f8 5e 06 08 push $0x8065ef8


By overwriting a conditional call to "assert" which is not really needed (if you know what you are doing), one is able to sneak in another call to "nvlist_add_uint64" for the "ashift" value. The "ashift" string comes from 0x8065f10, where previously the string for the "assertion failed" message was located. I am using a hardcoded value of 12 for "ashift" (the "push 0xc").

You can find the binary here.

If you want, you can check its (and my) integrity by doing a "cmp -lc zpool zpool-nxs310".

Unfortunately, there is no "hdparm" utility for Solaris, as there is for GNU/Linux. I had a look at the sources and porting it to Solaris will require a considerable effort. It is not difficult, at least not for setting/getting the various attributes, it is just that "hdparm" was probably never written with portability to other operating systems in mind (no criticism!). A lot of #ifdefs will have to be added.

Then there are the "smartmontools" which are written in a highly modular way, aiming for portability from the very beginning. The smartmontools deal with S.M.A.R.T. data, but adding some options to get/set APM (Advanced Power Management), AAM (Advanced Acoustic Management) and caching attributes would be very easy from a technical and coding standpoint. I am afraid, they just don't belong there. Otherwise someone would have done it long ago.

So I was back at square one today, wanting to mess around with my disks but not having a tool. So, I just wrote something.

Here are setapm.c and a corresponding setapm binary for s10u9 for configuring the APM feature attribute on disks and an almost identical setaam.c and setaam for setting the AAM feature attribute.

They are very minimalistic. They will not even read out the configureation values from the disk for you. You'll only be able to set them.

And this will only work on systems where the sata/ahci sd driver is used, not the legacy ata/ cmdk driver, because for the latter the USCSI ioctl is not implemented.

The "zpool iostat" command shows some zpool statistics:

foo@bar> zpool iostat
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
ei01 123G 109G 1 1 19.7K 18.5K
ei02 46.8G 112G 2 0 335K 92.8K
---------- ----- ----- ----- ----- ----- -----

The meaning of these figures is not really documented in the man page, but easy to deduce: It is the average number of read and write requests per second and the average amount of data read and written per second, counted from boot (or, to be more correct, zpool import) time.

With an additional -v parameter, we get the same information not only on pool level, but also on device level:

foo@bar> zpool iostat -v
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
ei01 123G 109G 1 1 19.7K 18.5K
mirror 123G 109G 1 1 19.7K 18.5K
c0d1 - - 0 0 13.9K 18.5K
c1d0 - - 0 0 13.6K 18.5K
---------- ----- ----- ----- ----- ----- -----
ei02 46.8G 112G 2 0 335K 92.8K
c0d0s7 46.8G 112G 2 0 335K 92.8K
---------- ----- ----- ----- ----- ----- -----

If you want the raw data for this, then you have to help yourself. See the zpstat.c I have just written. Compile instructions inside (very simple!). Usage: zpstat <pool1> [<pool2> ... <pooln>]. For each given pool, it will iterate through the in-core vdev tree structure and for each vdev it will try to extract statistics information. Output looks like this:

ei02
type: 'root'
id: 0
vdev_stats.vs_timestamp: 693868 seconds
vdev_stats.vs_ops[ZIO_TYPE_NULL]: 1
vdev_stats.vs_ops[ZIO_TYPE_READ]: 1835587
vdev_stats.vs_ops[ZIO_TYPE_WRITE]: 646427
vdev_stats.vs_ops[ZIO_TYPE_FREE]: 0
vdev_stats.vs_ops[ZIO_TYPE_CLAIM]: 0
vdev_stats.vs_ops[ZIO_TYPE_IOCTL]: 7185
vdev_stats.vs_bytes[ZIO_TYPE_NULL]: 0
vdev_stats.vs_bytes[ZIO_TYPE_READ]: 238795578368
vdev_stats.vs_bytes[ZIO_TYPE_WRITE]: 65714164736
vdev_stats.vs_bytes[ZIO_TYPE_FREE]: 0
vdev_stats.vs_bytes[ZIO_TYPE_CLAIM]: 0
vdev_stats.vs_bytes[ZIO_TYPE_IOCTL]: 0
vdev_stats.vs_read_errors: 0
vdev_stats.vs_write_errors: 0
vdev_stats.vs_checksum_errors: 0
vdev_stats.vs_self_healed: 0
type: 'disk'
id: 0
path: '/dev/dsk/c0d0s7'
path: 'id1,cmdk@AMaxtor_6L200P0=L41FZEGH/h'
path: '/pci@0,0/pci-ide@1f,1/ide@0/cmdk@0,0:h'
vdev_stats.vs_timestamp: 693868 seconds
vdev_stats.vs_ops[ZIO_TYPE_NULL]: 1
vdev_stats.vs_ops[ZIO_TYPE_READ]: 1835587
vdev_stats.vs_ops[ZIO_TYPE_WRITE]: 646427
vdev_stats.vs_ops[ZIO_TYPE_FREE]: 0
vdev_stats.vs_ops[ZIO_TYPE_CLAIM]: 0
vdev_stats.vs_ops[ZIO_TYPE_IOCTL]: 7185
vdev_stats.vs_bytes[ZIO_TYPE_NULL]: 0
vdev_stats.vs_bytes[ZIO_TYPE_READ]: 238795578368
vdev_stats.vs_bytes[ZIO_TYPE_WRITE]: 65714164736
vdev_stats.vs_bytes[ZIO_TYPE_FREE]: 0
vdev_stats.vs_bytes[ZIO_TYPE_CLAIM]: 0
vdev_stats.vs_bytes[ZIO_TYPE_IOCTL]: 0
vdev_stats.vs_read_errors: 0
vdev_stats.vs_write_errors: 0
vdev_stats.vs_checksum_errors: 0
vdev_stats.vs_self_healed: 0

You get basically the same information as from "zpool iostat", but before averaging, so you can a) calculate an average I/O size, and b) extract the figures, do something (for a longer while), extract the figures again and get an average figure for exactly this time period.

DISCLAIMER:
a) This code uses private interfaces of Solaris. It is not portable across releases. In fact, you will have to compile seperately for each release.
b) This is not the proper way to seriously gather information for performance-tuning or so. For this purpose, use real weapons. Like "dtrace".
c) I have written this for my own educational purposes (to learn about the libzfs interfaces and nvpairs and ZFS internals in general). I do not claim fitness for any particular purpose...

I have just written a small onefifty-liner to check the physical block size of disk devices on Solaris. I have done this to show that the WD20EARS does indeed (erroneously!) report a physical sector size of 512 bytes.

See: blocksize.c (Source) and blocksize (binary, compiled on Solaris 10/x86).

Note that on the x86 platform, this will only work if the disks are attached to the sd driver (because the DKIOCGMEDIAINFOEXT ioctl is implemented only there and not in the cmdk driver). This will be the case if you use an AHCI controller supported by ahci(7d).

In my case (this is on 5.10 Generic_142910-17)

root@ilbirs> format -e </dev/null | egrep "[0-9]\."
0. c0t0d0 <DEFAULT cyl 30390 alt 2 hd 255 sec 63>
1. c0t1d0 <ATA-Hitachi HDS72107-A70M-698.64GB>
2. c0t3d0 <ATA-WDC WD20EARS-00M-AB50-1.82TB>
3. c0t4d0 <ATA-Hitachi HDS72107-A70M-698.64GB>

root@ilbirs> ./blocksize /dev/rdsk/c0t3d0
dkmp.dki_capacity = 3907029167
dkmp.dki_lbsize = 512
dkmp_ext.dki_capacity = 3907029167
dkmp_ext.dki_lbsize = 512
dkmp_ext.dki_pbsize = 512
root@ilbirs> ./blocksize /dev/rdsk/c0t4d0
dkmp.dki_capacity = 1465149167
dkmp.dki_lbsize = 512
dkmp_ext.dki_capacity = 1465149167
dkmp_ext.dki_lbsize = 512
dkmp_ext.dki_pbsize = 512

The above WD20EARS is revision "00MVWB0". I have another one at revision "00J2GB0", unfortunately hanging on a controller unsupported by ahci(7d).

If anybody has a WDxxEARS correctly reporting a dki_pbsize = 4096, I would love to hear it.

Edit:
If anybody has a Samsung HD204UI, I would also like to know if this disk correctly reports the physical block size.

I am in the process of trying out Solaris 10 09/10 (s10u9) on my machines, and encountered problems on my Dell PowerEdge 840 Server. The NIC on that box is a Broadcom 5721.

The NIC does come up, but only receives broadcast and multicast packets, not unicast ones.

As a workaround, I am using the driver from Solaris 10 10/09 (s10u8), which works fine.

Update 02.02.2011:

Problem solved!! I tried to install Solaris 11 Express on this machine and again the NIC wouldn't work. This time I tried a bit harder and found out that I had disabled the PXE Extensions for the NIC in the BIOS. After switching the PXE Extensions back on, it works!

Must be some subtle initialization problem.

Until recently, some nasty pointer arithmetic/type conversion in httpd led to a miscompilation by newer gcc's on x86. See here and here for more information.

This patch is already included in httpd-2.2.17.

Something still seems to be wrong, however. When compiling with "-O2" and gcc-4.5.2, httpd would still fail for me (in https processing, http was doing fine). I tried one of the aforementioned workarounds (using "-O2 -fno-strict-aliasing") and the problem vanished.