ZFS and Solaris notes
2009-02-09 15:16:08
It is possible to install Open Solaris on a Dell server with an MD1120 disk array, and use ZFS on the individual disks in a JBOD config.
Trick number one: you can't actually just tell the PERC6 controller to put all the drives in a big JBOD config. You actually have to go in the BIOS and create 48 individual RAID 0 configs each with one drive. The UI for doing this sucks donkey balls, each drive takes about 10 keystrokes.
This config guide is awesome to get started.
In doing this I ran into funky bugs in the solaris termcap. This blog post has good info on the solution: just replace the Solaris xterm-color termcap with one that isn't broken.
We successfully set up a zfs with a pool of 8 raidz2 virtual devices, each with 6 disks. So a total of 4.6 TB of usable storage with decent I/O and redunancy. It's hanging off of a Dell R900 16-core Xeon with 128GB of RAM. Benchmarking it now with some mysql databases.
More:
zfs get all <POOLNAME>
shows all the compression and blocksizes and other props of the file system.