As a follow-up to my post about flaws in many SSD benchmark reviews, I just stumbled upon an excellent presentation by Christopher George, founder and CTO of DDRdrive, who presented ZIL Accelerator: DRAM or Flash (pdf) at the OpenStorage Summit 2010 on October 26-27, 2010. His company makes the DDRdrive X1: a DRAM-based storage device in the form of a PCIe card that is optimized for the typical aggressive random write workloads that a "ZIL accelerator" is submitted to, which is a device that the Solaris ZFS file system uses to store the ZFS Intent Log to accelerate synchronous writes.
On slide 31 of the presentation, Christopher explains that some builds of the Iometer benchmark tool use a repeating byte pattern for the I/O transfer data, whereas others use a pseudo random sequence. He then shows what exactly the blocks look like and explains why benchmarking with the former pattern is a bad idea as it will lead to unrealistic, artificially inflated performance numbers, as I was explaining in my previous post.
Another interesting tidbit of information in this presentation confirms a not so well-known fact about storage devices: some of them ignore the SCSI SYNCHRONIZE_CACHE or ATA FLUSH CACHE commands that are supposed to flush the on-drive cache to stable storage. Operating Systems rely on these commands to guarantee the consistency of file system data. It is rumored that many SSDs do so. I wish that a research group/hardware benchmarking site/storage company would one day publish a study on this to expose cheating SSD vendors.
In the rest of the presentation, Christopher benchmarks his DDRdrive X1 against two OCZ SSDs (Vertex 2 EX and Vertex 2 Pro) that do not cheat and that even implement a really nice feature: these SSDS have an on-board super-capacitor acting as a power backup to allow them to flush the cache to the NAND chips in case of a power outage. This is an ingenuous features that as far as I know only OCZ offers. Christopher then exposes that even the best SSDs like these 2 models significantly under-perform compared to the DDRdrive because of complicated wear leveling schemes and the write amplification problem that affect SSDs.
All in all, I really enjoyed reading this presentation. It demonstrates that a very solid understanding of the underlying technologies is required to properly benchmark SSDs and understand their performance characteristics.