Performance Test Metrics

One of the principal negatives I've been reading about Storage Spaces is the poor performance of parity-protected volumes. Having used UnRaid for some time, I'm quite used to the performance limitations of parity-protection and am quite happy to live with this in my use case (serving media files) as it's mostly about reading data and the benefits of parity protection in terms of maximising disk space usage and resilience are important factors to me.

However, I'm interested in the performance I'm getting and finding ways of improving access speeds and so have conducted some tests to benchmark the current system and have a way of determining the efficacy of any future improvements.

The Tests

Checking real-world disk performance over a network can be a tedious affair. Usually, I would copy a large-ish file from one machine to another, time the transfer and calculate a MB/s transfer speed. This can be problematic as you need to work around file caching settings on both machines to ensure accurate results. It's also somewhat time consuming when seeking a wide data set.

For these tests, I've used Aja System Test software. This is a free utility intended for assessing performance  of storage devices in video post-production environments. However, it has a lot of features that make it ideal for the kind of testing I want to do; multiple file size options, cache disabling, raw data or graph outputs and it runs on OSX or Windows, plus it does all the calculations!

For a comparable data set, I conducted the following tests;

1. Read & Write to Windows Storage Spaces locally on the server itself to determine the maximum attainable performance
2. Read & Write to Windows Storage Spaces over network
3. Read & Write to Unraid parity protected array over network
4. Read & Write to single drive locally on server to determine overhead introduced by parity/spaces
5. Read & Write to single drive over network

In the case of items 1-3, tests with a  full suite of file sizes ranging from 128MB to 1GB were conducted. For the single drive test, only 1GB data was used.

Hardware 

The hardware used was as follows;

MediaServer8
Windows Server 8 Beta running on an ASUS P5E-VM motherboard with an Intel Core2 Quad processor Q6600 clocked at 3GHz and 6GB RAM. A single storage spaces Virtual Drive is configured on top of 6x 1TB WD Green SATA drives connected to the motherboard. An additional 500GB stand-alone drive is connected to a SiL 3132 PCIe SATA controller.

UnRaid
UnRaid v5b14 running on an Asus PCH-DL , dual 2.4GHz XEON processors and 2GB RAM. Storage comprises a parity protected array of 3x 1TB WD Green SATA drives and 3x 500GB Hitachi SATA drives connected to a Supermicro AOC-SAT2-MV8 PCI-X SATA controller with the exception of one of the 1TB drives which is connected to a motherboard socket and used as the parity drive.

Client
A 2.4GHz Core 2 Duo iMac with 4GB RAM running OSX Lion.

All machines are connected to a GB switch on a Cat5e based network. The servers connected directly, the iMac via a patch panel. All machines reported full GB (1000b/T) connections.

Results

test results showing read/write speeds for a variety of file sizes in MB/s

The results were interesting to say the least. Tests run on the server itself revealed average read and write speeds to the storage space of 149.9 / 48.15 MB/s.

Compare this to speeds of 94.5 / 83.3 MB/s accessing the standalone drive. So the performance penalty of writing to the parity protected storage is about 35 MB/s or 42%.

Reading data from the storage space was significantly better (37%) compared to the standalone drive. This is likely due to the striped nature of the storage spaces configuration.

Interestingly, read and write speeds were fairly consistent across all files sizes.

Across the network, performance dropped significantly. Average writes to the storage space were 15 Mb/s with data read at 46 Mb/s. Writes to the standalone drive over the network were 34.9 MB/s with data read at 45.9 MB/s. Here, it seems the faster read speeds from the striped array have been lost and writing data across the network is significantly slower.

Average write speeds to UnRaid across the network were 12.9 MB/s with data being read at 62.55 Mb/s. So the Linux system on older hardware results is a slightly degraded write performance but significantly better read speeds. Hmm.

However, there is one anomaly that concerns me and will be the focus of the next stage of work. Have a look at these results for 1GB tests to each server;

1GB UnRaid Network Test

1GB Windows Server Network test

The UnRaid test graph (left) is as one would expect, consistent read performance (green) with lower but still consistent write performance (blue). (click on any image to enlarge it).

Note however, the Windows Server graph. Write performance was equivalent to read performance for most of the transfer but falls off at the end. This matches the instantaneous write speeds reported during the test. Most of the way, the tool was reporting speeds around 40MB/s but at the end, it seemed to get stuck for a period and then reported an average speed of around 15 MB/s. 

This end stage drop-off was consistent across all tests to this storage across the network but was not exhibited in either the local server test or tests of the standalone drive. 

What's going on here? It looks like Windows Server may be caching the data and writing all data or maybe just parity at the end which is slowing the operation down.

Conclusions

average read speeds for all tests (MB/s)

average write speeds for all tests (MB/s)

There was a significant drop off in writes to the parity protected storage spaces compared to stand-alone drives in both local and network tests and this is to be expected. The storage spaces array exhibits much better local read performance compared to a standalone drive  which is also to be expected.

Read/Write operations to both Windows Storage Spaces and UnRaid across the network were largely comparable with UnRaid having somewhat better read figures.

Instantaneous write speeds to the Windows storage space across the network were good but something is happening at the end of the process that is skewing the overall reported speed. This needs to be investigated. (I've posted a question to the MS Server Beta forums so let's see if anyone there can shed some light)

Also, general performance of both systems across the network is degraded more than I would have expected so I'll need to look at optimising the network itself and investigating the network connection on the Windows Server.

Lot's done, more to do.