Performance of Microsoft Storage Space 2016 on Dell PERC H700 RAID controller.

Before we get into this post let me be up front and say that this is a completely unsupported configuration by Microsoft. Much like UnRAID or ZFS, storage spaces wants direct access to the disk to work properly. You can force Storage spaces to work with RAID volumes, but if you have problems MS support will not assist. Your biggest issue will be handling failures, as Storage spaces will not be able to accurately predict SMART failures.

Now with that out of the way. Here is the setup for this. Dell T710 as the server, Perc H700 with 512MB BBWC, 2x Samsung 850 EVO + 2x 500 GB Seagate Constellation ES SATA drives in a Tiered mirror space, and 4 500GB Seagate Constellation ES + 4 1TB WD Black SATA disks in a Parity space with 30GB SSD Write Cache. I wanted to compare how drives configured on a RAID controller that had its own built in write cache would stack against my configuration.

We setup the test the same as the earlier test on my homelab server. 100GB Testfile from IOMeter and a 75% Read 512 and 75% Read 4k. I only tested against the host, not the VM, and we tested against both the Mirrored tiered drives, and the Parity array. Here are the results.

SERVER TYPE: Dell T710
CPU TYPE / NUMBER: Xeon x5560 x2
HOST TYPE: Server 2016
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Storage Spaces Tiered Mirror
Test name Latency Avg iops Avg MBps cpu load
512 B; 75% Read; 0% random 0.12 8601 4 0%
4 KiB; 75% Read; 0% random 0.15 6742 26 0%
SERVER TYPE: Dell T710
CPU TYPE / NUMBER: Xeon x5560 x2
HOST TYPE: Server 2016
STORAGE TYPE / DISK NUMBER / RAID LEVEL: Storage Spaces Parity
Test name Latency Avg iops Avg MBps cpu load
512 B; 75% Read; 0% random 0.11 8638 4 0%
4 KiB; 75% Read; 0% random 0.16 6169 24 0%

When we compare these figures to my setup we see a couple things. Firstly the CPU load on the host is considerably less. I was seeing between 10-15% CPU utilization during my tests, because my CPU has less compute power than a single Xeon, let alone 2.Next we notice that on average my latency was lower. This is due to the fact that I am using NVMe Cache instead of just SATA SSD cache for my tiers. Lastly we look at average IOPS and throughput which again are significantly higher on my system because we are seeing the NVMe cache really help things out.

Conclusions:
A RAID controller with its own dedicated cache on the card, not only is an unsupported configuration, however also really isn’t a substitute for NVMe write cache. Additionally Parity spaces greatly benefit from more powerful CPUs in terms of overall system performance. Replacing my stock i7-920 with an i7-965 with a QPI Bus speed of 1×6.2k should help cut the CPU overhead down some, and is about a $75 upgrade these days.

Advertisements

Using a Dell PERC H310 on an eVGA x58 motherboard to provide 8 SATAIII ports.

Continuing on my venture of rebuilding my Hyper-V box to be a super hyperconverged storage and compute box, I realized that I needed to be able to add more SATA drives to my system, because 7 was just not going to be enough. I decided I would purchase a HBA to do the job, and went to Reddit /r/datahoarder to ask what they thought would be the best option for this venture. The two cards recommended were the IBM M1015 and the Dell PERC H310. Both cards are dual port SAS RAID cards, that are really LSI cards underneath and can be cross flashed to the LSI IT firmware to allow for straight passthrough of disks (which is exactly what I needed, I’ll explain in another post about RAID disks vs Passthrough ATA disks in Storage Spaces). I found a PERC H310 on eBay for $44 shipped, so I went that direction. I also bought two SAS Breakout cables for $15 combined and then waited for everything to show up in the mail.

Once everything got here and was checked over to make sure it was OK, I got to the process of flashing the card using this fantastic guide by Vladan Seget. This proved extremely difficult in my situation, and all I can say is that I’m glad I have friends. The specific motherboard that I am using is 132-BL-E758 – EVGA X58 SLI running BIOS version 83.

Issue Number 1: No Boot.
After plugging in the card the system wouldn’t even POST. Just kept throwing the post code “86” eVGA’s manual shows this as “Reserved” so there is absolutely no help here. I reset the CMOS hoping that would fix it, and thankfully that was all it took to get the system to POST. Unfortunately I still ran into an issue where even after the machine showed all the system info and the JMicron AHCI controller info, it just went to a black screen. I couldn’t get to BIOS, windows didn’t boot. I assumed there was something wrong with the card as removing the controller allowed boot to Windows without issue. I tried different slots, multiple CMOS resets, cold boots everything I could think of. Eventually I just waited at the black screen and low and behold about 2 minutes later I finally see a “0 Virtual Drives Handled by BIOS” message, and the pre-boot completes and I am able to select my boot device! Woo hoo!

Issue Number 2: Megarec sees that the card is installed, but refuses to flash the card to an empty rom.
So this was annoying. I could run all the utilities to find info about the card, and see that it was healthy and attached and what firmware it was running, but I couldn’t get Megarec.exe to actually flash the card. it would just launch the .exe and sit at a blinking curser. Again, I tried multiple slots, different versions of FreeDOS and different USB keys and no dice. The solution was to put the card into a spare Dell R610 server that a friend had and flash it there. This did work successfully and I was able to get the base IT firmware loaded.

Issue Number 3: Random system resets when the card was installed.
This one took a while to diagnose. When the PERC was installed (again didn’t matter what slot) the system would randomly restart. Removing the PERC resolved this issue, but then I can’t use my drives. So what the hell? Well a bunch of googling later I found Yannick’s Tech Blog who explained that by masking pins B5 and B6 you block the SMBus signals from the card which are what generate your boot issues and for me were causing the system to reboot. After masking off the pins with electrical tape, everything is working as it should. In the last 24 hours I haven’t had any issues with performance, or stability of the system.

Once all issues were resolved I have been able to cleanly boot the system multiple times, and now that the card is in pure IT passthrough mode there is no longer the boot delay.