« An Important VMworld Announcement | Main | Join the NetApp Virtualization Community Party at VMworld 2010 »

August 17, 2010



A bit of a different question.....if you want to dedicated (2) vmkernels to iSCSI traffic and a separate vmkernel to NFS traffic, is it possible to keep iSCSI from using all 3 vmkernels and just use 2? It seems like it should be (possibly via non-routable IP subnets on top of the same VLAN) but I keep seeing more paths than should be there.

Vaughn Stewart

@Andrew - IMO if you are using 10GbE it doesn't matter. Use 2 links for all traffic.

If you are using 1GbE, then you can use multi-session TCP with iSCSI which will bind the iSCSI IO to 2 links. In order to bind NFS to the third link you will need to have the iSCSI traffic on one subnet and the NFS on a second as NFS uses the hypervisors routing table to decide which VMkernel port to use. You will also need to restrict the NFS exports to only the subnet of the NFS traffic.


Andrew are you by any chance attempting IP Hash for NFS as well as multiple TCP sessions for iSCSI? If so and you have some extra pNICs, you could do something like this:

It's nice to see that v2.1 of TR3749 finally explicitly warns about mixing iSCSI port binding and NFS NIC Teaming. However, the TR doesn't clarify that the warning wouldn't apply if iSCSI and NFS were configured on separate vSwitches.

Charlie Dellacona

Interesting. One of your replies on Chuck Hollis' blog claimed that RAID-10 did not offer enough protection for today's workloads. Yet this document is unclear about the fact and only states it with certainty about RAID-5.

For the record, are you claiming that the mean time to data loss of RAID-DP is superior to that of RAID-10 for an equal number of physical disks?



That has already factually been demonstrated in 2004 by Peter Corbett et al in this FAST '04 technical paper:

RAID-DP (high performance RAID-6) is more resilient than RAID-10, at 46% less parity overhead but with equivalent performance characteristics (unlike traditional RAID-6).

Charlie Dellacona


Thank you for the paper reference. However, I don't see anything addressing my question. Please point out the specific reference where it factually demonstrates that RAID-DP has an MTTDL superior to RAID-10 with all failure scenarios considered. Not just dual failure.

Examine multiple physical disk unit failures in a 16 disk group. On the first failure, data remains available for both RAID types. On the second with RAID-DP (and any RAID-6) data remains available. On RAID-10 data becomes unavailable 1/15th of the time (the random failure hits the opposing failed mirror). On failures n=3 to 8, RAID-DP always fails. For RAID-10 the likelihood of data unavailability (failure of a mirror of an already failed disk) is only (n-1)/(17-n), considerably better than 100% as in RAID-DP. Only on the 9th failure does data unavailability become certain with RAID-10, whereas it is certain for failures beyond 2 with RAID-DP. The resiliency improvement of RAID-10 over RAID-DP (or any RAID-6) grows as the number of disks is increased.

Further, I only asked about MTTDL. But since you want to talk about parity overhead, in a 4 disk group, RAID-DP and RAID-10 have exactly the same overhead (50%). In the more useful 16 disk case, RAID-10 overhead is 8 disks (50%) and RAID-DP overhead is 2 disks (12.5%) a reduction of 6 (37.5% of total). Where does 46% come from (a group of 50 disks?!?)? It’s not in your paper.

As far performance is concerned, RAID-10 will outperform RAID-DP for reads, since for any member block in a RAID-DP only 1 IO at a time can be done, and RAID-10 can do 2 (one to the disk containing the sector and one to its mirror.) Some systems, like the Symmetrix (and very old DEC HSCs…), will actually optimize between a disk and its mirror directing the IO to the physical disk that can service the IO soonest, a neat little optimization unavailable to RAID-6 derivatives. As for writes, RAID-DP has to update those additional 2 parity blocks so there are at least 3 writes, with RAID-10 there are only two so array IOPS performance will be better with RAID-10. I will concede that for full stripe writes, parity RAID schemes in general (RAID-DP included) have less overhead and will perform better, but they are not a typical case. RAID-10 has the further, but now small given fantastically fast processors, advantage of not having to calculate any parity.

RAID-10 is expensive, but like all things you don’t get what you don’t pay for.

The comments to this entry are closed.