With the recent release of Intel’s Haswell-EP (aka Grantley) range of CPUs we have a new assortment of motherboards to match. One of the Supermicro server offerings is the X10SRH-CLN4F single-socket (UP, as opposed to dual socket – DP) board.
Some quick specs:
- Socket R3 (2011-3) – supports E5-16xx V3 and E5-26xx V3 CPUs
- Intel C612 chipset
- Eight DDR4 ECC DIMM slots – 512GB max, 2133MHz support
- PCI-E lanes galore – x4 and three x8 v.3.0, x4 and x8 v.2.0
- Quad gigabit ethernet (Intel i350-AM4)
- 10x 6gb/s (SATA3) SATA ports onboard from chipset
- LSI 3008 12gb/s SAS controller onboard – 8 ports
- IPMI
- USB 3.0 ports – 2 rear, internal header, internal port
- SuperDOM support
- ATX format
That’s a pretty impressive feature list for an ATX board. How much is it? Currently you can get one landed in Australia for $450-500AU and Newegg in the US stocks it at ~$420US. Let’s take a look at the board itself, starting at the top right:
Curiously, despite the sticker there is only one 8-pin power connector on this board… which is quite normal for a single-socket board, except for a handful of overclocking-oriented boards. The 24-pin power connector is situated conveniently for most server PSUs in rack cases; for tower cases we still prefer them at the top (fan fouling issues aside) than the middle on the side as it’s easier to hide the cables! Hiding next to the 8-pin power connector is the power SMB connector:
Moving on down the board, we find two 4-pin fan connectors right below the 24-pin connector:
…followed by a third further down:
Front I/O panel connectors go here:
USB 3.0 header:
…and below that two SAS connectors, and another 4-pin fan header!
These are hooked up to the SAS3008 chipset, providing an additional 8 ports – at 12gb/s. 6gb/s and 3gb/s are also supported, and you can hook up SAS or SATA drives. If you have a chain of expanders you can attach up to 1,000 drives to just one of these chipsets. The SAS3008 chipset hides under the heatsink next door:
It runs a PowerPC processor (476) at 1.2GHz and utilises a PCI-E 3.0 x8 connector unlike the previous SAS2008 designs which were PCI-E 2.0. It also supports RAID0, 1 and 10. For more information, head to the product page. Right below the heatsink is a pair of little sockets:
These are a pair of SATA DOM power connectors, for powering small SATA disk-on-module units. Back towards the edge of the board below the USB 3.0 header, we have yet another 4-pin fan connector:
….and still another in the corner:
This leads us right into the plethora of SATA ports native to the new chipset:
Ten in total are supported at 6gb/s. Combine this with the eight additional ports from the SAS3008 chip and you have support for 18 drives without a single PCI-E add-in card. The orange ports support Supermicro’s SuperDOM modules – basically mini solid state drives which are powered through the orange SATA ports. At around $100US for a 16GB model it’s not a cheap figure per GB but if you’re looking for a compact way of having one or two OS SSDs it may be an option worth considering. Along the lower edge of the board we have some headers:
The three on the left here are USB 2.0 headers, the white is an IPMB connector for IPMB I/O cards, and the three headers to the right are SGPIO. To the left of this row is a serial port or COM header:
Moving back up the board, we have the PCI Express slots:
From the top, physical/electrical:
- x8 / x4
- x16 / x8
- x8 / x8
- x8 / x8
- x4 / x2
- x8 / x4
The top four – bolded – are PCI-E 3.0 and are connected directly to the CPU, the bottom two are PCI-E v. 2.0 and are connected to the PCH (chipset). The top slot is rather close to the lowest DIMM:
…so if you have a card which has components on the back – or a heatsink which wraps around significantly – make sure to check that it isn’t going to touch the lowermost RAM DIMM. Speaking of DIMMs:
This board has eight total, in two banks of four either side of the CPU socket. It supports unbuffered and registered DDR4 ECC DIMMs, up to 2133MHz in speed. Note that some of the Haswell-EP CPUs don’t support 2133MHz DIMMs – if you need the faster RAM speed make sure to double check the speeds supported by your Xeon. This board can take up to 512GB of LRDIMMs – e.g. eight 64GB sticks – but for many home and small business users the practical limit of expenditure is probably going to be more like 64 or 128GB.
Towards the I/O shield we can see the black heatsink over the quad-Ethernet controller:
…and then we move around to the I/O itself:
Left to right:
- Serial port
- Dedicated IPMI ethernet
- Two USB 2.0
- Two USB 3.0
- Four Gigabit Ethernet (Intel i350-AM4)
- VGA
It’s good to see USB 3.0 on the rear, as a USB 3.0 sneakernet is a practical way of migrating a lot of data in a lot less time than it takes via USB 2.0, and this is the sort of thing we are seeing more home storage enthusiasts doing. Dedicated IPMI is of course an excellent feature to have in a server motherboard, and Supermicro’s IPMI is quite well implemented. The Intel i350-AM4 gigabit ports support teaming; even if you don’t require more than a single gigabit stream coming out of your server it can be extremely handy to be able to separate VM traffic by physical ethernet ports; we have seen users make use of this by dedicating two ports to a firewall VM and use the remaining two for VM/file storage use. We also like to see a serial port on the rear as there are still a fair few devices around which require a serial connetion from time to time (UPS, networking equipment, etc.).
Moving around the board again;
Two more fan headers, 4 pin, in the top corner behind the serial port. And almost back to the start – four more DDR4 DIMM slots:
And finally, inbetween the RAM slots we have Socket R3, or 2011-3:
Note that this board has implemented the narrow ILM version of this socket – the square Socket 2011 cooler setups WILL NOT MOUNT. You can visually tell which it is by looking at where the mounting holes for the cooler are – here you can clearly see that they are arranged in a rectangular shape, with the long side running parallel with the DIMM slots.
The flipside of the board doesn’t hold a great deal of interest:
…though the I/O shield is surprisingly nice for a server board with the kind of padding you usually see on the nicer desktop boards:
Apart from the I/O shield, the only other thing you get in the box is a handful of non-latching SATA cables:
So why would anyone want this board? Huge PCI-E and RAM expandability with 18 SATA ports available natively, quad gigabit ethernet onboard and built-in IPMI combined with the choice of anywhere between 4 and 18 CPU cores makes for an extremely versatile package. Without a single add-in RAID or HBA card you can almost fill a 4U storage chassis like a Norco 4224, and the onboard quad gigabit ethernet means that you don’t need to use up a PCI-E slot to get those extra network ports. For most storage enthusiasts 4x1Gb is still more useful than 10GBe, as far as we have seen – not many people are making the jump to 10Gb for their home or small business yet due to the cost of switches and NICs. SAS 12gb/s is a nice nod towards the onboard controller remaining relevant once 12gb/s SSDs are more commonplace. The 4-pin fan support is excellent – certainly enough for a server chassis with a fan wall, two CPU fans and rear fans – and it should also be enough for almost any tower case setup.
This motherboard has potentially the recipe to be the perfect high-end home fileserver/virtualization host motherboard. Advantages over a socket 1150 setup include many more PCI-E lanes (and no PCI-E switches), twice as many RAM slots and CPU options with 6+ cores. The RAM in particular is very useful for all-in-one home servers which do file-sharing duties – often caching in RAM like ZFS’s ARC – and virtualization in the one box, meaning that the 32GB RAM limit of Socket 1150 can be reached quite quickly. There are also CPU options which are comparable to S1150 Xeons for around the same money, so if you don’t need 6 cores or more you don’t have to pay a premium for the CPU. The DDR4 RAM, on the other hand… prices are currently extremely varied, given it’s newly-released status – we have seen 16GB 2133MHz sticks go for as little as ~$150US and as much as $400US, so if you are in the market it will pay to keep a close eye on who is selling for those low prices at the moment.
As far as the Haswell-EP platform as a whole goes, it has impressively low power consumption – meaning lower power bills and less noise and effort to keep it cool – and the move to DDR4 helps, as it runs at 1.2V rather than 1.35V+. USB 3.0 and the huge number of PCI Express lanes really makes the platform applicable to a huge variety of server and workstation environments. Our take on the Supermicro X10SRH-CLN4F – it’s versatile, expandable and rock solid. If you’re building your own home or business server and wanted the platform to still be relevant in 5 years this would be a promising choice of motherboard to get you there.
Update 2014-02: Seems the prices have gone up a little… here’s the cheapest we can find currently: