Dual xeons

Having problems installing that new stick of memory? Found some great software or having issues with something? Or maybe want to chat about your PlayStation, X-Box, Nintendo, Sega, even your old Spectrum 48k....! Or maybe something you want to sell or acquire (computing related of course!). Let us know here...
Post Reply
bigsinky
UBT Contributor
Posts: 453
Joined: Wed Mar 28, 2018 8:27 pm

Dual xeons

Post by bigsinky »

just how hard is it to get dual xeons up and running. seen a matched pair of e5 - 2670 v3 with 12c24t for 349. A supermicro dual xeon motherboard at

https://www.scan.co.uk/products/supermi ... -lan-e-atx

it has PCI x 16 slots so will it take Nvidia cards? Also the board seems to only run windows server. Is this a minefield and a complete ball ache to do. i don't want to be having to update bioses and getting special drivers for video cards or god forbid go down the Linux route.

Advice guys please. i know chris did something on dual xeons but seemed to be a lot of faffing around. or i could jut wait on 3990x with 64 cores and 128 threads. i have a good kidney i could sell.
Image
UBT - Timbo
UBT Forum Admin
Posts: 9673
Joined: Mon Mar 13, 2006 12:00 am
Location: NW Midlands
Contact:

Re: Dual xeons

Post by UBT - Timbo »

Hi Neil

Chris C, Mark and I all bought some ex-Google dual Xeon Intel serverboards from the USA a while back.

Some of these had old BIOS's that caused some issues, though the Intel website had plenty of OS-specific drivers for the onboard hardware. (Updating the BIOS might not be worth the risk of ending up with a bricked mobo).

I know Chris went into a lot of depth over what these specific boards could/couldn't do...and in simple terms they can be made into mini-CPU crunching farms...with 32 cores, they can churn out some pretty good numbers and they are not that high requirement in terms of - I think Mark mentioned you could run them off a 500W ATX supply...or you will need a bigger one if you want to use a NVidia/AMD PCIe GPU card.

I had one board fail on me, but I picked up a second one quite cheap and transferred the CPUs and RAM from the first onto the 2nd.

The other issue with the Intel board is that it is a massive size, so a standard ATX case won't fit said Intel mobo. So, check the dimensions of the board and see if it will fit a case you might have lying around.

I think the Windows server boards might be an issue if you need drivers for it or even want to update the OS to something else. Again Chris did some research on this so he might be able to fill in some details.

regards
Tim
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: Dual xeons

Post by Woodles »

Hi Neil,

It's easy to get dual Xeons up and running :D

What makes you think the board only runs Windows Server? Here https://www.supermicro.com/support/reso ... _list1.cfm lists it as compatible with 32 bit and 64 bit Windows 7, Windows 8, Windows 10 and many variants of Linux.

Google lists several reviews that use an Nvidia GPU. The highest spec I've seen is a Quadro K5000, a $2,500 card when it was released. They won't run SLI but are capable of running at least two GPUs at a time. The problem you'd have would be getting a sufficiently new BIOS capable of recognising the newer cards. Something from a few years back should work but I'd be surprised if any of the RTX cards for example would. Around the GTX 750 Ti time should be fine. Having said that, I ran GTX 970s (2014) on mine with a 2011 BIOS successfully by just downloading the normal drivers from Nvidia (It wasn't possible to update the BIOS as there were no intermediate stages available and the latest ones required a minimum starting point of 2015) Best bet would be to use if as a pure CPU cruncher with a low spec GPU just for monitoring it.

Intel also have a very good supply of old motherboard drivers, you should have no problems getting all the onboard peripherals working.

The motherboards that Tim, Chris and I got used dual Xeon 2970 V1 chips (8C/16T chips, 32 threads total) that ran all core turbo of 3 GHz. I measured the power draw when one was fully loaded as ~240W. The V3s have more cores but a slower clock and only a 5W TDP increase so should be similar. The limiting factor is getting a PSU with two of the eight pin EPS connectors (you need both plus the 24 pin one for a dual CPU setup. A single CPU setup will work with only one EPS connector plus the main 24 pin one) Minimum specification seems to be 650W although a lower rated PSU with modular cables might work (you'll need two EPS cables but EBay do them) DON'T USE MOLEX TO EPS CONVERTERS WITHOUT A FIRE EXTINGUISHER PRESENT! I think the GPU power supply requirements will be the limiting factor.

Tim has mentioned the size and a couple of reviews have mentioned that even though listed as EATX, some of the mountings are in non-standard positions. If you're handy with a drill then that's not a problem. I even managed to fit one into a midi ATX tower case short term.

My boards run with EVO 212 air coolers on them in pretty cramped conditions (although uncased) and they've happily ran flat out 24/7/365 for several years at ~65-70 degrees with no discernible fan noise.

Like Tim I also had one set up fail and refuse to boot. My problem though was the speed sensor on the CPU fan header failing so it wouldn't boot as it didn't think there was any cooling. Swapping fan headers got it working again.

Mark
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Dual xeons

Post by chriscambridge »

Personally I wouldnt go down the dual Xeon route as there as much quicker, cheaper, and more energy efficient CPUs available nowadays.

E.G Threadripper or Ryzen (*see Mark for info on these as I know he has looked into or bought)

You can pick up a 16c (32t) Threadripper (1950x) for £175

https://www.ebay.co.uk/itm/AMD-Ryzen-Th ... OSwWMld-Oq~

And a X399 TR mobo for around £250 or less

https://www.ebay.co.uk/sch/i.html?_from ... 4&_sacat=0

I would completely agree with Mark (and Tim) on most points, especially about air CPU coolers such as the EVO212; personally I have never had any CPUs get anywhere near thermal constraint using air coolers, even while running AVX tasks (which increase temps substantially); apart from the first dual Xeon box I built which required a FRD BIOS update.
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Dual xeons

Post by chriscambridge »

Just so everyone knows.. there is a brand new Ryzen 3900x (and CPU cooler) on offer for £429.99 on EBay UK.

Enter voucher code POPSALE for the 15% discount at checkout to redeem. Voucher expires 23/12/19.

OEM AMD Ryzen 9 3900X 3.8GHz 12-Core AM4 CPU with Wraith Prism Cooler

https://www.ebay.co.uk/itm/OEM-AMD-Ryze ... SwD6Vd4Ohh
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: Dual xeons

Post by Woodles »

Damn, I thought that was a 3990X for less than £500!

Instead of dashing straight off to buy it I'll take a look at lunchtime.

Thanks for letting us know Chris.
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Dual xeons

Post by chriscambridge »

No worries, I thought I remembered you had your eyes on the 3900x..

Just so you know the 15% voucher code also works on TechSave and Ebuyer via Ebay (up to £60 savings).
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: Dual xeons

Post by Woodles »

I'm going to be spec'ing out a Threadripper system early next year, nothing stopping me getting the CPU early though.
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Dual xeons

Post by chriscambridge »

If I am not mistaken, the 3900x is a non-threadripper (TR4) Ryzen? Just a normal ryzen (AM4) so to speak.

Its my understanding that these CPUs have pins on the processor, unlike TR and other 'normal' CPUs where the pins on in the CPU mobo socket.
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: Dual xeons

Post by Woodles »

:doh: So it is Chris, obviously I need to read up on the different AMD offerings.
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Dual xeons

Post by chriscambridge »

I think the only differences that I have noticed so far are:

* CPU socket
* Pins on CPU
* Far less PCIE lanes
* Dual channel memory
bigsinky
UBT Contributor
Posts: 453
Joined: Wed Mar 28, 2018 8:27 pm

Re: Dual xeons

Post by bigsinky »

Some of the TRX40 boards are XLATX size as well. The new GIGABYTE TRX40 AORUS XTREME is a whopping £900 and massive. Big case needed for that piece of kit. I think perhaps in retrospect the AMD route is simpler and more efficient than dual Xeon. The 32c64t system should be about £6k so i better start saving
Image
damienh
UBT Contributor
Posts: 1685
Joined: Mon Aug 26, 2019 2:16 pm

Re: Dual xeons

Post by damienh »

In theory we should see more details about the 64c/128t Threadripper 3990x at CES (January), although it might take quite some time to come out.

They have confirmed its existence, so just awaiting clock speeds, price, and release date. That one is rumoured to be $5,000 just for the CPU ...

The 32c/64t 3970x is great though. Under water it runs all cores happily at 4GHz+.

Another option is the previous-gen 2990x. It potentially has some memory architecture challenges, but those 32c CPUs are dropping for £850-£900ish on eBay at the moment. That would be a drop-in upgrade for my 1950x, so I’m tempted!
galacticminor
Active UBT Contributor
Posts: 13
Joined: Wed Nov 28, 2012 12:00 am

Re: Dual xeons

Post by galacticminor »

Theyre not hard to get up and running, but mileage may vary with motherboard supplier. Supermicro are one of the better board makers - you'll usually find theyre compatible with standard ATX power supplies, ATX cases and all the other usual bits and bobs, unlike makers like Dell/Fujitsu that tend to produce their own proprietary power supplies and connectors.

I usually stick Ubuntu Linux on them and just set them up as a remote desktop client. Windows 10 or so would be just as easy to set up, but need a little more faff with drivers. You'll be able to get those off the supermicro website though.

Dual xeons are quite nice - if you can get the lower wattage ones then you'll get less heat out of the case. I'm running a pair of L5640s (Hex core + HT) with a pair of Nvidia 980s in at the moment and its doing quite nicely.

I realise I'm in a bit late here, but do also consider dual Opterons as well - a AMD Opteron 6380 2.50GHz 16 Core will set you back about £24 each. 2 of them is 32 work units running at once :-).

regards

Andrew
Post Reply