First mobo to support 16 GPUs for mining/crunching

Having problems installing that new stick of memory? Found some great software or having issues with something? Or maybe want to chat about your PlayStation, X-Box, Nintendo, Sega, even your old Spectrum 48k....! Or maybe something you want to sell or acquire (computing related of course!). Let us know here...
Post Reply
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

First mobo to support 16 GPUs for mining/crunching

Post by chriscambridge »

The ASUS B250 Mining Expert uses 3x 24-pin power connectors, 3x Molex PCIE power connectors, and supports up to 16 GPUs!

Not available in the UK just yet but coming soon.

Image
UBT - Timbo
UBT Forum Admin
Posts: 9673
Joined: Mon Mar 13, 2006 12:00 am
Location: NW Midlands
Contact:

Re: First mobo to support 16 GPUs for mining/crunching

Post by UBT - Timbo »

Hi Chris

What !! 3x PSU connectors...? That is going to be absolutely awesome....assuming there is some software that will be able to cope with 16x GPU's... ;-)

So, a good few bucks for the mobo, even more for 3x PSU's and then even more again for all the GPU's....

I guess payback time (based on Bitcoin or CRC mining) will probably be a number of years....

regards
Tim
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: First mobo to support 16 GPUs for mining/crunching

Post by chriscambridge »

It is my understanding that recent version of Linux and Windows can handle 16 GPUs..

You would also need some mighty GPU rack, probably the length of a table, in order to house 16 GPUs and 3 PSUs; also it would have to be bespoke design because Riser USB cables only work if they are quite short.

It would look pretty good though :)

I need one; that is definitely going on my 'what I would buy if and when I win the lottery list'.. :?

What ASUS are saying is because Miners/Crunchers keep buying all the Latest GPUs and even mobos, they are now producing bespoke mining stuff to ease pressure for Gamers lol; that is what they said on Facebook.
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: First mobo to support 16 GPUs for mining/crunching

Post by chriscambridge »

There does seem to be a serious lack of fan headers though on this mobo; I only see one, a CPU fan header?!
UBT - Timbo
UBT Forum Admin
Posts: 9673
Joined: Mon Mar 13, 2006 12:00 am
Location: NW Midlands
Contact:

Re: First mobo to support 16 GPUs for mining/crunching

Post by UBT - Timbo »

chriscambridge wrote:There does seem to be a serious lack of fan headers though on this mobo; I only see one, a CPU fan header?!
Hi Chris

There are 3x 4pin MOLEX connectors at the bottom left of the mobo - so, they could be used for fans...

OTOH, if the slots are going to be used for header cables, (to connect to off-board GPU's) then maybe the mobo won't need a lot of cooling...esp if it's the GPU's that are doing all the work !!

A GPU rack would certainly be worthwhile :-)

regards
Tim
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: First mobo to support 16 GPUs for mining/crunching

Post by chriscambridge »

True, I guess you would not really need fans, and if you did you could just use (PSU) molex/sata connectors to fan connectors. The molex on the mobo, if the same as other mining mobos, are for additional PCIE power so you don't burn out the mobo trying to draw too much power through it..
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: First mobo to support 16 GPUs for mining/crunching

Post by Woodles »

(I'm sure I commented on this last week?!)

Hi both,

Looks like 18 PCIe x 1 sockets and a PCIe x 16 to me, surely that's a possible 19 GPUs?

The CPU will provide 16 PCIe lanes and the chipset a further 12 so that wouldn't be enough but I think I see a switch chip in the middle of the sockets? That should provide enough.

With 16 GPUs you're going to need three PSUs anyway. With the connectors on the board it saves having to link them externally.

I believe that Windows 10 can "only" support 8 physical cards but they could be dual GPU cards? The limitation is in the PCIe address space. There's 2GB available but each GPU takes 256MB. I imagine Linux could be modified to accept more.

RE. fan headers, there seems to be an additional one next to the memory modules but other then the CPU, fan headers normally drive case fans and with 16 GPUs in the case I expect they'll be mains powered!

Mark
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: First mobo to support 16 GPUs for mining/crunching

Post by chriscambridge »

It is Asus who are saying just 16 so I guess they have limited the board to just this number.

I think I read somewhere that Windows can now handle more now, but I could be wrong. I would be surprised if the mobo only worked with Linux as it has only a small portion of the OS market.

I like the board though, clearly you have could have a pretty serous GPU setup on it.. (given the points above).
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: First mobo to support 16 GPUs for mining/crunching

Post by Woodles »

Hi Chris,

Not doubting you but it seems pointless to put 18 sockets on the board if you could only use 16 of them? It's not as if some might be blocked by GPUs as it looks like everything needs to be on extenders.

Ah, looking at the press releases, the problem is the drivers from AMD and Nvidia will only support eight GPU each so at present, the best you could do is eight AMD and eight Nvidia cards together. A new driver is due from AMD in 'late 2017' which will allow the full nineteen GPUs to be used.

Linux may have a small share of the general OS market but it has a much bigger share of the mining market and this motherboard is aimed squarely at the mining market. x1 PCIe slots wouldn't work very well for gamers neither would they need sixteen GPUs!

The reset vector for Intel processors is just before the 4 gig point in memory so the BIOS must be below here. To allow 32 bit DMA transfers (still the standard) between peripherals and the system memory, these peripherals address space must also be mapped into the bottom 4 gig of addresses. PCIe GPU drivers usually claim 256 MB of address space so theoretically you can have 15 GPUs in the system but once you've allowed for the BIOS and onboard devices you're limited to a lot less. Eight is the accepted maximum for Windows, I don't know if that's hard coded or whether it's slightly flexible. Linux kernels tend not to limit systems to 32 bit DMA transfers so usually a larger number of peripherals can be accommodated.

Specialised BIOSes or drivers will allow extra GPUs but with limited address space. I assume that's what AMD are doing for their improved driver. I expect Nvidia to follow suit.

Mark
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: First mobo to support 16 GPUs for mining/crunching

Post by chriscambridge »

I understand, that's pretty interesting.

I had not realized that Windows had those kind of limitations.

There is a new version of Windows about to be released, Advanced Windows 10 for Workstations, so perhaps Microsoft is trying to addresses these are other limited factors relative to Linux.

I still hate Linux though, lol :?
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: First mobo to support 16 GPUs for mining/crunching

Post by Woodles »

The 4G limit is pretty fixed for now until 64 bit DMA controllers become standard. The grabbing of 256MB for each GPU is the real limiting factor. No matter how you do the sums, 4GB split into 256MB chunks is never going to be more than 16. The drivers need to be persuaded to claim less memory not Windows developed.

When they do get the drivers sorted, how about one of these motherboards with 19 Titan Zs? Over 100k cores and 38 GPUs in one host! :D Granted you'd need over 7kW to power them all but there are three PSU connectors :lol:

Linux is alright, it's just that it's mainly still a command line interface so you need to remember and type commands and parameters rather than clicking on an icon and selecting items from lists or explorer windows.

Mark
Post Reply