UK BOINC Team Forum

Only one server rack so far in the cabinet, but hopefully I will be moving the other machines into this shortly.
The cabinet is 25U so that gives me enough space for 6 server racks in total.

Each server rack can take E-ATX, EEB, and CEB mobos; it also features, 2 front fans, 4 fans behind the GPUs/CPUs, and 2 back fans!

Hi Chris

That looks mega...hope you'll be able to afford the leccy bill, once that is full of mobo's/GPU's/PSU's etc.

Mind you, you won't need the heating to be on in the winter

Keep us posted on how this proceeds !!

regards
Tim

It can fit 6 racks, but I won't be running anywhere near that amount; I can barely afford what I am running now.

I decided on the server rack/cabinet route so I could constrain heat in a smaller room, and expel it more logically.

I will probably use the additional rack space for other stuff like PDUs, Switches, KVMs.

If I do any major updates I will try and take a photo and upload.

--

In truth, I have wanted a data rack cabinet since day one crunching, it just looks soooo cool!

2 new server racks and rails.

Very nice Chris, a lot more mechanical than most PC builds

Hi Mark,

Personally I think if you are running multiple high core machines with fairly powerful GPUs, in a house or flat, then it just makes complete sense to move them all into a dedicated room, and if you are short on space then to also move the machines into a server rack cabinet.

It is not just about being able to contain and remove the heat within the room, but also about all the noise from the fans. If you are only running a single box with a less powerful GPU then you would probably not understand this, but anyone that has quite a large setup would agree with these sentiments.

I can only imagine the heat and noise from your setup

Once it is all built and setup I will take a final photo and post it (and probably remove the initial ones).

Chris

Hi Chris

Just an observation

When I was younger, my soon-to-be step-father owned an apartment that had a warm-air central heating system. There was a gas-fired heater, which warmed the air and fed it through ducting in the walls - each room had a small grille+dust/particle filter at floor level, from which came the warm air.

So, with a bit of pipework and some DIY skills, I reckon you could feed the heated air from your server cabinet and run it around your house/flat !!

I would think there is a nice potential here for people to buy and install this sort of equipment - of course we wouldn't tell them about the crunching side of it and hence we can grab all of their credits in the process

regards
Tim

Hi Tim,

I actually have (cold air) input fans on the "data room" door (facing the data rack cab), and (hot air) output fans in the attic hatch which is also in the data room (above the rack cab). This is a DIY replication of the cold/hot air aisles you get in a data center; cold air in the front, hot air out the back (obviously).

In winter, I will just reverse the fans, which is basically the same idea as what you mentioned.

From day one I realized that there is no point moving the machines into an even smaller room unless you are going to control and remove or use the heat being produced. This does that nicely.

If I had the money I would use the 'liquid oil immersion' cooling system where you dunk the whole server rack (eg motherboard and GPUs etc) into flowing synthetic oil for cooling. The oil can then be used for other purposes such as heating water via a "ground source" heat pump, or heated further and pumped into radiators.



I think this is the future, and some many massive companies are now using this technique. Its definitely the way I would go, along with solar/wind arrays, if I were building/running a data center.

In case anyone is interested what these server racks have, look like inside, and offer..

The front has 3 sections; 3x HD bays on both the left and right (eg 6 bays). It also has 3x CD bays in the centre. And 2x fans, one behind each HD bays area. The front (dust) grill is also removable/droppable, which as you can see is what I have done. It also has two handles which in truth are very useful when moving the rack about, and or getting it in and out of the rack cabinet.



Inside, as you can see, is a horizontal 4 fan wall, 7 PCIE slots, and 2 back fans. As stated these racks fit ATX/E-ATX/EEB/CEB. They are standard 19 inch racks, and have a depth of 650 cm (ish). They also (just) fit full height GPUs and EVO 212 Coolers.

Hi Chris

WOW - not seen that before Impressive.

Liquid oil immersion...of course it goes without saying it must be non-conducting...but I wonder how long the servers could last in that sort of "environment". Clearly some clever "liquid engineering" must go into it, plus every part of the server would be cooled (chipsets, HDD, RAM) and not reliant on "force air" to circulate - though I would wonder how the CPU's and GPU's are cooled, as they would still be "hotspots" and unless the liquid oil was "forced" through, then there could be issues?

I know a few large "data centres" are being located in low temperature environments (Iceland and northern Scandinavia spring to mind) with the excess heat being "harnessed" and dispersed to local homes/business.

And it could very easily be run off DC volts (from solar panels) in an ideal world, which then reduces transmissions losses (over a distance from a suitably large power station) and then there's DC->AC->DC conversion losses too (from solar arrays feeding the National Grid).

Perhaps, one day, all PC's/laptops/games consoles/Sky boxes will have a dedicated DC power socket fed from a standardised solar array system bolted onto to any southern facing roof ?

regards
Tim

Hi Tim,

If I remember rightly, it was Intel, who had been testing this technology for the past few years to see if it had any impact on the hardware, and if it actually lowered costs due to better cooling. It is my understanding that this company has now moved to this method for a few of the data centers due to both of the above points proving to be positive.

If I remember rightly then the system removes all cooling fans from CPUs and GPUs etc, and let's the oil flowing do the heat transfer and cooling. I seem to remember them saying the only thing it did not work with was some moving parts such as HD's, however fans per se seem to work just fine and still spin in the oil, however obviously with no point due to the oil doing the cooling

3M sell this non-conductive, data-center immersive oil, so I think its now becoming quite widely used by the bigger companies due to it lowering cost, and because the heat gets put into a medium that makes it easier to reuse.

3M™ Novec™ 649 Engineered Fluid
"Heat Transfer, Immersion Cooling for Data Centers"
http://www.3m.com/3M/en_US/novec/produc ... 192&rt=rud

I think the future will be quite exciting for energy creation, distribution, and consumption. There are a lot of unique and clever ideas floating about and being researched. Many of which are connected with data centers due to the massive TDP and heat generation.

I do not really know much about AC/DC etc (apart from the obvious) but having a DC and/or AC plug on hardware surely would seem to make sense.

Finally the move/build is complete. All processing is now being done in the "data room" within the server rack cabinet.

However the room and area outside is tiny, so its quite hard to take a picture!

You can see a decent photo a few posts down from this one.

Hi Chris

Nice one...looking good.

I assume you made sure the server rack is on wheels...just so that in winter you can move it into another room, instead of putting on the central heating LOL !!

regards
Tim

No I will just reverse the cold air input and hot air output fans, on the door and ceiling.

Hi Chris,

Looking good

How does it perform noise/heat wise?

Mark

To be honest the cabinet and server racks look pretty bad in the photo above due to poor light and resolution, so I have removed it. This one is more representative of how good it actually looks!



The fans are little noisy, but not too bad once the "data room" door is shut. The temp's are excellent; the follow screenshots of GPU-Z and CoreTemp are running on a single Xeon CPU and 1080 GPU.

Hi Chris,

If I'd known when I started that I was going to end up with as many hosts as I have, then I would have gone down the rack mounted route as well. As it is, it's a little late to change now ... although you might have tempted me, those are very impressive temperatures for the CPU and GPUs All I need now is someone to buy a load of second hand computer cases, some of which 'may' have been modified slightly

Mark