duber’s render slaves

loocas | hardware,miscellaneous,opinions,technical | Saturday, January 23rd, 2010

duber render node

I finally started building the render slaves for my studio. The first dedicated render node I built is based on basic mainstream parts, nothing fancy, but with enough power so that the render node does make sense to be placed in a rack installation.

The basic idea, obviously, was to build as powerful a machine, as possible for the lowest price tag, as possible. Since I’ve been an Intel user since, well forever, I based the machine on a Core i7 860 (Lynnfield) CPU, DDR3 memory and the rest is pretty much optional. But for my purposes, I wan every machine in the studio, to basically follow this idea of having a dedicated hard-drive, preferrably pretty fast, for the OS and a dedicated one for all the offline data. So, each machine, including the render nodes, will host a C: drive with all the software and programs on and a D: drive that’ll be setup to support all the files that we’ll work with. The workstation will have some other HDDs optionally, but these two drives are neccesary in order to rule out variables in the pipeline I’ve been building for a few months now.

duber render node

Anyways, the render node’s config is a Core i7 860 (Lynnfield) CPU, 8GB of DDR3 RAM (I couldn’t get 16GB, 4x4GB, sticks unfortunately, but the mainboard supports them), Corsair I believe, a Gigabyte mainboard (H55M-UD2H), Western Digital hard drives, one 10krpm 74GB VelociRaptor and the other is the same as I have in the PowerVault MD1000 DAS, a 500GB RE3 HDD. The case is a Chieftec 1U chassis. That’s about it. No graphics card, no nothing.

duber render node

duber render node

I’m still waiting for the 1U coolers, which will be Dynatron K199, hopefully enough to cool down the CPU in such a tiny space. And a few more misc bits, like a rack shelves or flexible PCI-e riser cards that’ll allow me to temporarily place a backup graphics card in the case for installation, which’ll be removed from the system after that.

The OS of choice is, none other, than Windows 7 Professional x64. I have several reasons for this is:

  1. Win 7 is pretty fast
  2. It’s modern and flexible, meaning I can further enhance the performance by disabling tons of stuff and it’s gonna support the latest HW
  3. Win 7 will communicate better with the Windows 2008 Server I have running my entire pipeline tools
  4. And NO! I can’t run Linux on any of my machines!

duber render node

I’m planning on experimenting with AMD CPUs as well, to see which ones perform best in the rack installation. The plan is to buy a new render node each bi-month, so by the end of the year, I should have at least 6 of these installed and online in the rack. For now, however, I’ll have to wait for the last parts to arrive from the US and then perform some benchmarking. I’ll keep you updated, of course! ;)

26 Comments »

  1. Hi,

    whitch RS for 3Dsmax are you goi´n to use?

    Comment by Oltskul — January 24, 2010 @ 18:55

  2. If by RS you mean render manager, it’s gonna be Deadline from Prime Focus. A bit pricey, but extremely flexible.

    Comment by loocas — January 24, 2010 @ 19:06

  3. looc: yep,noup, jeah… I meant Render System…

    Comment by Oltskul — January 24, 2010 @ 22:01

  4. A Render System? What is that? A renderer? A render manager?

    Comment by loocas — January 24, 2010 @ 22:07

  5. ie. FinalRender

    Comment by Oltskul — January 25, 2010 @ 18:12

  6. finalRender, absolutely! Some jobs might require mental ray, but that’s a standard package. finalRender is the renderer of choice for us here. :)

    Comment by loocas — January 25, 2010 @ 18:19

  7. Hi loocas,
    I am building a server right now (but webserver) too and maybe I will inspire myself by your setup … is your setup stable after some first settings or would you choose some other components?

    thx for info
    igi

    Comment by Iguana007 — January 29, 2010 @ 03:37

  8. Heya igi,

    the setup has been stable so far and I don’t see any problems with it. The DAS is amazing and the older Dell machine as well. So far, so good. I can only recommend Dell for this kind of stuff. :)

    Comment by loocas — January 29, 2010 @ 10:32

  9. niiicceeee man! I’m happy to see you proceeding. and it seems as you have a good plan! I hope it will work out.

    did I understand that right…you want to have all your data on each machine locally? or is the D drive just a buffer space?

    I was just recently talking to some guys about exact that same issue.
    optimal is to have a central storage where all the files are. this way you can easily plug in a new disk.
    the problematic part is that all the render nodes start asking for the files at the same time. so the bottleneck are the harddisk reading speed of the server and the amount of ethernet connections on the server.
    I’m looking for a solution. do you have some special solution for that?

    Comment by goran — February 6, 2010 @ 19:58

  10. Heya, goran, dude! :)

    As for the storage issue, yes, as a matter of fact, I do have a solution for that. :) But it’s a little more complicated than just saying “buy fast storage drives”. It requires a more elaborate setup on all fronts. The D: drive on the render slaves is there for various reasons. First is a buffer space. Then it’s the disk drive mapping issue. And last, but not least, is the temporary storage for check-ins/outs.

    But this is for a bit longer discussion. I might write some more about this particular issue in the future.

    Comment by loocas — February 7, 2010 @ 00:09

  11. Hi, did you recieved your dynatron cooler finally?from which website,ebay?Iam interested how its working in 1U rack, if is it enough to cool it on long run? We had issue with few nodes in rack, becose in long run rendering it started to be overheating and than it loose efficiency. Your render nodes setup looking really good and well selected in terms of price. If it will be good i will build probably same machines like you.

    Comment by Milos — February 7, 2010 @ 23:51

  12. Yea, and Deadline is defintly best solution, i really like it.

    Comment by Milos — February 7, 2010 @ 23:52

  13. Hi, Milos,

    yeah, the Dynatron coolers are quite cool and pretty damn efficient as far as I can tell. I had the machine rendering a test scene overnight and the CPU temperature didn’t go above 65 Celsius degrees! Quite impressive.

    I bought the coolres on eBay, of course ;) and so far I’m very satisfied with them. But, I’m yet to really stress test the render node to really see how good it performs.

    Comment by loocas — February 7, 2010 @ 23:54

  14. Hi, how much is Tjunction of your CPU, 100? What iam afraid is that if you have 10 render nodes in one rack than they are starting to heat bit more to each other, that can couse really issue. Iam not trying to stress you, i like your solution and i will build to same if your will be fine. thnx

    Comment by milos — February 8, 2010 @ 09:44

  15. Heya, Milos,

    the TJunction was set to a 100, I believe, I didn’t change any of those defaults.

    As for the heat. No stress, I’m aware of that issue and I’ll be monitoring that once I have more render nodes at full utilization setup and running. Right now, it’s not an issue. The room is quite cold down there, but as soon as summer or maybe even spring comes, the temp might go up and I’ll be considering buying a dedicated Rack cooling solution. We’ll see. But right now, the heat doesn’t seem to be an issue.

    Comment by loocas — February 8, 2010 @ 11:07

  16. Yea, you are right. So happy rendering…

    Comment by milos — February 8, 2010 @ 12:38

  17. and thanks for sharing…

    Comment by milos — February 8, 2010 @ 12:47

  18. Hi
    I really wants this. But I just curious, why do you choose intel I7 not xion? Is it only for price?

    Comment by Shetu — May 5, 2010 @ 13:57

  19. Yes, primarily for the value/performance ratio. These render slaves are quite cheap to build. About $500 USD each and are very cheap to operate on a 300W power supply. The render power is quite good. Obviously not as good as a dual Xeon system, but again, the value/performance ratio plays a crucial role.

    Comment by loocas — May 5, 2010 @ 17:22

  20. Hey there, huge thanks for sharing this info, gonna make couple of render nodes also, OK, I’m reading this now )

    Comment by Farid — August 13, 2010 @ 11:24

  21. 500USD! whaou. That’s pretty cheap. That should be wihout OS because Win 7 by itself is already 150USD. I’m looking to for a way to run the 3dsmax render slave on linux to avoid this extra cost. But to date, I’ve not find any solution.
    Did you overclocked it ?

    Thanks for sharing.

    Comment by lolo — August 27, 2010 @ 14:32

  22. Hey, lolo,

    I didn’t overclock the machines, don’t have to. I also prefer stability over a slight increase in speed.

    As for a Linux render slave, I’m afraid you’re out of luck as 3ds Max won’t run on a Linux machine. Not without some heavy virtualization, which will drain a ton of performance out of the HW. Also, the VM software will cost quite a lot. The Windows OEM licenses are, relatively, cheap. I paid much more for rendering licenses than for the OS ;)

    Comment by loocas — August 28, 2010 @ 16:24

  23. hi im following your blog very close and it has been an inspiration for me to get my renderfarm and build my own studio here in medellin colombia. i have one question, do the renderslaves need more than 8 gigs of ram?
    is it true that renderslaves consume less memory than a workstation.

    thanks in advance for sharing all that knowledge with us!

    Comment by Juan David — May 13, 2011 @ 03:39

  24. Hi, Juan,

    well, generally, render nodes don’t need specific HW and SW for actual work as workstations do, so, for example, if you’re not rendering on GPUs, you don’t need a dedicated graphics card on render nodes. But I wouldn’t say that render nodes consume less memory. Perhaps a little bit, but not in general. As for 8GB of RAM, for the projects we do here, this is enough. But the boards do support up to 16GB, so, shall I need to upgrade, I’ll just buy more RAM and swap the sticks. :)

    Comment by loocas — May 13, 2011 @ 08:30

  25. I just find you page, very interresting. Do you system still run ? Do you add more render node since.
    Could you tell me if the node are loudly? I plan to build a few inhouse, but as they have to be in the same room, Noise Is important.

    Thanks.

    Comment by Olivier — February 23, 2012 @ 23:17

  26. Hi Olivier, yes, the nodes are extremely noisy, I certainly don’t recommend putting them in the same room where you work or even sleep. And since then I’ve added to a total of 10 render nodes, which is a maximum I’ll ever want for my render farm. Anything beyond that will have to be purchased outside. I don’t want to spend more money on hardware upgrades, maintenance and running costs in-house.

    Comment by loocas — March 3, 2012 @ 23:10

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress | Theme by Roy Tanck