Tuesday, December 3, 2013

Setting up the CGRU renderfarm manager

Alright, here's a step-by-step sort-of tutorial on how to setup the awesome CGRU renderfarm management system.
First of all, a little introduction. CGRU is an open source renderfarm management solution that supports various software, namely the following:

  • Blender
  • Maya
  • Houdini
  • Nuke
  • NukeX
  • 3ds max
  • Softimage
  • After Effects
  • Cinema 4D
My main reason to try it out was the Blender support (because the default Netrender addon is a mayor PITA honestly), so I cannot say anything about how it works with other software so far - but it works like a charm with Blender! 
Once set up, it's pretty self-explaining, and the running system looks like this: 

CGRU in action
But how do we get to this?
It's pretty straight-forward. Other render managers often require you to have a running Apache/mySQL/php/whatever similar. With CGRU it's as simple as copying a folder and set it up with a few clicks. So let's get started! 
Disclaimer: I have a Windows-only renderfarm here, so I'll describe the steps for Windows only. Also there might be a way to do some sort of centralized setup instead of doing it on every single machine. But since my renderfarm only consists of 10 render machines, it was easier doing it this way.

CGRU can be downloaded here: http://cgru.info/#downloads.html

After unzipping the downloaded file, copy the extracted folder onto every machine you want to render with. So much for the installation, now the setup (you'll have to do the following steps on every machine).

Open the folder and execute the "start.cmd" file. After doing this, you should have a new CGRU icon in your systray:
The popup menu of the systray icon
Right-click on the symbol and navigate to Software - Setup Soft - Blender (or any other software you might wanna setup). In the newly opened window, navigate to your Blender directory and choose "blender.exe".

Quick tip: I copied the blender folder to a mapped network drive accessible by every machine in the network, so updating Blender is a matter of just replacing that one folder instead of having to do it on every machine.

Okay, we're almost good to go. You have to decide which one of your machines will act as a master - take note of its IP. Now (for every slave), right-click again on the systray icon and navigate to AFANASY - Set Server... Paste the masters IP in the popup window.

That's it for the setup. To start rendering, navigate to the CGRU folder you downloaded, open the "start"-folder, followed by the "AFANASY"-folder.
  • on the master, start the "_afserver.cmd"
  • on every slave, start "render.cmd"
  • on your workstation, start the "start.cmd" in the downloaded directorys root
You can also start render.cmd on the master, so it will also render along. Now you can either place shortcuts to the respective .cmd-files on the desktop of every machine for easier access or add them to the Autostart folder, so the services will be started at boot time.

That's basically it, if done correctly you should now have a working renderfarm! Adding new jobs can be done either via a Blender addon which is included or via the systray icon. Job- and rendernode management is done by systray icon menu - AFANASY - Start Watch. I won't go into detail on that, because the documentation over at www.cgru.info is quite good. 

So I hope this made the first few steps on setting up your renderfarm a little easier. If you have questions, post in the comments or shoot me a tweet @tobkum :)

Thanks for reading, feedback is encouraged!

Monday, November 25, 2013

Renderfarm management

Just a quick heads-up because I have a feeling that too few people know about this (and everybody complains about the native network renderer in blender - including me ;)):

There is a great open source renderfarm manager available at www.cgru.info. We started using it at our studio (www.breitbild-vfx.com) a few weeks ago. Works flawlessly. Aside from Blender, it also supports After Effects, Houdini, Max and some more.
Setup is hassle-free, and you don't need any webserver, database or similar stuff running. Current version is 1.7, a major overhaul (2.0) is currently in the making. Load-balancing, rendernode management, job queue, priorities, job dependencies, you name it - it's all there.

If need be, just poke me here in the comments or over on Twitter (@tobkum), and I'll do a step-by-step setup description.