I’ve had a chance to cooperate with James Hindley from FPA Architects on a pretty comprehensive article on Linear Workflow in 3D. Go check it out!

I’m trhilled to be able to finally showcase, at least, some of our very own Python implementation into 3ds Max!

First off, our primary reason for writing our own, proprietary, Python connection to 3ds Max is Tactic by Southpaw Technology. An awesome asset management system entirely written in Python that I decided to invest in and integrate our tools and software packages into. Another reason for this connection, later came up, was the need for writing much more complex scripts with complex GUIs, since, as you probably know, a few functional lines of code are hardly enough in a modern, efficient, VFX production of today.

The heart of our Python integration is dotNET from Microsoft. I can’t express myself enough how much I appretiate this framework! The brain of our Python integration is IronPython. Also a product from Microsoft, completely open source and free, which are two very important aspects for any pipeline tool in any production facility of any size. Not the price as much as the availability of the software. And with IronPython and Microsoft, I am certain that this piece of software will be around for years!

So, what have we done?

• We’ve developed a bridge that connects MAXScript and Python via dotNET.
• The bridge converts most of native Python and MAXScript data types to and from one another.
• We’ve implemented several, custom, methods in Python that allow us to directly communicate with 3ds Max via MAXScript.
• We’ve also implemented error reporting so that when an error is thrown in Python, it’ll return the exception in MAXScript for debuging.
• Everything is based on dotNET and written in C#.
• The best part? Our Python bridge is completely platform and 3ds Max version independent!

Let me talk about the custom Python methods first. We’ve incorporated a class called mxsCon, that stores several methods for direct communication with 3ds Max as well as a variable
that stores passed arguments from MAXScript to Python for greater flexibility, especially when executing short code snippets.

The initial design meant to provide a way of directly running Python code from within MAXScript. However, with more complex tools needed nowdays, this proved to be insufficient and thus we’ve developed a way of executing more complex Python scripts written externally using IronPython. This works perfectly well if you plan on using 3ds Max’s interpreter and just want to pass values around MAXScript and Python. This is perfectly efficitent and enough for writing really complex tools based on Python. In our case, it’s the connection to Tactic.

However, later, it occured to me that since we have most of the hard work done, why not bring everything over to the Python side. Since Python is such a great and powerful language, why bother with MAXScript! So we developed another method for executing MAXScript code directly within Python scripts and have the result, evaluated in MAXScript, be passed back to Python. IronPython that is. Since all the value conversions had already been done, this proved to be an extremely powerful and efficient way of writing very complex tools, now even, with very complex GUIs. Since everything is based on dotNET, we can utilise IronPython and some of the IDEs available for super quick GUI development. I used SharpDevelop, an open source IDE for writing dotNET applications, for all my GUI and code stuff. Then I just hook up MAXScript commands that I pass Python values to in order to perform some actions in 3ds Max, like opening a file from our server stored in Tactic. Or, create objects and list them in a ListView. Or even write a direct 3ds Max – Maya translator! Why not?

Anyways, here are the methods and their purpose in 3ds Max:

• Python.run( <string> ) — will execute a Python script wrapped in double quotes (effectively a string).

Example:

Python.run
(
	"import sys
	mxsCon.setResult( sys.version )"
)

In this case, Python.run() will return whatever the mxsCon.setResult() will pass to it (more on that later).

• Python.runFromFile( <string> ) [ args:#() ] — will execute a Python script stored in an external file and alternatively pass it custom arguments.

Example:

Python.runFromFile @"C:\myPythonScript.py" args:#( 10., true, #(1,2,3), "some string" )

In Python, you’ll have to catch the passed arguments in order to use them, however the conversion of the data types will be done automatically for you.

• Python.inspect( <data type> ) — this method will inspect what data type you passed to it and how it will convert it to Python.
• Python.getError() — this method returns the last unhandled exception in Python code.
• Python.setVerbosity( <int> ) — allows you to set the level of verbosity for error reporting.
• Python.getVerbosity() — retreives the level of verbosity you currently use.
• Python.resetPy() — a very useful method for resetting our bridge residing in memory in case you’ve changed something radical in the source code etc…

And here are the custom methods developed in C# for IronPython use in the scripts. These only exist when run via our Pythoner.dll module. If you run the Python scripts without Pythoner, you’ll have to handle their lack of presence. I’ve done this via a custom class that I import in the scripts I use. This is extremely flexible since when I run the code outside of 3ds Max, I can still use its functionality without having to deal with error handlers all over the place.

• mxsCon.setResult( <data> ) # this method will return whatever it encapsulates directly to 3ds Max as well as in Python should you need to work with it later in the script.

Example:

import math

def myFunc( arg ):
	sum = arg + 100.
	return mxsCon.setResult( [arg, sum] )

if __name__ == '<module>':
	myFunc( math.pi )

The myFunc function will not only return a List of whatever you pass to it as well as the result of addition, but it’ll also pass this List directly to MAXScript for later use in 3ds Max.

• mxsCon.getResult() # this method will only return the last value stored in memory using the .setResult() method. Useful for later usage in Python.
• mxsCon.convert( <data> ) # this method will convert the input data types from Python to MAXScript compatible data types, but will return then wrapped as a string. This
  may be useful for other usage reasons. I used it for debugging or printing values in Python in MAXScript format.
• mxsCon.execute( <data> ) # here’s where the magic happens. This method is quite complex. It not only evaluates whatever is passed to it in a String format, but it’ll also wait for the result from MAXScript and return it within Python. Another, very cool, feature of this method is similar to String formatting capabilities known from Python.

Some examples of the mxsCon.execute() method:

# assume here's a ton of complex Python code that invokes this method
returnedVal = mxsCon.execute( r'Box()' )
# will create a box in 3ds Max and return OK (converted to True) back to Python.

returnedVal = mxsCon.execute( r'getDir {0}', r'#scripts' )

The .execute method will first evaluate its input into a single string,
in this case: r’getDir #scripts’, in a similar fashion the .format method works in Python.
Then it’ll execute the command in MAXScript and retun its output back to Python.
The output, in this case, will be a string: ‘C:\Program Files\Autodesk\3ds Max 2009\scripts’

But, enought with the talk, see it for yourself in action in this short Windows.Forms demo. (clicking the image will take you to the video)

I’ve finally received all the parts for my first render slave and immediately got into assembling and installing it. The rig is quite simple, very mainstream and very affordable. A standard ATX board placed in a 1U rack chassis, a Core i7 CPU, 8GB of RAM, two drives, one primary for the system and software and the other for buffering stuff, no DVD drive, no GPU, 350W power supply and that’s it. All running 64bit Windows 7 Professional.

I was kinda curious how this baby would perform once installed in the rack and how it’ll compare to my workstation, which is also a Core i7, 12GB of RAM, tons of HDD space (about 4TB), GeForce GTX285, some BD and DVD drives and other stuff all crammed on a standard ATX main board.

Here’s a quick video of the setup and of the first benchmark, for which I used the latest 64bit FryBench renderer. Quite surprisingly, the render node performed a bit faster than my workstation (might be due to the fresh install, though). It rendered the glass scene in 00h:07m:12s.

Music © Frank Klepacki

My workstation finished the scene in 00h:07m:29s.

I’m quite pleased with not only the speed of the render node, but also how cool it is. The temperature never went above 70°C, which is very, very good! Especially in a crammed rack setup. But obviously, the more nodes in the rack, the more heat they’ll produce and I’ll certainly have to deal with that later, especially during summer.

The entire studio ethernet ran on a standard 100Mbps line, 100BASE-T Fast Ethernet, which used to be enough. However, with the addition of a render farm and a fast centralized storage, I needed an upgrade. Thankfully, nowdays, a Gigabit Ethernet is becoming pretty mainstream as well, so I didn’t really have to put too much money into the whole network. All the standard, mainstream, main boards come with a 10/100/1000Mbps network cards integrated, the 1000BASE-T switches and routers are also pretty cheap, so all I really needed was a new switch, a bunch of CAT6 cables (however, CAT5e would have been enough as well) and a bit of re-wiring. The new network topology can be seen at the top.

Here are a few shots of the setup when I was working on it (I have a habbit of documenting a lot of stuff, when I come to think about it. ).

Anyways, we’re on a 1Gbps line all across the studio now and when I tested to copy a 14GB file from one PC to another the average data transfer speed was about 35MB/s, which took about 7 minutes or so. Not too bad. Obviously transferring tons of tiny files will always be an issue, but it should help with the larger ones, such as the scenes, source textures, renders, point caches etc…

Hell yeah! After a few months of putting the gear together, getting all the paperwork done, installing the electrical and network cabling, we finally run our own data management platform with an added bonus of a private render farm. How cool is that?!

I finally started building the render slaves for my studio. The first dedicated render node I built is based on basic mainstream parts, nothing fancy, but with enough power so that the render node does make sense to be placed in a rack installation.

The basic idea, obviously, was to build as powerful a machine, as possible for the lowest price tag, as possible. Since I’ve been an Intel user since, well forever, I based the machine on a Core i7 860 (Lynnfield) CPU, DDR3 memory and the rest is pretty much optional. But for my purposes, I wan every machine in the studio, to basically follow this idea of having a dedicated hard-drive, preferrably pretty fast, for the OS and a dedicated one for all the offline data. So, each machine, including the render nodes, will host a C: drive with all the software and programs on and a D: drive that’ll be setup to support all the files that we’ll work with. The workstation will have some other HDDs optionally, but these two drives are neccesary in order to rule out variables in the pipeline I’ve been building for a few months now.

Anyways, the render node’s config is a Core i7 860 (Lynnfield) CPU, 8GB of DDR3 RAM (I couldn’t get 16GB, 4×4GB, sticks unfortunately, but the mainboard supports them), Corsair I believe, a Gigabyte mainboard (H55M-UD2H), Western Digital hard drives, one 10krpm 74GB VelociRaptor and the other is the same as I have in the PowerVault MD1000 DAS, a 500GB RE3 HDD. The case is a Chieftec 1U chassis. That’s about it. No graphics card, no nothing.

I’m still waiting for the 1U coolers, which will be Dynatron K199, hopefully enough to cool down the CPU in such a tiny space. And a few more misc bits, like a rack shelves or flexible PCI-e riser cards that’ll allow me to temporarily place a backup graphics card in the case for installation, which’ll be removed from the system after that.

The OS of choice is, none other, than Windows 7 Professional x64. I have several reasons for this is:

1. Win 7 is pretty fast
2. It’s modern and flexible, meaning I can further enhance the performance by disabling tons of stuff and it’s gonna support the latest HW
3. Win 7 will communicate better with the Windows 2008 Server I have running my entire pipeline tools
4. And NO! I can’t run Linux on any of my machines!

I’m planning on experimenting with AMD CPUs as well, to see which ones perform best in the rack installation. The plan is to buy a new render node each bi-month, so by the end of the year, I should have at least 6 of these installed and online in the rack. For now, however, I’ll have to wait for the last parts to arrive from the US and then perform some benchmarking. I’ll keep you updated, of course!

Anyways, the server room is finally plugged to the local central switch. It is not online yet, T-Systems will have to go through yet another buerocratic procedure prior to setting up the line. But, the hard work is done and all I need now is an electricity revision and the T-Systems green light.

I took some shots with my phone this time as I forgot my camera home. So, please, excuse the terrible quality.

We finally got to the first stage of installing and prepping the server room this weekend. The first stage was to get electricity to the computers. The problem with this is, firstly, the power drain, then the network connection. Unfortunately, I didn’t have any other choice but to place the server with the DAS and the render slaves in a storage room, located on the ground floor in the building my studio is placed in (former flat). The room is great since the server and the running machines don’t bother anybody, but, it’s not properly air conditioned, it didn’t have any electricity power (except for the light) and it wasn’t connected or even remotely being able to be connected to the LAN switch.

I’ll deal with the air conditioning later, since it’s pretty damn cold here nowdays (below 5 degrees Celsius) and thus the temperature will be fine up til about April or so. But power was the first major issue. So this was the first step I had to take care of. I filed tons of paperwork, dealt with the local bureaus and finally hired the electricity gurus (pictured) to connect the storage room to my flat fuseboard. This presented several challenges. The first one was the power input. It had to be upgraded so that the fuses would sustain the higher power drain from the computers (both in the storage room and in the studio). Then we had to figure out how to connect the fuseboard, located about 20 meters behind three walls, to the storage room. The last bit will, yet again, be all about paperwork and bureaucracy. I’ll have to deal with that this week.

The second step will involve T-Mobile and their T-Systems (its subsidiary company, our internet provider) to connect the server’s switch to the local LAN switch, that is located in the underground level (the garage), also about 20 meters away. That’ll be another story and I still haven’t received the date the installation will be done, yet. But I’ll cover that as well, don’t worry.

Anyways, I thought some might be interested in this whole process of “putting stuff together to run a small studio”, so I took quite a few photos during this first installation. All this took about 9 hours of continuous labor, by the way. And for those who understand electricity, the fuse box we had to install in the fuseboard was a 32 amp (upgraded from the standard 25 amp), 3 phases, 240V for the rack (it should sustain about 30KW take-off), or so I was told.

The fuseboard was alive! I wouldn’t even dare to undo the cover, let alone reconnect the cabling! Anyways, as I’ve said, this is only the first of several steps. I’m still missing a special type of a socket outlet in the server room for the rack, that should come in about a week or so. Then a complete electrical revision. Then some more paperwork. Then the LAN switch interconnection. And that should be it. We should be able to access the server and its data storage and the render slaves from the studio, 5 storeys away. Hopefully all will be done in two weeks, tops. I’ll keep you updated.

That’s right! I needed to convert a fairly complex, proprietary, LUT for preview purposes to one of the IRIDAS’ formats. I chose .ilut, since it’s a really simple, yet, extremely powerful format!

Let’s take a look at the syntax first. On the online documentation IRIDAS site, you’ll see all the different LUT formats FrameCycler supports, as well as the syntax for those files. The .ilut format is pretty flexible. It supports either an XML (ASCII) syntax, or an inrerpretable script format. For my complex LUT I used the XML format. The LUT wouldn’t be of much use to you as it’s used on a specific machine with a specific graphics card, specifically calibrated display and generally a different color workflow than the rest of the machines at the studio. But, to demonstrate the usefulness of this LUT file format, I’ll show you my sRGB LUT I created a while ago using the interpretable script syntax for previewing linear images (OpenEXR for example).

But first, let’s take a look at the math behind the sRGB gamma curve. Basically, the sRGB standard specifies that whenever the input value is less or equal to 0.00304, the output value will equal to the input value multiplied by 12.92. In all other cases, the output value will utilise a slightly more complex formula, a 1.055 times the input value on the power of 0.416 (it’s a periodical float value, thus it’s being expressed as a fraction 1.0/2.4) subtracting the value of 0.055. Quite simple.

Here’s the expression in IRIDAS’ .ilut compatible format:

<IRIDAScript>

//sRGB LUT script

out = 0;

if (in <= 0.00304)
{
	out = in*12.92;
}
else
	out = (1.055 * pow(in, (1.0/2.4)) - 0.055);

red = out;
green = out;
blue = out;

</IRIDAScript>

As the documentation suggests, all the user variables should be pre-defined prior to being used. The out variable is my own variable that holds the output. The in variable is, I assume, a built in variable that holds the input value. The formula is simlpe, as explained above. Here’s the same formula written in Python, for those who struggle with the .ilut syntax (even though I consider it trivial):

output = 0.
input = .5 # in reality this will go through the whole interval of <0.0, 1.0>

def sRGB(input):
	if input <= .00304:
		output = input * 12.92
	else:
		output = 1.055 * pow(input, (1.0/2.4)) - 0.055

	return output

output = sRGB(input)

All you need is simply to save the .ilut script above to a file with the .ilut extension, save it in the LUTs folder within the FrameCycler folder, restart FrameCycler and you're done. You can then choose the LUT from the Calibration drop down list (lower right corner) to use the LUT on your Linear sequences. This will then calculate the pixel values (I think on a GPU, not sure though) for each frame and display the image in a correct sRGB format on your standard monitor.

As a side note, the huge benefit of using the script syntax as opposed to the XML format is that the values are being dynamically calculated, so, you're not constrained by the LUT precision.

I bought an aging file server, HP ProLiant ML350 G-5, and I want to transform it into a modern workstation that’ll be used for compositing tasks (Nuke, Photoshop and maybe After Effects). However, I’ve ran into some issues. Mainly, the server doesn’t have a dedicated PCI-Express 16x slot used for modern graphics cards, it only sports 3 PCI-e 4x slots, which sucks (how much it suck I’ll know after some benchmarks). The server also has only one CPU, a quite old and slow one. And, only 2GB of DDR2 RAM. The RAM is the least of an issue, the CPUs are quite expensive to upgrade, so I’ll keep the one already in there for the time being. But the GPU is the bummer! I’ve documented an approx. 15 minutes long video of “hacking” a regular graphics card in the PCI-e 4x (actually of the 8x size) slot, for those curious ones out there (I sure was!)