As you might know there has been a significant change in the latest Shotgun API that’s somehow transparent to the CPython users, but presents a rather significant roadblock for IronPython users (including our duberPython bridge, that is based on the IronPython engine).

First, let’s discuss what’s changed in the API so dramatically that it breaks IronPython compatibility. It’s the introduction of a JSON formatting that requires a few specific CPython libraries that are not available in IronPython. The effect it has on CPython users is a faster data transfer to/from Shotgun, but other than that, the API looks to be unchanged from a user point of view. You still keep calling the same methods and you’re getting back the same objects. From IronPython point of view, you’ll hit a roadblock as there are a few modifications you’ll have to make to the Shotgun modules in order to make them run in IPy without issues.

With huge thanks to the Shotgun Software support, I was able to work around the few limitations and start using the latest API without issues inside IronPython. Here’s what you have to do, in order to make this happen:

• First and foremost, download the latest Shotgun API from the github site. Place it some place you’d like to use it from, most likely in the Lib\site-packages folder, though my folder structure is completely different and based on different rules.

Now if you try to instantiate the Shotgun class, you’ll get tons of errors, so, allow me to save you a few minutes (hours in my case, to be honest) searching and working around them.

• The first thing you have to overcome the “IDNA” encoding, which isn’t supported directly in IronPython. This isn’t as clean and easy as it could be, though the workaround is rather trivial, but bear in mind you might break something potentially doing this. I haven’t come across any issues yet, so, I’d consider this modification safe. Find the iri2uri.py module in the httplib2 folder, open it and change line 71 from authority = authority.encode(‘idna’) to authority = authority.encode(‘utf-8′). This effectively changes the codec under which you’ll be dealing with the unicode strings. Again, it’s not a clean modification and you can potentially run into issues doing this, but in my experience, all seems to have been working fine so far.

• The next problem will be SSL authentication, which, again, is tripping IronPython’s SSL module. To fix this, open the shotgun.py module and on line 73 change NO_SSL_VALIDATION = False to NO_SSL_VALIDATION = True.

• Then IronPython will still be complaining about a module called zlib, which is a compiled CPython module that cannot be used inside IronPython. Luckily there is a C# variant based on the same code as zlib. Download it from Jeff Hardy’s bitBucket site and place it somewhere where IronPython can source it (i.e. within your PATH). I put it in a recommended folder called DLLs directly in the IronPython folder. This means that the IPy interpreter will source the DLL automatically, but bear in mind you’ll have to add a reference to this library manually in your scripts when run outside the shell.

• The easiest way is to add these two lines in your imports into the shotgun.py module:

  import clr
  clr.AddReference("IronPython.Zlib")

This way you’ll make sure the zlib module required by some of the shotgun modules is going to be readily available.

That’s it! Let’s test it. Oh, one more thing, I’ve found these Python variant couples to be perfectly compatible and working in production: IronPython 2.6.2 and CPython 2.6 and for clr4 folks IronPython 2.7.1 and Python 2.7.2.

IronPython 2.7.1:

duberPython (utilizing IronPython engine 2.6.2):

I hope this helps you out with your IronPython and Shotgun scripting and I’m certainly looking forward to the new versions of the Shotgun API.

A huge thank you to the Shotgun Software support team!

Here’s a video demonstrating the power and practical usage of Shotgun (data) brought over to 3ds Max natively via our Python plugin, duberPython.

Every TD knows that command line tools are among the most powerful in their arsenal of tricks and secrets.

I want to mention RVIO, as today it saved me quite a lot of time (again), which is absolutely key when a deadline is coming.

My client requested a minor tweak of animation (a lip sync, to be concrete) on an almost finished shot. So, the general approach would be to do the change, have the animation data go through the pipeline and at the end have the finished frames ready to be loaded in an existing edit, which then gets rendered out and the final result gets showed to the client.

All fine, until you realize your render farm is completely full with other shots, so you have to skip the “beauty” pass rendering and only present the client with a, somehow, polished preview directly from your 3D package, which isn’t the safest way, trust me. But this client is great and understands that what he sees is actually only a preview of the animation.

So, the last piece of puzzle to solve is to get the preview assembled with additional layers of information (such as frame number, shot name etc…), basically a slightly customized overlay. All this sounds nice and simple, you just open up (in my case) Premiere Pro, swap the layers, render out the portion you need and be done with it.

But this certainly isn’t the TD way.

Fortunately for me, I’d purchased a license of RVIO some time ago, so I have an extremely powerful and versatile tool in my arsenal of VFX software that just gets the job done, no matter what.

The problem with the Premiere Pro approach, actually, there are two problems: 1) my lazyness and 2) the time it actually takes to get it done. Besides, this way you are also exposing your project to errors. You might accidently save the project and when you least need it, it’ll jump back at you and you’ll have to be dealing with this fuck-up in the least desirable time – the minute before the deadline.

So, the easiest approach for such a trivial task would be something that allows you to combine an image sequence with your sound file and spits out a movie file with the correct size, color, the whole package, basically.

RVIO is such a tool and it is aimed at TDs and people who aren’t afraid of using the command line (which you certainly should NOT be as it is the most powerful, fastest and actually easiest way of getting things done, even in 2011). It allows for both simple and extremely complex image, movie and sound file conversions. See the RVIO site for more info on that.

So, what is the concrete example I’m talking about? Something as simple as delivering a correct file and presenting it properly to the client can be a complex process in a large pipeline. In my case, as I’m a fairly small boutique, what I needed to do was to take an image sequence of the fixed animation, place it in an existing timeline (effectively replacing the previous, fully rendered, version), setting up markers for in-out points, setting up the export settings, correct file name, correct video and audio codecs, correct output size etc… Then have it exported, review the file, upload it to our server and present the link to our proprietary review site for clients to the client.

This entire process isn’t too difficult to manage in five to ten minutes, but presents a bit of a security threat that I mentioned above and also isn’t very flexible or comfortable. Sure, for people that can only deal with their software via nice and polished user interfaces, this is the only way to get the job done, but for a TD, there is a much smarter, faster and cooler way.

Here come the super-powers of RVIO. It’s strictly a command line tool. But to perform all (and many more!) tasks that I just mentioned, all you need is this line of code:

rvio [ "inputFrameSequence_12-125#.tif" "inputSoundFile.wav" ] -codec avc1
-outkeyinterval 20 -outdatarate 1000000 -outgamma 0.85 -outres 1024 576
-audiocodec aac -o "outputMovieFile.mp4"

That’s it! See how awesomely simple, fast, effective and cool this is?!

Now, this was just a command line command you execute directly and you get a outputMovieFile.mp4 in the correct format, the correct in and out of the animation, correct sound and everything between. However, imagine what you can do with this! If you know Python, or on Windows, preferrably, IronPython, you can do WONDERS!

I’ve tied RVIO with Shotgun via IronPython so that I can do very easily, effectively and super fast deliveries to our client on a project with about 120+ shots. It’s not a huge project, though, but the sheer number of shots definitely calls for scripting. There is no way I’d be sitting at 4am at the computer and manually editing and converting 120+ shots to a desired format for review whenever there is a change in the animation or compositing! That’d be a suicide and a complete waste of resources.

What I did was calling a loop on all the shot folders where our final comp files were being stored, picked the rendered files and fed them to RVIO with a few parameters set specifically for that project, similar to what I just showed you above, pulling a few variables from Shotgun, such as in and out cuts, shot name and other data. Then it automatically takes the resulting movie files and uploads them to our review system via FTP. That was it! With about fifty lines of code I achieved a task that’d take me at least three hours of intensive file hassling and messing around! All of this then takes about ten minutes to process, which is perfect for a coffee break.

Besides, it’s recycable, in a sense that when there is something changed in any of the shots and we need to re-deliver, I just run the script again.

This is what I love about my work! The effectivity, productivity and the cool factor, of course.

Just a quick shout about the compatibility of duberPython and the latest and greatest IronPython 2.7.1 release (released a couple of days ago). All working smoothly and quickly, as expected.

Should you need more info on duberPython or what we’re doing with it and Shotgun or Tactic, just drop me a line and I’ll be more than happy to show you how cool duberPython is.

If you’re interested in a live demo or more information about the duber Python and Shotgun integration in 3ds Max, drop me a line via e-mail at: ldubeda[at]duber.cz

Until recently, I’ve been handling Dictionaries two ways:

1. String parsing (i.e. a very primitive way of handling foreign data types, not really recommended)
2. Manual wrapping (i.e. passing in a List or Array object and converting it back to a Dictionary object)

String parsing is the worst possible way of handling such an issue. It’s very cumbersome, highly unintuitive and with MAXScript string capabilities, extremely difficult. Manual wrapping, on the other hand, is rather more elegant, faster and you can use other, known, data types to construct your future Dictionary and have that converted in Python natively. The down side is, you have to be very careful with the way you’re handling the future Dictionary. The thing is, only a tuple of list pairs can be successfully converted to a Dictionary in Python. This is a bit limiting as we don’t have any specific way to tell what is a Tuple and what is a List in our Python implementation in MAXScript as there are no such data types available. So, what I did was I had an Array sent to Python (an actual .NET Array object, by the way) I had that converted first to a Tuple, which is pretty straight forward and then have that converted to a Dictionary. Worked fine, but it was a bit difficult to construct more complex Dictionaries, especially nested Dictionaries, in MAXScript.

Recently I was writing some .NET code and I bumped into Collections namespace. I then investigated further the Hashtable objects and realized I could substitute Dictionaries in MAXScript with Hashtables using .NET classes and objects! Obviously, again, it’s not as elegant as native Python, but it’s closer than manual wrapping of Lists and Arrays!

All you need to do is instantiate a Hashtable object in MAXScript:

HT = dotNetObject "System.Collections.Hashtable"

and then you can call the .Add() method for adding the Key Value pairs! Pretty cool! Also, you can access the Values or Keys similarly to Python’s Dictionaries:

HT.Add "Key" "Value"
HT.Item["Key"]

This is far more productive and intuitive than my previous methods!

Also, then you can pass the Hashtable object directly to IronPython and have it converted to the native Dictionary object using the dict() method, without any issues! Again, this is a much straight forward way of doing such a, rather, complex data type conversion.

I’m currently implementing this approach to my Shotgun wrapper I use with our Python plugin. All seems to be working perfectly fine!

The problem was that a dircet MD5 object comparison of files originating from the same source, i.e.:

from System.Security.Cryptography import MD5CryptoServiceProvider
md5 = MD5CryptoServiceProvider()

file1crc == file2crc

returned, in IronPython 2.7, expected True, but in earlier versions, I got False.

This is the actual checksum converted to a string, so you can see they really are both identical (though computed from different sources):

>>> System.BitConverter.ToString(destMD5)
'2A-74-7D-5F-CC-FD-B6-75-57-C0-30-28-E4-BB-A4-0C'
>>> System.BitConverter.ToString(srcMD5)
'2A-74-7D-5F-CC-FD-B6-75-57-C0-30-28-E4-BB-A4-0C'

Interestingly enough, when I compare the two strings, the result is True.

After an hour or so of experimentation, I had to re-write the script to compare the strings rather than the actual MD5 checksum objects directly, in order to make my script compatible in earlier versions of IronPython.

I still haven’t received an answer to this behavior on the IronPython public mailing list, so, I really don’t have an answer as to whether this is a bug or some kind of a feature that isn’t clear to me. I’ll report back when I get more info.

Once again, after the promising IronPython Tools for Visual Studio 2010 were included in the installation of IronPython 2.7, I decided to give Visual Studio Express another shot. Well, VisualStudio Integrated Shell, to be exact, as Express doesn’t specifically support IronPython.

I’m fairly inexperienced with Visual Studio and also I don’t code very large script/programming projects, thus a full Visual Studio suite is a complete overkill for my needs, however, I really like Instellisense and Autocomplete features it offers, but mainly I really dig the Forms or XAML designers.

So far, I’ve only used SciTE and toyed a bit with SharpDevelop. For 90% of my scripting work (connecting to provided APIs and gluing tools together), where I don’t need to design any GUI, I’m just fine with SciTE’s capabilities. It has Python syntax highlighting, it is extremely customizable, it is the same editor found in 3ds Max (I wish there was a maxscript lexer!) and it’s very lightweight. Also, it’s free.

Anyways, I installed VS Integrated Shell and IronPython 2.7. All worked just fine, no issues, no special steps needed for the two to work together, which is fantastic. I opened up Visual Studio and started some coding. Immediately I simply fell in love with Intellisense. It’s just awesome! Really. The WPF or WinForms designers are absolutely fantastic and generally, the whole .NET support is (naturally) flawless. I’m not much of a debugger guy (again, I don’t code huge projects), I’m better off with some print functions here and there for testing, but I hear the debuggers are also fine. Lastly, the entire integration of .NET, IronPython shell, the script editor, managers, designers etc… everything is just pure gold (from a SciTE user point of view).

However, very soon you bump into tiny quirks that may or may not bother you. For me it’s essentially three, minor, things:

1. The error highlight feature (the wavy underline) will start annoy you very soon.
2. No Tab/Indent guides as I was used to from SciTE.
3. No current line highlighting.

That’s pretty much it! I know these things are nothing, to some, but I just can’t stand them. But if you don’t mind these, definitely go check Visual Studio Integrated Shell (required for IronPython Tools), it’s a fantastic IDE and if you’re into a much more hard-core coding, you’ll love its vast options and features.

For me, I’ll stick with SciTE for now as it has been the editor of choice of mine for years and has never failed me. Of course I’ll be using Visual Studio for the WPF design, that’s just priceless! But then again I can simply design the GUI, save a XAML and reference that in any other IDE of choice for the “gluing” part.

Anyways, if the dev team fixes these three minor things, I’ll be very happy to use Visual Studio full time for my IronPython scripting, it’s just that good.

I’m trhilled to be able to finally showcase, at least, some of our very own Python implementation into 3ds Max!

First off, our primary reason for writing our own, proprietary, Python connection to 3ds Max is Tactic by Southpaw Technology. An awesome asset management system entirely written in Python that I decided to invest in and integrate our tools and software packages into. Another reason for this connection, later came up, was the need for writing much more complex scripts with complex GUIs, since, as you probably know, a few functional lines of code are hardly enough in a modern, efficient, VFX production of today.

The heart of our Python integration is dotNET from Microsoft. I can’t express myself enough how much I appretiate this framework! The brain of our Python integration is IronPython. Also a product from Microsoft, completely open source and free, which are two very important aspects for any pipeline tool in any production facility of any size. Not the price as much as the availability of the software. And with IronPython and Microsoft, I am certain that this piece of software will be around for years!

So, what have we done?

• We’ve developed a bridge that connects MAXScript and Python via dotNET.
• The bridge converts most of native Python and MAXScript data types to and from one another.
• We’ve implemented several, custom, methods in Python that allow us to directly communicate with 3ds Max via MAXScript.
• We’ve also implemented error reporting so that when an error is thrown in Python, it’ll return the exception in MAXScript for debuging.
• Everything is based on dotNET and written in C#.
• The best part? Our Python bridge is completely platform and 3ds Max version independent!

Let me talk about the custom Python methods first. We’ve incorporated a class called mxsCon, that stores several methods for direct communication with 3ds Max as well as a variable
that stores passed arguments from MAXScript to Python for greater flexibility, especially when executing short code snippets.

The initial design meant to provide a way of directly running Python code from within MAXScript. However, with more complex tools needed nowdays, this proved to be insufficient and thus we’ve developed a way of executing more complex Python scripts written externally using IronPython. This works perfectly well if you plan on using 3ds Max’s interpreter and just want to pass values around MAXScript and Python. This is perfectly efficitent and enough for writing really complex tools based on Python. In our case, it’s the connection to Tactic.

However, later, it occured to me that since we have most of the hard work done, why not bring everything over to the Python side. Since Python is such a great and powerful language, why bother with MAXScript! So we developed another method for executing MAXScript code directly within Python scripts and have the result, evaluated in MAXScript, be passed back to Python. IronPython that is. Since all the value conversions had already been done, this proved to be an extremely powerful and efficient way of writing very complex tools, now even, with very complex GUIs. Since everything is based on dotNET, we can utilise IronPython and some of the IDEs available for super quick GUI development. I used SharpDevelop, an open source IDE for writing dotNET applications, for all my GUI and code stuff. Then I just hook up MAXScript commands that I pass Python values to in order to perform some actions in 3ds Max, like opening a file from our server stored in Tactic. Or, create objects and list them in a ListView. Or even write a direct 3ds Max – Maya translator! Why not?

Anyways, here are the methods and their purpose in 3ds Max:

• Python.run( <string> ) — will execute a Python script wrapped in double quotes (effectively a string).

Example:

Python.run
(
	"import sys
	mxsCon.setResult( sys.version )"
)

In this case, Python.run() will return whatever the mxsCon.setResult() will pass to it (more on that later).

• Python.runFromFile( <string> ) [ args:#() ] — will execute a Python script stored in an external file and alternatively pass it custom arguments.

Example:

Python.runFromFile @"C:\myPythonScript.py" args:#( 10., true, #(1,2,3), "some string" )

In Python, you’ll have to catch the passed arguments in order to use them, however the conversion of the data types will be done automatically for you.

• Python.inspect( <data type> ) — this method will inspect what data type you passed to it and how it will convert it to Python.
• Python.getError() — this method returns the last unhandled exception in Python code.
• Python.setVerbosity( <int> ) — allows you to set the level of verbosity for error reporting.
• Python.getVerbosity() — retreives the level of verbosity you currently use.
• Python.resetPy() — a very useful method for resetting our bridge residing in memory in case you’ve changed something radical in the source code etc…

And here are the custom methods developed in C# for IronPython use in the scripts. These only exist when run via our Pythoner.dll module. If you run the Python scripts without Pythoner, you’ll have to handle their lack of presence. I’ve done this via a custom class that I import in the scripts I use. This is extremely flexible since when I run the code outside of 3ds Max, I can still use its functionality without having to deal with error handlers all over the place.

• mxsCon.setResult( <data> ) # this method will return whatever it encapsulates directly to 3ds Max as well as in Python should you need to work with it later in the script.

Example:

import math

def myFunc( arg ):
	sum = arg + 100.
	return mxsCon.setResult( [arg, sum] )

if __name__ == '<module>':
	myFunc( math.pi )

The myFunc function will not only return a List of whatever you pass to it as well as the result of addition, but it’ll also pass this List directly to MAXScript for later use in 3ds Max.

• mxsCon.getResult() # this method will only return the last value stored in memory using the .setResult() method. Useful for later usage in Python.
• mxsCon.convert( <data> ) # this method will convert the input data types from Python to MAXScript compatible data types, but will return then wrapped as a string. This
  may be useful for other usage reasons. I used it for debugging or printing values in Python in MAXScript format.
• mxsCon.execute( <data> ) # here’s where the magic happens. This method is quite complex. It not only evaluates whatever is passed to it in a String format, but it’ll also wait for the result from MAXScript and return it within Python. Another, very cool, feature of this method is similar to String formatting capabilities known from Python.

Some examples of the mxsCon.execute() method:

# assume here's a ton of complex Python code that invokes this method
returnedVal = mxsCon.execute( r'Box()' )
# will create a box in 3ds Max and return OK (converted to True) back to Python.

returnedVal = mxsCon.execute( r'getDir {0}', r'#scripts' )

The .execute method will first evaluate its input into a single string,
in this case: r’getDir #scripts’, in a similar fashion the .format method works in Python.
Then it’ll execute the command in MAXScript and retun its output back to Python.
The output, in this case, will be a string: ‘C:\Program Files\Autodesk\3ds Max 2009\scripts’

But, enought with the talk, see it for yourself in action in this short Windows.Forms demo. (clicking the image will take you to the video)