Checksums in 3ds Max

loocas | 3ds Max,maxscript,Python,technical | Sunday, August 10th, 2008

After reading a very interesting and helpful article about checksums and how practical they are for comparing large datasets over at Adam Pletcher’s Tech Art Tiki blog, I was immediately interested in such methods as I’m doing some R&D on data management in a larger creative environment and need such a feature. Unfortunately, MAXScript natively doesn’t support MD5 hashes (or any other kinds of hashes), so you’re pretty much stuck with just a few options.

One of them is, as Adam describes, utilizing Python from within Max and call its native MD5 functions. There are two ways, but both are very limiting. One of them is registering a COM server from within Python and calling the functions through OLE objects from Max. The other possibility is using blur’s implementation of Python’s interpreter, blurPython, and calling the module, somehow, natively. Both of these methods work and are quite easy to setup, however, both of them are very hard to be deployed in a more complex environment.

What I need is an integrated MAXScript solution that yields the same result as the MD5 implementation in Python, so that I’ll be able to directly compare results from within Maya and Max for example. After searching the web and looking into the MD5 algorithm I quickly realized that re-writing the algorithm myself in MAXScript isn’t the way. But thankfully, Microsoft implemented a vast number of hash algorithms into its .NET framework as I learned on the web, so this looked like the obvious choice.

After a few minutes at the MSDN central I found what I’d been looking for. An actual MD5 implementation. However, as .NET is quite a comprehensive and large framework, the implementation isn’t as straight forward as the one found in Python. Nevertheless, all you need are two classes:

"System.Security.Cryptography.MD5"
"System.Text.Encoding"

These are the .NET strings you’ll need for instancing the classes in MAXScript. So, let’s take a look at the code block:


md5Class = dotNetClass "System.Security.Cryptography.MD5"
encoding = dotNetClass "System.Text.Encoding"
MD5 = md5Class.Create()

In this block, I’ve declared three variables, the first two were instances of the .NET classes and the last one, MD5, is a newly created object from the Cryptograpy.MD5 class called by the .NET .Create() method. You have a few options for the method to specify how the MD5 algorithm should treat your input, but I left it at default as I don’t need anything specific.

Now we are set to go through the last step, which is providing the MD5 object with arguments for computing the hash. This is done through another .NET method called .ComputeHash(). Unfortunately for us, the ComputeHash method only accepts either a Byte[] array or a Stream object (not sure whether it’d take a Max’s fileStream object directly, but mu guess is it wouldn’t) to generate the hash based on the input data. This is where the System.Text.Encoding class comes in handy, because it has a method for converting a string through a concrete character table (ASCII, UTF-8, etc…) to bytes. I’ve chosen the UTF-8 even though I’m certain I’m not likely to use anything beyond the ASCII table definition. The procedure to get the byte code of your string is as follows:


encoding = dotNetClass "System.Text.Encoding"
encoding.UTF8.GetBytes(myString)

Bear in mind that you can use different character tables to facilitate your concrete needs, but basically this is all you need to pass the MD5 function to calculate your checksum. With this knowledge, the following code is all there is to calculating correct MD5 checksums inside 3ds Max:

md5Class = dotNetClass "System.Security.Cryptography.MD5"
encoding = dotNetClass "System.Text.Encoding"
MD5 = md5Class.Create()
hashArr = MD5.ComputeHash(encoding.UTF8.GetBytes(myString)) -- it's smarter to store the resulting array of byte values for later manipulation

The interesting part is that by default, this method returns an array of decimal numbers:

#(13, 87, 80, 212, 152, 226, 236, 248, 58, 163, 91, 29, 33, 199, 60, 102)

I don’t know of any .NET method or a property that’d return a standard hexadecimal value as we’re used to from, for example, Python. However, it is easy to overcome with a formatting function (since Max 2008 or with an AVG extension in previous versions) called formattedPrint. I wrote a custom function that solves this whole issue and does exactly what you’d expect from a natively implemented function, it simply expects a value and returns a hexadecimal (correctly formatted) string. Make sure you check the documentation for the correct formattedString arguments as they are quite important in this case for correct formatting of the hexadecimal values! Here’s the code:

fn MD5Checksum input =
(
    md5Class = dotNetClass "System.Security.Cryptography.MD5"
    encoding = dotNetClass "System.Text.Encoding"
    myHash = "" as string
    myString = "" as string
    try (myString = input as string) catch (return false)
    MD5 = md5Class.Create()
    hashArr = MD5.ComputeHash(encoding.UTF8.GetBytes(myString))
    for o in hashArr do
    (
        myHash += formattedPrint o format:"02x"
    )
    return myHash
)

To sum this up, let’s compare this custom MD5 function based on .NET classes with Python’s native implementation. For the sake of completeness, here’s the code you’ll need in order to generate an MD5 hash in Python (I’ll use the same name “myString” for the string variable as in the MXS example above for clarity):

import hashlib
hashlib.md5(myString).hexdigest()

Yeah, I know, very simple, isn’t it ;) Yet again, Python kicks some serious ass here! Anyways, let’s test it on a long string (just to make sure both of the methods yield the same result) like:

"ThisIsAVeryLongStringThatIsHardToReadButShouldStressTestBothMethodsEnough"

Both MAXScript and Python return (strings):

"0d5750d498e2ecf83aa35b1d21c73c66"

So, this is it, now you should be more familiar with .NET classes, how they work from within MAXScript and that the whole .NET framework is one hell of a good job by Microsoft for developers of any kind, such as TDs working in DCC applications, which is sweet! ;)

14 Comments »

  1. An interesting read, but the above fuction doesn’t work for me:

    Runtime error: dotNet runtime exception: Cannot widen from source type to target type either because the source type is a not a primitive type or the conversion cannot be accomplished.

    Any ideas?

    Comment by Giantrobot — October 16, 2008 @ 13:26

  2. Hmm, strange, it works like a charm here…?

    Are you sure you have the latest .NET redistributable? If you’re running WinXP, try downloading the latest .NET pack from Microsoft.

    I’m running Vista x64 and Max 2009 x64, but also make sure you have the AVGuard Maxscript extension installed if you don’t run Max 2009.

    Comment by loocas — October 16, 2008 @ 14:39

  3. Have latest .NET

    Running 2008, could that be the problem I wonder?

    Comment by Giantrobot — October 16, 2008 @ 16:03

  4. Hmm, just make sure you have the AVGuard MAXScript extension as that got bundled with MXS in 2009 I believe.

    If you’ve got everything, then I have no clue where might be the problem.

    Have you tried declaring the .NET classes alone? If that works then .NET isn’t the problem, if it fails, then… :)

    Comment by loocas — October 16, 2008 @ 16:22

  5. 2008 comes with AVguard extensions included.

    Taking it line by line all works fine until
    hashArr = MD5.ComputeHash(encoding.UTF8.GetBytes(myString))

    when I get:
    Runtime error: dotNet runtime exception: Cannot widen from source type to target type either because the source type is a not a primitive type or the conversion cannot be accomplished.

    Comment by Giantrobot — October 16, 2008 @ 18:40

  6. That’s strange, unless you haven’t declared the “myString” variable with anything that can be converted to bytes by .NET.

    try after declaring the .NET classes this:

    hashArr = MD5.ComputeHash(encoding.UTF8.GetBytes(“TEST”))

    Comment by loocas — October 16, 2008 @ 19:12

  7. I get the same error, tried i on another computer using 2008 too, also same error.
    Strange…

    Comment by Giantrobot — October 16, 2008 @ 19:57

  8. Hmm, this really is strange, I can’t think of anything that could potentially be causing this problem :(

    Comment by loocas — October 16, 2008 @ 20:06

  9. Interesting, I just tried running the code in Max 2008 and I get the same error you are talking about! So, I assume, this must be a dotNET version incompatibility issue or something like that. As I said, in Max 2009 the code works just fine, without any problem.

    Comment by loocas — October 16, 2008 @ 20:10

  10. It seems that the ComputeHash() method can’t take an array of bytes from within MAXScript in 2008. Interesting. But thankfully they fixed it in 2009 :)

    Comment by loocas — October 16, 2008 @ 20:22

  11. Tested the functions on 3DS Max 2017 and it looks like the python implementation is about 10x faster….

    (14.388s) Func: DotNet_MD5
    (1.502s) Func: Python_MD5


    fn DotNet_MD5 input =
    (
    md5Class = dotNetClass "System.Security.Cryptography.MD5"
    encoding = dotNetClass "System.Text.Encoding"
    myHash = "" as string
    myString = "" as string
    try (myString = input as string) catch (return false)
    MD5 = md5Class.Create()
    hashArr = MD5.ComputeHash(encoding.UTF8.GetBytes(myString))
    for o in hashArr do
    (
    myHash += formattedPrint o format:"02x"
    )
    return myHash
    )

    fn Python_MD5 str =
    (
    if ::hashlib == undefined then ::hashlib = python.import("hashlib")
    (hashlib.md5(str)).hexdigest()
    )

    fn Test function_name =
    ( Local fun = execute function_name
    local st = timestamp()
    if classOf fun == MAXScriptFunction then for a = 1 to 50000 do fun ("Hello_" + a as string)
    format "(%s) Func: %\n" ((timestamp()-st)/1000.) function_name
    )

    test "DotNet_MD5"
    test "Python_MD5"

    Comment by doob — October 17, 2016 @ 22:19

  12. Very nice, doob! But not surprising. CPy is compiled right at front, while .NET uses JIT compiler. At least that’s what I assume makes most of the difference.

    Good info, thanks!

    Comment by loocas — October 17, 2016 @ 22:35

  13. doob, nice find, thank you! FWIW, half of the time for the .NET version is spent in the formattedPrint loop.

    Comment by Martin — January 9, 2017 @ 17:59

  14. Ha! Mr. Breidt himself! :) cheers. Also, yes, the formatted print loop isn’t the fastest, unfortunately. Can be heavily optimized, of course.

    Comment by loocas — January 9, 2017 @ 20:49

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress | Theme by Roy Tanck