How to... animate the future

Whether for flight or fancy, we have always been fascinated by motion.

From the superimposed actions of animals depicted in Paleolithic cave paintings, to drawn sequences of battles on darkened murals in Egyptian tombs, shadow puppets, magic lanterns, zoetropes, hand-drawn cels, armature wire and mystery wind-blown apes, and the first advent of computer-generated toy characters to the digital humans and photoreal VFX we have today, we have always found a way to capture and then tell the stories that would otherwise not be seen. 

Anyone who has ever worked in animation knows how hard it sometimes is to simply add a few frames of head and tail to each animated performance.

Technology has kept up with and been driven by this very human need to share a vision and communicate. Today, we have come to expect and rely on physically-accurate simulations, photoreal and real-time rendering, and machine learning that superpowers image, motion and data analysis.

Above: Boo Wong, Group Director of Emerging Technology, The Mill

Let’s stop there for a moment.

Last year, as we wrapped production on a multi-month VFX-filled commercial of a giant furry character tearing through a city, our client circled back around to say thanks, nicely done, and now can we talk about creating 30 more spots of the same said creature?  But in a fraction of the time?  Did we already say three weeks?

In normal history we might’ve said thanks so very very much, it was simply amazing to have had the chance to work with you on the one fab spot but please do come back when you find several more months in your calendar.  Instead, as fate, innovation and the state of game engines would have it, The Mill’s Creative Technology team were just then pushing the limits of visual fidelity on interactive AR and VR projects.  And we were inspired, therefore, to bring Monster’s VFX character into the real-time world.

In collaboration with one of our creative directors, Jeff Dates, we built a real-time character animation system we later called Mill MASCOT that uses live inputs from puppeteer to drive a photoreal furry creature for our clients at Monster and KBS.  We set up a day-long “shoot” where our puppeteer performed and we captured, live on set, over 4 hours of final pixel animation - i.e. no post rendering or composites.

This was a huge advancement from a standard CG animation job, where it could take weeks or even months to see a digital character rendered and comped into a scene. Using the Mill MASCOT workflow and a live animation session, our render times ran at 42 milliseconds per frame, meaning that final pixel performance and capture were, for all extents and purposes, instantaneous. 

Anyone who has ever worked in animation knows how hard it sometimes is to simply add a few frames of head and tail to each animated performance.  And here for Monster, we had over 4 hours and multiple versions of final rendered shots for the editor to use as selects from which to cut the thirty spots. 

What Mill MASCOT has shown is that using real-time technology is not just for interactive products.  It has the potential to be the most powerful tool yet in our arsenal of filmmaking.

The companies making game engines have very much realised this, too. Both Unity and Epic Games have enterprise divisions targeting Film and TV, or Animation. It’s not just a way of producing content faster, although game engines do that exceptionally well as in the case of MASCOT.  Real-time technology brings with it all the game logic that allows for smart cameras to auto follow objects in scenes, trigger lighting or cue other behaviors, develop character AI, set up A/B testing for animation and performance, as well as utilize built-in scalable and concurrent workspaces for collaborators.