Teenage Mutant Ninja Turtles: a visit to ILM

“It’s the science of capturing the data, it’s the science of retargeting that onto the creature and it’s the animation artistry coming in and intervening and really interpreting what that data is.”

That’s how Teenage Mutant Ninja Turtles VFX supervisor Pablo Helman succinctly explained how they used motion capture in Teenage Mutant Ninja Turtles to a room of journalists. It all sounds so simple, doesn’t it? Oh, we had to capture data, so we just used science. Then we ran a quick paintbrush over the science and Ninja Turtles fell out. Easy, really.

The lengths the team at ILM have gone to are pretty incredible, though. The time, effort and expertise that’s invested in making sure there are no tells, no giveaway signs that what you’re seeing isn’t there, involves taking the term ‘detail oriented’ very, very seriously.

There’s a new Teenage Mutant Ninja Turtles film coming out and I’ve been pretty excited about it. We’d received an invitation to go and speak to some of the people who had created the special effects for it. Not a bad day when that call comes through.

I was sent over to see the good folks at ILM (Industrial Light & Magic). ILM is the special effects company started by George Lucas. You’d know it once you got there, too. A Yoda statue complete with water feature sits outside, there’s a full scale Boba Fett in the lobby and a Star Wars presence (along with bits and pieces from some of the many other movies ILM have worked on over the years) throughout the buildings I got to see. The security guards don’t carry light sabres, though, so there’s still some room for improvement, if not much.

I was so excited to talk to ILM about the new Teenage Mutant Ninja Turtles film. Although I hadn’t seen the film yet, I had seen plenty of trailers for it and while I can understand the complaints some people have about the new designs of the Turtles’, the execution of those designs looks stunning.

During my visit I got a chance to speak to some of the key figures behind the Turtles in the new TMNT movie, which has been directed by Jonathan Liebesman and produced by Michael Bay (neither of whom were in attendance, I should highlight). Luckily for me, I found the people at ILM to generally be friendly and chatty. It might have helped that they seem genuinely excited by what they’ve achieved with Teenage Mutant Ninja Turtles and that I was genuinely excited to hear it.

Let’s start with the most controversial element of the film: Megan Fox.

No, I’m teasing. I mean the designs of the Turtles. With noses and lips and clothing and accessories, the Turtles in this film look quite different from those many of us grew up with, or indeed those from almost every other previous iteration. They’ve been subject to some online derision, although how much of that is genuine response and how much is just another internet pile on, it’s hard to judge.

“You’re probably wondering how we have the designs that we have.” Pablo Helman astutely suggests as part of an introductory presentation. “We tried all kinds of things, with a really wide scope. We started with something really on one side. We also had some stuff that was very benevolent and very small and very much like some of the [previous] versions of the Turtles. And we decided, with Michael Bay and Jonathan Liebesman, to come back to a little bit more realistic.”

Pablo Helman leads the day. A confident, charming chap, Helman is the VFX Supervisor (which, as far as I could tell, meant he was running the special effects show) of Teenage Mutant Ninja Turtles.

He showed us (the assembled journalists) some of the designs they’d been working on. Some of it looked even more harsh, or ‘realistic’ to borrow the term they’d chosen to use, than the final designs. They reminded me a little of the Soul’s Winter comic books (a three part Teenage Mutant Ninja Turtles comic by guest writer/artist Michael Zulli). One of the drawings did have a traditional, round-snouted, classic-looking Ninja Turtle approximated in the modern animated style and it looked weird. That’s not to say that this style of Ninja Turtle wouldn’t have worked, but that the version they showed us looked really quite odd. A more cynical man than me might think we were shown an interpretation that didn’t work to Inception the idea that it couldn’t work at all into our subconscious. Not me, though. If I even suspected that was what had happened I’d probably find a way to subtly imply it, perhaps by attributing the suggestion to a non-existent cynic.

“I think if we had gone that way, you can see, we would be wrong.” says Helman, in soft, almost sympathetic way. Like, he knows we might have wanted it this way, and they tried, but it just wasn’t going to work out. The natural progression from that tone of voice would have been to drape a reassuring arm over my shoulder and to call me ‘champ’. I would have liked that.

“He’s the least human of the characters we created,” animation director Tim Harrington explained of Splinter, the Turtles’ mutant rat sensei, who was created with less emphasis on motion capture. “If you look at his face, there are really not any human qualities at all. He’s just all rat. So, because of that, we took a lot of liberties with the animation, we were able to incorporate a lot of rat-like behaviours into his movement.”

The Teenage Mutant Ninja Turtles, on the other hand, make full use of motion capture technology. Teenage Mutant Ninja Turtles is the first film to use ILM’s new MUSE facial capture system, which was the focus of the day and something I’ll be coming to in a bit.

The MUSE system wasn’t the only innovation applied to Turtles, though. New clothing for the motion capture, complete with unique barcode-like markers for each performer to help identify who’s who, was used. With the innovative use of facial capture it’s easy to forget that body capture is still a huge part of the process.

“We have three cameras around the set that triangulate the performance of the actor. We track that movement with a digital version of the actor” says Tim Harrington. “Once we have that, we take that motion data and we apply it to the Turtle.”

Over the course of the day I saw a few clips of the film that featured some very exciting-looking fighting (is there any other kind of fighting?). I asked associate VFX supervisor Robert Weaver, who was part of the on-set team for ILM, what they’d thought the biggest challenge of capturing the fight scenes would be.

“Probably the physical constraint of having a giant shell on their back,” he replied. “You would think that that would limit you, but these guys were so talented that they could stand there and do a back flip like that – boom. No running, nothing. Just standing – backflip. Ok, we need that six more times to shoot again, he would do it six more times. So, I thought that was going to be a really terrible physical constraint but it wasn’t.”

Now, onto the MUSE capture system. The MUSE system was explained to me repeatedly, out of necessity. I’m handsome but I’m not the sharpest blade on Shredder’s forearm. It was a lengthy process (creating the Turtles, I mean, not explaining it to me, although…) and the team were rightly keen that we understood the full scope of what they’d done.

“That is a huge endeavour with R & D and bringing it from research and design all the way through to meeting the needs of production, which, you can imagine how pressurised production time is,” explained Robert Weaver. “So we had very little build up time to create a system that would work at the speed we needed it to work. And not just the speed, but most importantly, that would capture the subtleties. We were, I think, very successful at capturing the just-so-subtle nuances in the characters.”

The idea is that by capturing the details of how a human face moves and then applying the movements to CG character, you’ll get something more believable or that looks more real. Pablo Helman told us, “We had 138 tracking markers on the faces. Throughout the day we had to recalibrate that about two or three times.”

A further requirement of the MUSE system was that it be flexible, Helman continued. “It’s a real big, big process, especially when you are dealing with a comedy. That is the main difference between us and another show; when you’re dealing with comedy, you’re dealing with four actors, funny actors, sitting there at a table with a situation, with a script, coming up with funny ideas and ad-libbing. You’ve got writers that are just throwing lines, ‘Here, just try this, try this…’ To do that, what happens is that you’re not set with only one take. Now you have all kinds of takes you put together and that’s why we needed the system to be editable. That’s what’s different about everything that we’re doing.”

“To do that we designed a helmet that for the first time would have two HD cameras. Right from the beginning we started capturing three and half times more the resolution and the amount of data that normally we would with regular motion capture. To do that, you’re basically capturing about one terabyte of data a day per performer. So, you have to manage all your data and that’s part of the new system.” I can’t even begin to imagine what their Dropbox subscription costs.

He went on to tell us that, “We decided we were going to capture the data and at a raw level we were going to be able to manipulate that data to make a performance.”

So, they capture huge amounts of data that can be fiddled with at a raw level, so that there’s a high level of detail in the finished article but also a high level of flexibility in creating it.

While the ILM team were full of praise for the actors, they made the point that the final performance was really a collaborative effort. I cocked an eyebrow at the idea at first. Oh right, the special effects guys want credit for the animation and the performance, do they? It turned out that I’m an idiot and that they had a point, and the example they used to demonstrate what a simpleton I am also helped me get a grip on how they were using MUSE.

They demonstrated their process using a moment from the film that you might have spotted in one of the TV advertisements. It features Donatello, played by Jeremy Howard (from Galaxy Quest), stepping up to take the lead in an action sequence. Donatello is the Ninja Turtle more renowned for fixing machines, and this is a notable character moment for him. The films Associate Animation Director Kevin Martel told us how it works, slowly, but in a kind way, like he had worked out my level.

“Donatello says ‘Allow me to be the badass for once!’ and then he turns and runs. We went through and we looked at other takes that Jeremy gave, there was one that seemed particularly funny to us where he was like ‘Allow me to be the badass for once’ and then he had a moment of doubt where he kind of stopped and he looked back a couple of times. He had to build up his courage, rather than just running off. We found that interesting.”

They showed us an unfinished version of the animation where just that happens. I’d get into describing what the early animation looked like but video presentations rarely come to life in print. In fact, going by the theory that a picture is worth a thousand words, at twenty four frames per second the eight second clip would take me one hundred and ninety two thousand words to describe. From experience I can confirm that Den Of Geek won’t publish my articles if they’re that long, so here’s a sixteen word version: just imagine the version you saw in the TV spot, if you saw it, but cruder.

Vivid, no?

“It was taken from a different performance of Jeremy where you can see he’s got some internal self-doubt, looking back, nodding, okay, I’m gonna do it.” said Martel. The scene was made up of two different takes, one with the some of the more standard elements of the scene and the take that featured the hesitation, which is where the editing capabilities of the MUSE system are required. See, I was following the entire time.

“Then, in the final performance, we pushed it even further in animation, where he gives a little smile before he goes. Adding that little nervous smile definitely changed the feeling of the scene” says Martel. “The great thing is we can really take bits and pieces of all sorts of performances that the actors did and when the director sees stuff he’ll have new ideas. We can animate those ideas and mix and match. It’s a really great way to come up with the ideal performance for the scene.”

In some instances, a single shot will feature multiple takes from the actors, sometimes recorded months apart and in different locations. It’s all seamlessly animated together so you’d have no idea that this was the case, unless a person standing in front of you had just proved it with pre-animation clips showing you the actor performing part of the take on location and another part in front of a green screen. Even then you’re still suspicious, assuming you’re me, and start wondering whether you should contact the authorities over suspected witchcraft.

As the day progressed, after extended exposure to special effects experts who had repeatedly proved to me that I don’t know anything, you’d think I’d have become more trusting. But when they told me you could actually see the faces of the actors in the Turtles’ faces, I did not believe them. ‘I have seen the actors, I have seen the Turtles, I understand that I am being lied to’, was my internal response.

To demonstrate that I was again very wrong, Tim Harrington showed us that actor Alan Ritchson’s face could be seen in the face of Raphael and explained how that could possibly be the case.

“We sit him [Alan] down in a chair with a proprietary system of like eight cameras that are all around him. He goes through fifty-five different expressions that we captured. Once we have those expressions we scan them, bring them into the computer, we put them onto our digital version of Alan. We clean them up a little bit, make sure they’re good. Once we’re happy with that we can transfer it to the Turtle.”

“The interesting thing that we discovered when we were doing this, now, when you look at Raphael without Alan’s facial expressions he physically does not look anything like Alan. But once you take Alan’s facial expressions and put them on Raph, he starts to come through. That’s something we didn’t anticipate.”

They showed us a side-by-side comparison and you could totally see it. It was mad, because once you’ve seen it you see the actor in the Turtle’s face every time.

There was a downside to mixing human and mutant turtle faces, though. As per Pablo Helman, “There is a really a very fine line between an honest smile and a wince. You can see, for instance, in monkeys, when they smile it’s more like a wince. Teeth are very difficult in animals and creatures. If a dog is showing you its teeth, it’s not good news. It’s something that we had to really take into consideration.”

With so many challenges involved, in the face of criticism and with the ability to create Splinter free from human-faced constraints, you might wonder why they even bothered incorporating the human elements of the face at all. Is this a case of the temptation of ‘can’ over ‘should’?

The team at ILM would likely disagree. Their design process was clearly thorough and there’s no denying that they believe in what they’ve done. More importantly, though, they were able to utilise the actor’s performances to make the Turtles’, the focus of the movie, as full of life as possible.

“We don’t want to lose any of the subtlety or any of the nuance that the actors brought to the scene” says Kevin Martel of the animation process. With the Turtles’ performances coloured by all of this humanity, they hope Leonardo, Donatello, Michelangelo and Raphael will feel more real than ever.

Teenage Mutant Ninja Turtles lands in the UK on Friday 17th October.

Follow our Twitter feed for faster news and bad jokes right here. And be our Facebook chum here.