Who’s Responsible for Motion Capture Performances?

Who’s responsible for motion capture performances? Well. the answer’s Andy Serkis, right?

Seriously though, if anyone is synonymous with this relatively young effects method, it’s him. Gollum, Kong, Caesar, and most recently Snoke in Star Wars: The Force Awakens wouldn’t be the same without his input, on sets and in the suits.

But during press for Dawn Of The Planet Of The Apes, Serkis made some comments about the progress made in creating Caesar: “[Weta Digital] have now schooled their animators to honor the performances that are given by the actors on set. And the teams of people who understand that way of working now are established. And that’s something that has really changed. It’s a given that they absolutely copy [the performance] to the letter, to the point in effect what they are doing is painting digital makeup onto actors’ performances. It’s that understanding which has changed as much as anything.”

These seemingly harmless remarks prompted outrage amongst the VFX industry. Most of the objections, including Ferdinand Engländer’s here, surrounded Serkis’ use of the term “digital makeup” and how it sold short the importance of animators’ work on the characters. The problem is, many who’ve worked with Serkis have bought into this same thinking. Rise Of The Planet Of The Apes co-star James Franco even used “digital makeup” when reflecting on the film. So where does the balance lie?

We need to trace the history of motion capture back. If Jurassic Park’s partly digital T-Rex and Raptors count, we’ve had digital characters as early as 1993. The next stage was then for characters that could deliver emotive performances, with human actors. Jar Jar Binks is an early example, not of capture per se, but shooting on The Phantom Menace featured as close to a physical reference of the Gungan as possible. Ahmed Best was in a physical suit, except for Jar Jar’s head consisting of a kind of hat on his head, with a black visor masking his eyes so as to make the other actors on set focus on Jar Jar eyes instead of referring to Best’s eyeline.

Serkis was the next big step playing Gollum in the Lord Of The Rings films, though it wasn’t a motion capture role at first. To be fair to Serkis, his feelings may be in reaction to a long-perceived stigma against such acting. As he will tell you, in The Lord Of The Rings he was supposed to just be a voice actor at first. That was his audition process, but from the moment Peter Jackson saw the video of said audition, one thing led to another. Appreciating Serkis’ physical performance and how it was interlinked with the voice, he put him in shots with the actors, wearing a unitard to refine the blocking and lines between the actors for the animation reference.

He and others have talked about how, understandably, having an actor in a white spandex suit where Gollum was supposed to be was surreal in an environment like the Lord Of The Rings sets, where everything else was so carefully designed to create Middle Earth. But Gollum himself was redesigned to better reflect the actor’s facial structure and expressions. Then they went into motion capture sessions, a bodysuit with dots which special cameras capture as data for a 3D representation of body and skeleton movement. More and more, Serkis’ dedication to the role led people to greater accommodate his contributions.

Countless crew members from directors, fellow actors to the VFX artists themselves have praised Serkis’ sheer dedication to his performance. Dawn co-star Toby Kebbell called him “the biggest influence” on his own mo-cap work as villainous ape Koba.

The original actor is absolutely an invaluable reference. As Serkis mentioned, the technology has been developed to closer follow his performance. He can guide and hint towards the character’s movement and behaviour, but what’s arguably just as important is because that actor was on set, interacting with the other actors and the environment, the finished CG character has greater presence and physicality in the shot. They are clearly a part of what’s going on, particularly whenever said character makes contact with the other actors, instead of obviously just added in long after production wraps.

Plus with motion capture there’s the ability to rotoscope the animation according to the actor’s movement, to animate by tracing over the footage rather than animators trying to keyframe a character from scratch. It’d be easy to think animators’ jobs were made easier by this development.

But the animators still have to make the person-in-a-suit into a character as designed, and it’s not simply digital makeup – animators do have the freedom to change things which better fit a moment, and in a lot of cases it’s a necessity. Simply, the more traditional keyframe animation, where a digital element is created from scratch, remains crucial.

Randall Cook clarified in an exhaustive comment on Cartoon Brew that on The Lord Of The Rings, however important a physical reference Serkis was, in most shots he was completely replaced by keyframe animation. There was the opportunity to track Serkis’ motion in shots and rotoscope the animation to match that movement, but Cook insisted that even the actor’s energy could not sometimes portray Gollum’s agility with complete accuracy. “There were some scenes where we used no Andy reference at all.” More intricate stuff such as facial and finger movement all had to be keyframed.

Now, part of Serkis’ point was that motion capture continues to be closer linked to the live performance, hence why we increasingly see the process referred to as performance capture rather than motion capture. Later on with The Hobbit, the mo-cap suits had dots to track finger movement and a head-mounted camera to track facial movement, all to derive more data from the live performance. But it’s a performance which still needs interpreting by people other than the actors and director.

The main reason is the same one why the finished characters can’t just be recreated through complex makeup, prosthetics or man-in-a-suit solutions; the anatomy of these characters are fundamentally different from a human’s. For instance, how exactly do Benedict Cumberbatch’s human face and movement translate to The Hobbit’s firedrake dragon Smaug? Cumberbatch had the same suit and setup for performance capture work as Serkis did, but there you have a creature whose skeleton, muscles and movement clearly don’t correspond to our own, and Weta knew it. Yet they were still interested in using Cumberbatch’s performance to provide “clues”, as Joe Letteri puts it, to Smaug’s character.

The animators indeed did what they could to accommodate the actor’s vocalisations, bringing Smaug’s performance to the front of his face to make it as recognizably emotive as possible instead of the feeling of a model’s jaw moving up and down whenever it talks. And it works excellently.

Computer-generated animation is a way to create convincing characters, but also provide a subtler range of emotions that costumes and makeup often lack for all their physical presence – this is more appropriate for when the character is needed to be a major one, capable of giving a nuanced performance alongside the human actors. Someone like Gollum probably couldn’t be recreated without the aid of digital effects and still be true to the decades of artwork the character had already inspired. Unless Andy Serkis was prepared to be Christian Bale in The Machinist, while also somehow having huge eyes, for more than a year of shooting. That’s not even accounting for the fur on apes like Kong or Caesar, which was on average a million hairs for each ape and has to be adapted for all the various weather conditions in the film.

The dispute is another example of how sole authorship in film is a very slippery concept. Actors, even the best ones, need direction to realise their characters, and yes, those makeup artists to greater sell the illusion on screen; directors can only realize a vision if there are the hundreds of cast and crew there to help them achieve it; a writer’s screenplay will be rewritten and improvised with during an actual shoot. What about the studio pressures that forced them to insert an action scene? Yet we talk in such absolute terms all the time, as a film belonging to solely to a director, for instance. And yes, the balance of power varies on every single film, but unless we hear a lot of behind-the-scenes rumors suggesting otherwise we refer to the same defaults. Because (and I’m as guilty as anyone of this) it’s much easier and more convenient to discuss and evaluate films that way.

How is this any different from the cast and crew working to create this character? I think we do the same for motion capture actors because, without denying their own big contributions, we like to think of the human element behind the character rather than the digital work, and the hundreds of artists responsible for that, as well.

Serkis would perhaps argue that the character is increasingly his, as motion capture technology continues to improve. So the balance of authorship is supposedly shifting. But given the variables already mentioned, credit to the animators too (again, something Serkis gives); it’s just easier to see them as the background people who churn out shots instead of being actual artists.

So it’s not surprising that Serkis received a backlash. Once again, being completely fair to the actor, he did temper his comments in a later statement, pointing out the Weta artists’ importance in terms of “that very delicate phase between taking the actor’s raw underlying emotional performance and translating it into the actual final rendered character.”

Dawn of the Planet of the Apes director Matt Reeves also spoke about this in Den of Geek UK‘s interview with him for the film’s release: “The miracle of Rise was that Weta was able to translate performance so faithfully to the emotion that you were able to identify with a CG character at a level that I’d never had before. I’d appreciated great CGI before, but I’d never watched a movie where I essentially felt I was that CG character. Andy Serkis has played many CG characters, and you marvel at them. You go, ‘Wow, that’s a great character.'”

Really, the backlash Serkis faced is just one of recent incidents in which, unintentional or not, the critical role animators play in realizing these memorable characters has been marginalized. Serkis may never be the biggest name in movies, but he at least gets some wider recognition for his work in realising those characters. How many people, by contrast, actually know of Joe Letteri, Weta Digital’s most senior figure? It doesn’t help that other VFX-minded people seem to buy into the ‘digital makeup’ theory, including Dawn’s VFX producer Ryan Stafford, who said their methodology was to ‘put a great actor into the role and then you just transfer their performance directly onto the CG model.’ Intentional or not, the artists’ contributions are being marginalised. And not just in this case. Because few in the industry seem to fully value them, the industry sees fit to squeeze their salaries and timeframes for completing projects as much as possible.

Salaries, as well as timeframes in which to do their work, have been severely pressed by studios and filmmakers who assume that major effects shots can be rushed out like magic. The furore surrounding Life Of Pi was the ugliest point, with active protests at the industry’s mistreatment and misunderstanding of visual effects. Rhythm and Hues, the studio that aided the film’s VFX, was actually bankrupt by the time of the ceremony. When Life Of Pi won the Oscar for Best Visual Effects that year, visual effects supervisor Bill Westenhofer‘s acceptance speech was cut short by music when he tried to speak out against continuing job losses. Here’s what Westenhofer had to say to Bill Desowitz after the ceremony:

“At a time when visual effects movies are dominating the box office, that visual effects companies are struggling. And I wanted to point out that we aren’t technicians. Visual effects is not just a commodity that’s being done by people pushing buttons. We’re artists, and if we don’t find a way to fix the business model, we start to loses the artistry. If anything, Life Of Pi shows that we’re artists and not just technicians.”

And they are artists. Possibly the most important part of Randall Cook’s comment is “…Andy really should be considered the principal author of Gollum’s performance, but there’s a hell of a difference between principal author and sole author. The animators who helped shape Gollum’s performance [in LOTR] are actors of a very special type, working at a high level of achievement.”

It’s an interesting debate, where the credit for such praised characters truly lies. With people constantly calling for Serkis to be Oscar-nominated, is it all part of an attempt to feel like we need to legitimize our use of CG in some way? After all, digital effects continue to have a less-than-stellar reputation among may cinephiles. For every example where it’s genuinely thrilled audiences and added something (e.g. Gravity), pushing the medium forward, there’s another three examples where all the artists’ talent is used to fill in spaces of the latest effects film forgotten in a fortnight.

So it’s reasonable to think that all the critical praise for the motion-captured characters comes from knowing there was a human presence partly responsible for its creation, while the digital side of the creation is still viewed as unremarkable because hey, it’s all the same effects used to create the stuff we don’t like, right? Maybe we’re all too quick to assume that quality digital effects are too easy, and that’s why the VFX artists are so keen to have their efforts recognised.

We have a situation where both parties’ work is obscured to some degree. We technically don’t see Andy Serkis but a digital creation when we watch the finished product, and it’s also too easy to put two pics of Serkis and the finished character together, and assume the model was just ‘digital makeup’, rather than a ‘translation’ of the live performance which is just as integral to the character.

Motion capture is a unique thing in film history, where the work of invisible crew members has never been more visibly prominent or critical to a film’s success or failure. It’s not just a big background or set-piece, it’s key characters that are now being created to stand and perform with the main actors.

But it’d be wrong to think of performance captured characters as ones of a dual identity, a la the corrupted hobbit who’s its most iconic creation. Serkis might have actually said it best during the Lord Of The Rings days: “it just feels like it’s this kind of confluence of all these different skills and people’s energy that is making this thing special.”

This article comes from Den of Geek UK.