Gemini Man’s Digital Effects Supervisor Bill Westenhofer On the Future of CGI

This article comes from Den of Geek UK.

Actors have been playing opposite themselves for decades – think Alec Guinness killing himself again and again in 1949’s Kind Hearts and Coronets. But Ang Lee’s new film Gemini Man takes things to a whole new level, pitting Will Smith against a wholly digital recreation of his younger self on the screen.

Oscar-winning visual effects supervisor Bill Westenhofer worked closely with Lee to bring the 23-year-old ‘Junior’ Smith to life. Den of Geek spoke to him about working on these challenging new effects, and what they might mean for the future of the industry.

First of all, can you take us back to the beginning – how long were you working on this project? Did you ever have any doubts when you first took on the project? And what convinced you that this could be done successfully?

From August of 2017. You know, every time you try to do something that hasn’t been done, there’s going to be doubts. But I think when we looked at the situation, we didn’t have much of a choice in order to do the job, because we had Will Smith in the same frame as his younger self. We knew we had to go ‘full digital’.

But we looked at what had been done and was state of the art, and the attempts that had gotten pretty close, like Rogue One and Blade Runner 2049. We knew that with where things were at, that we felt confident enough that we could push it the rest of the way there.

The filmmakers have said that they don’t classify it as de-ageing, but creating a new character. Can you talk me through how it’s different?

So, typically, it’s more relevant in the visual effects world where it matters to us. But the de-ageing that you see in Captain Marvel, that is taking Sam Jackson and putting him in a costume, and putting a wig on his face, and putting on a lot of makeup as much as you can – and then it’s a post-process.

Maybe the simplest metaphor is: imagine taking somebody into Photoshop where you smooth out the wrinkles, and you reshape the face a bit, and try to remove the years that way. We couldn’t do that because we had both of them in camera at the same time, and often fighting each other. So we couldn’t just put Will in Junior’s costume and do that. We had to build him as a completely digital asset, more like Gollum or the apes in Planet of the Apes where it’s just computers doing everything. There’s no photography that we’re using to manipulate it. It’s all created in a computer that way.

So how did those scenes work, when it was Will fighting Will?

We had Will play his older self, and we had a young acting double. His name is Victor Hugo – no relation – a 23-year-old British actor who is Will’s height and the same skin tone, and everything that Will had when he was 23.

read more: Gemini Man Review

So we’d do the scene with them, and then after we’d finished live-action production, we went back to all the scenes where Junior was with Henry, and set up a motion-capture stage with a bunch of cameras around. Will’s wearing a suit with a bunch of tracking markers on him, and a lot of markers on his face. He then performs the Junior role. We have in the monitors and in the TVs, we have a playback of Henry, so he can actually play against himself and hear the dialogue he gave on the day. We actually have the guy, Victor Hugo, stand in for Henry just for eye line and, again, for a performance double.

But we then capture just the data of Will performing Junior. And that data is then fed in to drive the completely CG version of him. We erase Victor from the shot, and then put in our digital Junior.

That must have been quite a feat for Will himself then.

Yeah. [laughs] The good news is, he’s done this kind of thing before. You know, he did Aladdin. He’s done a lot of effects movies. So the abstraction of it all came a little bit easier than it might be to most.

But, yeah – what’s really important to know is that the performance, and what you see, is 100% Will. There was no magic button that we could push to say, “Make a 50-year-old acting feel like a 23-year-old acting.” He had to take on his younger self.

So Ang and he did a lot of work at the beginning of the shoot, just reading through lines as the Fresh Prince reading through lines, and as the character from Six Degrees of Separation. He actually read it as Will’s father, and that was appropriate because Will’s father was a military guy. And while Junior is the Fresh Prince age that we’re used to, he’s a different character. He’s very sombre and he’s very serious, in more of a military sense. That’s kind of worked into it as well.

I suspect it might have been weirdly emotional for Will to be looking at his younger self. Was that something he said?

Yeah. I’m jealous. We’d all love to see our younger selves come back to life. He did get a kick out of seeing that, especially when some of the first renders started coming in.

We read that it took a year to finalize the first shot of Junior before Ang Lee was satisfied. Is that right?

Absolutely. With Weta, we looked at what had been done before, and we spent a lot of time on the tiny details and the geeky science of it all, down to things like the fact that there are four different types of melanin in the skin, and they react to different angles of light. We built the eyes with a dark retina, like a white glare on the front, this layer of film they call the conjunctiva.

Human beings are experts at faces in particular, because we evolved for millions of years to see nuances, like a crooked smile means that you’re lying, or there’s something wrong with somebody. That’s what the uncanny valley is meant to describe. If the result is looking really real but something’s off, it’s uncomfortable.

So we had to obsess for a year on all of those kind of details. The end result, the success, is not necessarily any one thing – it’s a symphony of all those little details coming together.

You’ve worked with Ang Lee before and you won an Oscar for Life of Pi, which is amazing. How much does Lee push you to push boundaries in FX?

All the time. That’s what he likes doing. It’s always climbing the next mountain that’s exciting. You know, if you’re not a little bit afraid of a job, it’s kind of not worth doing. The fun is trying something that hasn’t been done before.

When you asked where we felt we could do it, there was the clinical analysis of what had been done before, and where the state of the art was. But you know, we thought about the same feelings that we had on Life of Pi about whether we could pull off a real tiger, and find the same determination to do that on this one.

So he literally said, “Let’s do for humans like we did for the tiger on Pi.”

How do you compare Ang’s attitude to VFX compared to other filmmakers you’ve worked with?

What’s great about Ang with visual effects in particular is that he considers it a visual art.

Everyone loves articles about the technology behind the latest visual effects thing, and the latest and greatest. But really, if you think about what visual effects brings to the set, no one questions the artistry of a DP or a cinematographer. But visual effects artists are bringing as much. We’re making suggestions to story. We’re putting in backgrounds and designing things in the same way as them. It’s just recognizing that the visual effects teams bring that same level of artistic creativity to a film.

I really appreciate that. It’s great because he gives us room to explore those things. He won’t give you directions like, “Make that two points redder.” He’ll just say, “I want this sky to feel more melancholy or brooding.” So it’s up to us to interpret the emotions, and how to portray that, which is a lot of fun.

Another thing that struck us, there was a quote from Will Smith in which he said, “He’s a CGI character and now there’s a completely digital 23-year-old version of myself.” We wondered what you thought about the implications of that for the industry and for actors?

You know, any new technology – not just in visual effects, but in anything – you have to think about: what does this mean? What are the new implications?

In this case, going down the road, actors are going to want to think about the rights to their digital assets, and their virtual selves, and think about how that works. I do think from a practical standpoint, it will forever be cheaper and easier just to work with an actor.

But there’s no point in just putting Will Smith as himself. If you want 50-year-old Will Smith just film him – it’s better than the digital. It did take us two years and several hundred people working for all this time to pull off something that looks real.

People have to think about the story they’re doing. If they want to show a historical figure and they can look exactly like the person was, you’d have that technique and you’d have that possibility.

We do have to think about: what are the ethical lines? I’m sure audiences and people will help figure that out for us.

You said it took you two years and all those people, but now there is a 23-year-old Will Smith. What if they wanted to do some Men in Black prequel? Do you own the rights to that? Who owns Will?

Will still has the rights to his image. The data itself? I personally don’t even know where the rights stand – whether it’s a Paramount thing, or if it’s just something sitting on Weta’s servers. That’s a good question.

But you wouldn’t be able to just put Will Smith in a movie without him authorizing it. But you could have a young Will Smith movie.

It’s a completely fascinating concept, and who gets paid for the other person’s performance – and you built it.

What’s also very important is that the data is kind of an inanimate object until we get a performance from Will Smith. The reason Will is so charismatic is because he does these things subconsciously. He does what he does without thinking. If an animator had to go take that and do it themselves, we over-analyse everything. We aren’t Will Smith. It’s not a common thing, which is why actors are so unique.

Would it be possible then for someone else to play young Will Smith?

You could technically find an actor who could do it. But they won’t do it in the same way, unless you find the perfect mime.

A good example would be Peter Cushing on Rogue One. They had another actor playing him because Peter Cushing had died. One of the challenges was keeping the Peter Cushing of it all. Another actor moves their face in a different way. It’s just those little things. It’s amazing how millimetres make the difference.

I told you about the trouble with the brooding face. But other mechanical things: the way Will smiles; you know, if we weren’t exactly right there, and there was just something off – we’d be moving literally like a millimetre and that would make the difference.

Gemini Man is in theaters now.