A Geek’s Guide to VR Storytelling

We currently sit on the brink of a new revolution in storytelling…and it’s cool as hell. Slowly but surely, VR is making its way into the home-entertainment sphere, with companies like Oculus, Sony, Samsung, and others pushing the VR experience on several fronts, from gaming to medical training to journalism and activism.

For movie fans, VR’s rise to prominence has brought forth a veritable universe of possibilities. Now, the transportive qualities of film have been amplified beyond belief, with VR headsets opening the door to narrative experiences that reach unprecedented levels of immersion and emotional interactivity.

That being said, VR is still in its infancy, with artists and engineers still learning and discovering every day on their way to unlocking the medium’s true potential. To demystify the new language of VR and walk us through the current state of VR moviemaking and innovation, we spoke to two of the industry’s leading VR storytellers, artists who, with their respective teams, live on the boundary of discovery and innovation in this wild, ever-fluctuating art form.

Our experts:

Saschka Unseld, Pixar vet and current Creative Director of Oculus Story Studio

Maureen Fan, CEO and co-founder of Baobab Studios alongside Dreamworks vet Eric Darnell

Note: Thanks to the San Francisco International Film Festival for their help in facilitating these conversations.

VR PHILOSOPHY

Baobab Studio’s latest animated film, Invasion!, is a War of the Worlds-inspired short set on a frozen lake that sees the viewer embody one of a pair of bunnies who together thwart an alien takeover. The short, narrated by Ethan Hawke, is emblematic of Baobab’s approach to VR, which is exclusively based in CG imagery.

“We only do animation,” says Fan of Baobab’s credo. “We think it’s way better than live-action. Live-action is kind of constrained by reality, and animation is only constrained by the imagination within the director’s head. Animation takes you to different worlds, and it makes you feel like you can reach out and touch it, [which is] basically the definition of virtual reality.”

Like Baobab, Unseld’s team at Oculus Story Studio believes in the power of animation and has a similar goal of inspiring wonder and amazement in their audiences.

”I think our philosophy is twofold,” says Unseld. “We put ourselves up with the motto to ‘inspire and educate.’” OSS’s latest movie, Henry, is a comedy that follows a loving hedgehog who loves hugs but has a lonely birthday party because of his balloon-popping spikes. “With every narrative VR experience that we do, we want to try something completely new. With Henry, the idea is to utilize comedy reminiscent of a traditional animated film and see how that works in VR.”

The second prong in OSS’s mission benefits the art form at large. ”When it comes to the word ‘educate,’” continues Unseld, “we would never say that, by ourselves, we could find out all of the different ways to utilize VR to tell stories. We want to share the knowledge that we have. We share things online—we share what we’ve learned, we share code, we share characters. Through our university outreach…we’re trying to help kids learn how to tell stories in VR.”

BEYOND GIMMICKS

Many of us who have tried VR associate it with horror games, rollercoaster simulators, or tech demos that swirl floating objects around your head. Emphatically, Fan believes the true beauty of the medium lies in the richness and emotional engagement of narrative experiences.

“We believe that, right now, a lot of the stuff going on in VR is tech novelty, gimmicky stuff,” says Fan. “There are a lot of tech demos and there’s this need to prove that VR is different. At the end of the day, when the novelty wears off, it’s going to be the stories that people remember.”

While VR movies have taken on many forms, including immersive journalism presentations, Fan believes the best way to push the medium forward is to create entertaining content for all demographics. “Right now, a lot of stuff out there is really depressing, to be honest,” says Fan. “There are, like, five films on Syria right now. I get doing it, because the topics themselves matter. Really, what we want to create is universally appealing stuff. That niche content is great for early adopters, but we want to bring VR to the masses. The way to do that is through storytelling.”

NEW LANGUAGE

The greatest challenge that faces studios like Baobab and OSS is that they’re essentially pioneers in their field, creating an entirely new storytelling vocabulary as they experiment and learn.

”Learning is a good way of putting it,” says Unseld. “Constantly feeling like we have no idea and we’re figuring it out as we go is probably more accurate. It’s like trying to make films without films ever existing at all before you. You just try and see what works.” The fact that this is a leader in the most prominent VR company in history talking speaks volumes to just how raw and untapped a medium VR really is.

With her partners at Baobab, Fan is exploring the fascinating ways VR affects the relationship between audience and character.

“My partner Eric felt that, while games are awesome, you don’t get the same amount of empathy for the characters as you do in a film,” explains Fan. “His hypothesis is, in a dark room with a projector, you get to be completely passive. You’re in the dark. You don’t matter. All you do is think about how the characters on screen are feeling. But when you [play a] video game, you start thinking, ‘Who am I? What do I need to do next? What do the game-makers want me to do?’ It reveals a man behind the curtain, and you’re no longer mesmerized by the experience. You’re splitting your attention between what you need to do and [connecting with] the characters.”

Eventually, Darnell hit a breakthrough while working on Invasion!.

“Eric wanted to experiment, and learned a lot of techniques,” Fan recalls. For example, he made [the viewer] a secondary character instead of a primary character so that you don’t feel like, ‘Shoot…I have to do all this stuff?’ We noticed that, when the bunny comes out to sniff you, you feel like, ‘Oh my gosh, the bunny’s acknowledging me.’ We saw people try to play with the bunny, play fetch and stuff.”

The team then figured out a way to make the connection to the friendly fuzzball even more immediate. “The bunny hides behind you. People were surprised at how much they cared about this bunny. They felt the need to protect her. You’re not worried about what the filmmaker wants you to do, but you’re still completely absorbed in the story.” This extraordinary sense of culpability is just one innovation makes VR unique.

REINVENTING THE WHEEL

To create their stories, studios like OSS and Baobab have developed new tools and methods that help them shape their ideas more freely.

“One of our guys built an engine [that allows all of us] to be in the scene at the same time,” Fan says of one of Baobab’s more innovative tools. “We can scrub through the animation, mark [things off]. We’re moving things around in VR together so that we can work inside [the scene].” Essentially, the team can inhabit and shape the virtual spaces they’ve created in a hands-on, almost godlike way. It’s kinda freaky, and it’s really, really cool.

The viewer’s freedom to swivel their heads and look wherever they like during VR movies like Invasion! is a liberating thing, but it naturally creates issues for the filmmakers. “It’s challenging because you don’t get to control composition,” says Fan. “Some people think you can’t do traditional storytelling in VR, but we believe you absolutely can. It’s about having content that’s interesting enough to get people to look where you want them to look. If you can’t get them to look where you want them to look, it just means you made really boring stuff.”

Directing the viewer’s eye is key to the VR moviemaking process, but with a bit of inventiveness, it’s completely possible to direct the viewer’s attention. In Invasion!, for example, Baobab uses some clever set design to guide our eyes to a specific spot. “There are these islands on the ice,” says Fan. “They’re purposefully put there because they create natural frames in the image.” Deceptively simple techniques like these blend seamlessly into the story and “point” to objects and characters of interest without us even noticing.

CUTS OF DOOM!

When Unseld started working with VR, he quickly discovered that many traditional filmic tools are inapplicable in VR. ”Our brains are so used to thinking in the frame of how film works,” says Unseld. No longer do VR filmmakers have direct control over what the audience sees, which either completely changes or outright obliterates many of the techniques handed down from generation to generation in film school. Unseld forges forward: “To get away from thinking in film or game patterns, I try to more open-mindedly explore what we can do in VR.”

Of all the film techniques, you’d think editing or quick cutting would be the first things left in the dust. The thought of getting zipped between different locations in rapid succession or even at a slower rate is stomach-turning for most people. But, astoundingly, cutting is one tool that actually translates quite well to VR.

”[Some say] you need to keep the camera still, but we don’t do that at all,” says Fan of cutting, zooming, and all other forms of spatial manipulation. “In the early days of cinema, when that train came out, people hopped out of their seats. Look what cinema has become. They’re saying the same things about VR that they did about cinema. The language developed.”

When asked about what it feels like to be in one spot one moment and in an entirely different spot the next, Fan is insistent that it’s a seamless sensation. ”It’s totally fine. People say cuts don’t work, and maybe now you think it’s a little jarring. I do think it takes time, but we’re fine with graduating people into that.”

Unseld, who’s experimented with cutting extensively, agrees. ”Editing fully works. It’s a tricky thing to figure out. People originally thought it wouldn’t work because, why would you suddenly be in a new place? It could be confusing. But the realization was that bad cuts don’t work. Bad cuts are really confusing. It’s about figuring out how good cuts work.”

As with most techniques, OSS found that old-school editing techniques only needed to be refined to work in VR. It’s more about the type of cut rather than the speed or number of them. “In a 360 degree environment, you kind of know where a person will be looking if it’s staged and shot well,” says Unseld. “What works really well, for example, is if you’re one hundred percent sure a person is looking at an actor and the rest of the [environment] is a desert, let’s say, you can easily cut to another shot if the actor is in the same spot. It doesn’t matter if it’s [somewhere else] in the world. That’s the kind of editing people are working on right now.”

A NEW MODE OF PERFORMANCE

Since the advent of film, there have been two predominant forms of acting: stage and screen. Now, with the sense of intimacy and proximity VR offers, there’s a third mode for performance animators and thespians around the world to master (should they choose to hop aboard the VR train, of course).

”On stage, you need to project through your voice and through your body,” says Unseld of the oldest acting style. “In a film, the close-up kind of rules supreme; much smaller things come across really well. But if you watch an actor on set…it’s very different on set than how it feels through a camera. What good actors have learned amazingly well is how to stage themselves towards the camera, what angle is good for their face, what movement will come across well on-camera. Compared to how real people behave, it’s a very active use of your body. In VR, that feels really strange. Toning it down even more is something actors will need to learn. It’s a new way to use your body to express emotions. That’s going to be a learning curve as well.”

Modes of narrative are also drastically changed in VR. “The protagonist in a movie is us; they are our way into that world,” Unseld explains. “In VR, the protagonist is a separate person from us. That kind of changes how slapstick comedy works, like when someone falls on their face. I laugh at Charlie Chaplin because, on some level, I associate myself with [him]. It’s like it happened to me, so I can laugh at it. In VR, that relationship to the protagonist is different; it’s clearly a different person. So the way I would react to someone else falling on their face is with worry rather than laughter.”

THE VR INVASION

Perhaps the greatest challenge VR evangelist face is the introduction of VR headsets as an essential component of home entertainment.

The tallest hurdle is obvious. “Besides the hardware being big, it’s the price point,” says Fan. VR headsets just aren’t affordable enough yet for most consumers to consider purchasing (the Oculus Rift will cost you $600 at launch), especially those outside of the gaming and tech end of the spectrum.

But, according to Fan, the more pressing issue involves capturing the attention of the casual consumer. Her last gig was as vice president of games at Zynga, so she knows precisely what she’s talking about.

“The thing I care about is the audience and if I can fit into their lives. People who play casual games don’t consider themselves gamers. It’s a few seconds in a Starbucks line, pushing a few things around. It’s about minimal effort. Needing to sit in front of a computer and devote thirty hours to this game is not mass market. Hardcore games are a smaller market, casual games are a bigger market, and TV and film is an even bigger market. It’s almost like, the more interactive you go, the more you ask the audience to do, the smaller the market is.”

Cultivating that kind of audience connection will surely take time, but companies have been doing their best to expedite the process. “On March 11th, Samsung gave away free Gear VR with every single Galaxy S7 sold, and they bundled our content with it,” says Fan of one of the biggest VR hardware promotions to date. “Ours was the most universally appealing content. We were really excited.”

Unseld agrees that getting interesting, quality stories into the hands of a wide variety of audiences is key. ”It’s about creating wonderful experiences and making things that people want to see. That’s the biggest challenge we at Story Studio have set out for ourselves.”

But there is a barrier to entry for first-time users that’s become a point of frustration for OSS and many a VR moviemaker. ”It is a big challenge when we’re trying to tell stories and people are distracted by seeing VR for the first time,” says Unseld. “Normally, that audience is not that ready to listen to a story yet because they’re having fun and are very distracted.”

It’s a real issue, though Unseld’s found it’s one that’s overcome with experience. “I think once someone has seen four to six VR experiences, they’re over the fact that it’s VR. It’s great because then we can start talking about telling different kinds of stories. We can actually start to move into not just talking about the medium but talking about individual experiences.”

ONWARD…

As VR storytelling blossoms and we see teams like Baobab and Oculus Story Studio continue to innovate, learn, and create the blueprint for VR filmmakers of the future, we owe it to ourselves as geeks and all-around lovers of movies to keep our collective finger on the pulse of this ever-developing art form. We’re lucky enough to witness the emergence of what could very well be the greatest thing to happen to movies since, well, the birth of movies!

For a trailblazer like Unseld, the joy of VR is that it makes the old feel new again.

”I like a sense of wonder and amazement,” he says with warmth in his voice. “It’s so hard to get that sense in a movie nowadays. People are so jaded and have seen everything. It’s nice that, in VR, you can have very simple things and excite people and give them a sense of wonder and magic. Having that ability back is incredible. Seeing people’s reactions when they put the headset on is beautiful.

Henry and Invasion! are out now.

Image Credits:

Invasion! courtesy of Baobob Studios

Henry courtesy of Oculus Story Studio