Among the highlights of yet another fun, and somewhat overwhelming, San Diego Comic-Con was meeting the Teenage Mutant Ninja Turtles. No, not some eager fans in Turtles cosplay. We came face to face with the real Michelangelo and Donatello, albeit in a virtual reality experience built by Nickelodeon’s Entertainment Lab.
Let us explain. Nickelodeon offered press a chance to interview the voice cast behind the new Teenage Mutant Ninja Turtles series. The catch? The interview with Brandon Mychal Smith (Mikey) and Josh Brener (Donnie) took place in character in a virtual NYC.
The new TMNT series, Rise of The Teenage Mutant Ninja Turtles, looks completely unique, so stepping into this vibrant rendition of the Turtles’ NYC was an absolute treat. When I arrived, I selected an avatar for myself: Invader Zim’s head. With my VR headset and my giant Zim head screwed on correctly, I stood in front of a green screen. On the other side of the room, behind a curtain, Smith and Brener ad libbed an interview in character.
Watch the video here! And read on below for an explainer on how the technology worked.
“The virtual reality experience was developed using Unreal Engine. Creating this experience in a game engine allowed the ability to create real-time interactions and conversations. The experience streams in Mikey and Donnie puppets by using NewTek NDI Technology. NDI allows streaming of large video files over a shared network. The Mikey and Donnie puppets were created and are driven live using Adobe Character Animator. MIDI Keyboards, with mapped animation cycles, are used to trigger the various poses Mikey and Donnie can do. These MIDI Keyboards are puppeted live during the interviews. NDI allows the puppets to be streamed out of Adobe Character Animator, and picked up by various other machines which use these live animations for the experience and compositing.”