News Tech

The real-time motion capture behind ‘Hellblade’

Hellblade is built on Epic Games’ Unreal Engine. In early 2016, Kim Libreri, the company’s CTO, visited Ninja Theory’s offices to see how its latest project was progressing. Before joining Epic, Libreri was Chief Strategy Officer at Lucasfilm and worked on visual effects for more than 25 films, including The Matrix, Speed Racer, Poseidon and Super 8. “He invented bullet time in The Matrix,” Antoniades says simply. “But he’s not one of these hotshot, Hollywood-type people who look down on games. He’s a lifelong gamer who sees video games as being at the cutting edge of innovation.”

Libreri wanted to showcase Unreal’s capabilities with a real-time motion capture demo at GDC, a prestigious video game developer’s conference. Ninja Theory had the assets and collaborators to make it happen and immediately agreed to Epic’s proposal. “We thought it would be a cool demonstration of how game engines bring something very different,” Libreri says. “Normally, you would only associate that kind of fidelity — from an animation and lighting and texting perspective — with movies. And we were like, we can use pretty much the same techniques but do it live, because of the power of the Unreal Engine.”

The only problem? GDC was eight weeks away. Ninja Theory, Epic, 3Lateral, Cubic Motion and Xsens, a company brought in to handle body tracking, needed to move quickly. For Cubic Motion, it was particularly tough. Typically, the team takes hours to crunch, or “solve,” facial data gathered during a mocap shoot. “Now we had about sixteen milliseconds to track, solve and output that data to Unreal,” Barton explains. Thankfully, Cubic Motion had been working with Ninja Theory for some time and had been training its system to work with Juergens’ face. Still, it needed some refinements.

A week before the presentation, the system was barely working.

“When she’s driving it live on the big screen, there can be no tracking errors, certainly no catastrophic tracking errors, because that would just make her face explode, for example,” Barton says.

A week before the presentation, the system was “barely working,” according to Antoniades. All five companies spent the last three days in San Francisco fighting to iron out the kinks. “It was like an operations room,” he recalls. “I saw it as compressing two years of R&D effort by lots of different companies into a few weeks.” But everything came together. On the day, Juergens was able to drive Senua without any problems. Once the scene had ended, Antoniades explained that it was, in fact, an actor controlling the character live. The crowd went wild as Juergens sang “Do You Want to Build a Snowman,” dispelling any fears that the presentation had been faked.

Barton says he felt relief more than amazement or pride. “Because there are a lot of things that can go wrong in a real-time demo,” he says, “especially when it’s the first time anyone has done it at that level. So it was relief, but also a lot of pride that it came together in such a short amount of time.”

Later that year, Ninja Theory demonstrated the technology again at Siggraph, a conference for visual effects and interactivity. It was part of a real-time graphics competition that included Pixar, Industrial Light & Magic, Oculus, Square Enix and Uncharted developer Naughty Dog. For its second outing, Ninja Theory showed how it was possible to shoot, capture and edit a scene using performance capture and Sequencer, a cinematic editing tool that runs inside Unreal. In this version, Juergens performed twice in quick succession — once as Senua and a second time as a projection of her inner voice.

On the second time through, Juergens was able to act against her previous performance. Both takes were then combined inside Sequencer to create the final scene. It was enough to impress the judges and land Ninja Theory the award for Best Real-Time Graphics and Interactivity showcase. “We had just created a whole scene with two characters talking to each other,” Antoniades says. “Camera, framing, environment, everything. I think that really demonstrated how powerful it is.”

Source link

Post Comment