New Approach to DIY Mo-Cap Unwraps at SIGGRAPH Real-Time Live!

Published on July 31, 2018

A fresh approach to do-it-yourself, full-performance motion capture will be one of the gamechanging solutions showcased at this year’s Real-Time Live! at SIGGRAPH. The session “Democratising Mocap: Real-Time Full-Performance Motion Capture with an iPhone X, Xsens, IKINEMA and Unreal Engine” will take place Tues., August 14 from 6-7:45 p.m. at the Vancouver Convention Center in West Building, Ballroom AB.

Cory Strassburger, co-founder of LA-based cinematic VR studio Kite & Lightning, will demonstrate how an iPhone X, used in tandem with Xsens inertial motion capture technology, can be used for simultaneous full-body and facial performance capture, with the final animated character live streamed, retargeted and cleaned via IKINEMA LiveAction to Epic Games’ Unreal Engine – all in total real time.

Taking a “Beby” character from Kite & Lightning’s upcoming Bebylon game, Strassburger will demonstrate this simple set up powered by accessible technology for real-time character capture and animation. He will show how the new approach does not rely on applying markers or setting up multiple cameras for a mocap volume, but only on an Xsens MVN system, a DIY mocap helmet with an iPhone X directed at the user’s face, and IKINEMA LiveAction to stream and retarget (transfer) the motion to “Beby” in Unreal Engine.

“Thanks to recent technology innovations, we now have the ability to easily generate high-quality full-performance capture data and bring our wild game characters to life – namely the iPhone X’s depth sensor and Apple’s implementation of face tracking, coupled with Xsens and the amazing quality they’ve achieved with their inertial body capture systems,” says Strassburger. “Stream that live into the Unreal Engine via IKINEMA LiveAction and you’ve got yourself a very powerful and portable mocap system – one I’m very excited to show off at SIGGRAPH 2018.”

The Real-Time Live! session will also cover the implications and potential of this process on future creative projects, revealing how new scalable workflows can empower games, CG, and animation industries at the indie level without the need for huge budgets.