Challenge

Mod was engaged to design a “final pixel” real-time character pipeline and mocap shoot where social media influencers could design their own avatar, live on-camera, and then perform on Australia’s first “3D” (trompe l’oeil effect) video billboard in Melbourne. While the talent were all seasoned online performers with large audience bases this was their first experience with mo-cap so they needed support with what would work best for the format.  

Solution

We built a web-to-game-engine system using Unreal Engine, readyplayer.me and Xsens so that 10 seconds after designing their web character, the performer was driving the character live on-set with face, body and hands mocap. We managed two parallel sets (one full-body, one head only). We recorded both mocap cinematics data and green screen video in parallel so that the editorial had quick access to material as well as flexibility in post.  

Impact

The project demonstrates the exciting opportunities around ‘digital humans’ moving between different virtual story worlds. In this instance, the target billboard was comparatively low resolution so the assets were re-rendered out of Unreal for higher quality. The robust on-set solution demonstrated the potential for next-to-real-time virtual production.