VIRTUAL PUPPETEERING

I made this demo to test out some ideas around animating digital puppets in real time with Unreal Engine. This is the same custom toolset I would eventually use to animate our old pal Barf.

The character’s limbs are driven by ragdoll physics, and the body is tracked to the movement of a Vive controller. Performance sessions are captured in VR. I wrote custom scripts for toggling between a closeup perspective for finer performance, and a zoomed out view for traveling across the stage.

The other fun piece of this tech demo is the virtual camera rig. I mounted Vive tracker to a simple shoulder rig. The position and orientation of the real world rig drive the camera in Unreal. The phone displays the virtual camera view in real-time.

Ideally, I would capture performance and camera moves at the same time, but since I was both performing and recording for this demo, I made use of the Sequencer tool in post to layer the takes I liked best.