When you communicate in the real world, much of that is made up of the facial expressions you exhibit while talking to others. When you enter the world of Virtual Reality, it feels important to bring those facial expressions with you.
This challenge is not an easy one, given you have a headset strapped to your face. This important detail means that the traditional face tracking camera pointed at your face won’t work.
Facebook Labs have created a prototype headset with face tracking cameras. This embeds an array of wide-angled cameras to see parts of your face, track your movements, then re-assembles that as a single 3D representation of your facial movements.
The first implementation of this included as many as 9 cameras which obviously adds significant costs to the headset, instead the company turned to AI because, well its the answer to everything lately.
In this instance AI, the trained AI model understand what regular facial movements and gestures look like. When the content from the revised 3-camera tracking system, the AI can map the partial video content to a smooth, fully believable facial recreation.
Check out the video below for a full run through.
The technology certainly looks impressive, but may still be years before we see it implemented in Facebook-owned Oculus headsets. For now we have cameras and sensors looking out into the world to track motion controllers, but before long, there’ll be cameras pointed back at us.
This technology should deliver a much more interactive and more realistic VR experience, one that may be the door that unlocks the ability to have serious meetings in virtual spaces.
