"Authenticity isn't just crucial to the success of this, it's crucial to protecting users as well," he says. You thought harassment was bad online? You thought VR, which adds embodiment and personal space to the mix, made it even more viscerally disturbing? You ain't seen nothing yet. Deepfakes, AI so powerful it can create faces from nothing, data privacy, misinformation campaigns, and toxic behavior have all become very real issues on a very real internet-and as VR and AR begin to make inroads toward becoming humanity's dominant communications platforms, funded by a social media company that's been at the epicenter of some of those issues, they'll become even more pressing. While Codec Avatars are still little more than a research project, we're learning about them at an uncertain time.
#VIRTUAL REALITY FALLS ON FACE SOFTWARE#
Known internally as Argent, these HMCs point infrared LEDs and cameras at various areas of the face, allowing the software to re-constellate them into the person’s likeness. While VR wearables of today are known as head-mounted displays, researchers at FRL have created a line of HMCs, or head-mounted capture systems. Regular users won’t have Mugsy and the Sociopticon in their living rooms, after all-they’ll only have their VR and AR headsets. As research scientist Jason Saragih tells me, the data has to be interpreted. As anyone who struggled through video compression woes in the early days of the internet knows, that's where the "codec" in Codec Avatars comes from: coder/decoder. The more information it captures, the stronger its "deep appearance model" becomes and the better it can be trained to encode that information as data-and then decode it on the other end, in another person's headset, as an avatar. The point is to capture as much information as possible (Mugsy and the Sociopticon gather 180 gigabytes every second) so that a neural network can learn to map expressions and movements to sounds and muscle deformations, from every possible angle. So my time in there is less about facial expression and more about what I'd describe as Lazy Calisthenics: shaking out limbs, jumping around, playing charades with Belko via webcam. Where Mugsy concentrated on your face, the Sociopticon helps the Codec Avatar system learn how our bodies move-and our clothes. (Before joining Oculus/Facebook, Sheikh established its predecessor, Panoptic Studio, at Carnegie Mellon.) The Sociopticon looks a lot like Microsoft's Mixed Reality Capture Studio, albeit with more cameras (180 to 106) that are also higher-resolution (2.5K by 4K versus 2K by 2K) and capture a higher frame rate (90Hz versus 30 or 60). If the word panopticon comes to mind, it should-though it would be better applied to the second capture area, a larger dome known internally as the Sociopticon. "Suck your cheeks in like a fish," technical program manager Danielle Belko tells me while I try not to succumb to paralyzing self-consciousness.
#VIRTUAL REALITY FALLS ON FACE SERIES#
In Mugsy, research participants spend about an hour in the chair, making a series of outsize facial expressions and reading lines out loud while an employee in another room coaches them via webcam. "It'll be big if we can get this finished," Sheikh says with the not-at-all contained smile of a man who has no doubts they'll get it finished. But the FRL team is ready to get this conversation started. At best, they're years away-if they end up being something that Facebook deploys at all.
They're also nowhere near being ready for the public. (They've since moved to a larger space near the Carnegie Mellon campus, with plans to expand again in the next year or two.) Codec Avatars, as Facebook Reality Labs calls them, are the result of a process that uses machine learning to collect, learn, and re-create human social expression.
Yaser Sheikh and his team are finally ready to let me in on what they've been working on since they first rented a tiny office in the city's East Liberty neighborhood. That's why I'm here in Pittsburgh on a ridiculously cold, early March morning inside a building very few outsiders have ever stepped foot in.
0 kommentar(er)
