Meta haptic glove prototype lets you feel VR objects using air pockets – The Verge

You cannot pet a dog in Meta’s new, high-tech virtual reality gloves. But researchers are getting closer.

Meta (formerly Facebook) is known for its high-profile moves into virtual and augmented reality. For seven years, though, it’s been quietly working on one of its most ambitious projects yet: a haptic glove that reproduces sensations like grasping an object or running your hand along a surface. While Meta’s not letting the glove out of its Reality Labs research division, the company is showing it off for the first time today, and it sees the device — alongside other wearable tech — as the future of VR and AR interaction.

At a simplified level, Meta’s haptics prototype is a glove lined with around 15 ridged and inflatable plastic pads known as actuators. The pads are arranged to fit along the wearer’s palm, the underside of their fingers, and their fingertips. The glove also acts as a VR controller. The back features small white markers that let cameras track how the fingers move through space, and it’s got internal sensors that capture how the wearer’s fingers are bending.

When you put on the glove and enter a VR or AR experience, a sophisticated control system adjusts the level of inflation, creating pressure on different parts of your hand. If you’re touching a virtual object with your fingertips, you’ll feel the sensation of that object pressing into your skin. If you’re gripping a virtual item, the long finger actuators will stiffen, creating a sensation of resistance. These sensations work alongside visual and audio cues to produce the illusion of physical touch.

The tech draws on the relatively new field of soft robotics, replacing bulky motors with tiny air valves. Meta has been working on it nearly since it acquired the Oculus VR startup in 2014. It developed its first prototype — one finger with a single actuator — in 2015.

One of the first experiences that Reality Labs head Michael Abrash recalls was looking at a virtual plate from inside a VR headset — where a single actuator, combined with the virtual image and the sound of rubbing the rough ceramic, was incredibly convincing. “I saw the plate, and I saw my finger on the plate, and I heard the sound — that kind of scraping sound across it — and I felt the vibration,” he says. “And I will tell you, I was running my finger over a ceramic plate.”

Simulated touch isn’t an entirely new phenomenon. Even combining VR with something as simple as controller vibrations can make people feel more like they’re touching something, and many companies have worked on wearables that either track users’ hands or provide haptic sensation. Some even integrate temperature sensations, which Reality Labs isn’t prioritizing.

But Meta is poised to mass-market haptic gloves in a way other companies can’t. Most haptics device makers sell specialized products for military, industrial, or academic institutions. By contrast, Meta produces the dominant consumer-oriented Quest VR system and is putting billions of dollars toward building a “metaverse” that integrates VR and AR. If it launches a haptic glove system, Meta can guarantee the system will work on the Quest and encourage app developers to use it.

Reality Labs imagines gloves being one of multiple controller methods for future glasses and headsets, alongside more lightweight solutions built on electromyography or EMG — a system that reads nerve signals on your arm and translates them into digital input. Meta acquired EMG wristband company CTRL-Labs in 2019, and the EMG team works separately from the haptic design team, although the two technologies could easily overlap.

“[Doug] Engelbart and Xerox PARC are the only time that fundamentally the way we interact with the digital world has ever changed,” Abrash says — referring to the invention of the mouse and the Alto desktop computer design in the 1970s, something that helped set the course of modern personal computing. “AR glasses are going to require that to happen. You’re not going to walk around with a keyboard and mouse. You’re not going to be taking out your phone to interact with it.” For some tasks, you might want a simple wristband. For others, you’d want fuller tactile sensations.

There are still many barriers to producing consumer-ready haptic gloves, though.

On one hand, the team wants to dramatically increase glove actuator density, going from tens to hundreds or even thousands within a few years. Right now, the gloves can offer a sense of objects’ contours, but not fine distinctions between surfaces — it relies instead on the suggestive power of audio and images. “You could pet a dog, but you wouldn’t feel the texture,” says Reality Labs engineer Katherine Healy. “You need high-density actuation to be able to really get that sensation, and this glove does not do that.” Meta might not be prioritizing something as specific as fur, but it wants to produce more broadly high-quality sensation.

On the other hand — or more literally, the same hand — the gloves need a dramatic size reduction. Meta’s prototype is svelter than something like a ‘90s Nintendo Power Glove, but that’s not saying much. Healy says the device needs to be light enough that people feel like they can normally interact with the real world as well as the virtual one, and it needs to go fully wireless, rather than relying on a tether like it does now.

To make things even more complicated, haptic gloves need to fit precisely against wearers’ skin. That could require designing differently fitting versions for each buyer, possibly through a process like 3D knitting, which can produce an object of clothing from a custom-sized digital design. Then there are practical questions like how to clean a high-tech glove — right now, it’s carefully wiped down with alcohol. “With garments in general, we expect them to be washable,” says Healy. “We would love to be able to create a glove that could be washed. How? We don’t know yet. But that’s part of our vision.”

If the gloves are commercialized, they’ll face the same privacy and security challenges as other wearables that collect biometric information. Abrash compares using technology like EMG wristbands to typing on a keyboard, something that’s rarely described as a potential threat. But wearable technology can analyze people’s physical motion at a fine level, while even basic typing patterns can predict the onset of diseases like Parkinson’s. Meta will need to establish policies governing how much of that data leaves users’ devices and who else has access to it.

For now, the team is just trying to work out how real is real enough for VR touch. “One of the interesting things about haptic gloves is that obviously we can’t reproduce reality exactly, which is different from audiovisual stuff,” says Abrash. A super-dense screen could be almost distinguishable from a real image, and a sound wave could capture the sensation of hearing somebody speak. But Meta can’t realistically make a glove that will stop your hand from going through a virtual table. Instead, it has to find the point where you’ll suspend disbelief and accept that the table is there, even if it objectively feels more like jello than hard wood. “There’s a new physics where nothing is solid on a large scale.”

And as for the dog? “It seems likely to me that if the haptic gloves work out, you’ll be able to pet a dog, but it will be a virtual dog and it will be a slightly different experience,” says Abrash. “But basically — emotionally, experientially — it will feel as real.”