Mark Zuckerberg on Facebooks VR future: New sensors on Quest Pro, fitness and a metaverse for work – CNET

Five years after Facebook released its very first PC VR headset, and over a year into the pandemic, VR has been getting a closer look in a world where remote work has become standard and virtual life has become normal even without headsets.

I met with Mark Zuckerberg in person a year and a half ago to talk about the next steps for VR and the possibilities of augmented reality, just a few months before much of the globe went into coronavirus lockdown mode. Now, as the world is figuring out how to slowly reopen for business, I spoke with Facebook’s highest-profile VR advocate again — this time remotely — to talk about how his latest VR headset, the Oculus Quest 2, is doing. In a world of remote work where VR headsets still don’t fit into the picture too much — just 5.5 million headsets were estimated to be sold last year — I wanted to hear what Facebook’s CEO thinks will come next.

Zuckerberg says that the Oculus Quest’s greatest strength against its competition is its convenient wire-free experience, and that bringing the price down from the original $399 to $299 in October was a strategic move, intended to get more people to embrace VR. But Zuckerberg says he wants to upgrade the VR experience even more with the Quest Pro, a device that could include new sensors — face and eye tracking or maybe even fitness — in a higher-end self-contained system. The new sensors could add a greater sense of “presence” as part of Facebook’s plan for social VR. It could also come at a higher price, as Zuckerberg says, “there’s some ability for it to be a little more expensive.”

But the overall goal for Facebook right now, Zuckerberg told me, is to widen adoption so the world’s largest social media network can create more social opportunities for engaging in a virtual world. And he’s willing to lose money to win over people. 

“We’re not approaching this from the perspective of, how do we charge people as much money as possible and make profit on the devices?” Zuckerberg said in our 30-minute conversation. “What we saw was virtual reality is really about this sense of presence and therefore, it’s about social connection, more than it’s about whatever the technology is.” 

For Zuckerberg, this isn’t about resolution or processor speed. It’s about creating an immersive world to fall into. 

“We want to get as many people as possible to be able to experience virtual reality and be able to jump into the metaverse and … to have these social experiences within that,” he adds. “That’s really where our bread and butter as a company is in terms of building those experiences. That’s also what our business is.”

Facebook is getting closer to launching this world in the form of a large-scale social metaverse called Facebook Horizon that, with creative tools and user-created worlds, looks reminiscent of apps like AltspaceVR, Rec Room, and maybe even Fortnite, Roblox and Minecraft. Zuckerberg calls Horizon a “very big priority” for Facebook, something that will “play a big role toward helping to build out this broader metaverse that will go across all of virtual and augmented reality.” It’s an approach that feels similar to Microsoft’s.

Zuckerberg plans for Facebook’s employees, who will be able to work remotely in our new postpandemic, hybrid workplaces, to start testing Horizon. He says it’s an important “dogfooding” step for developing the platform more as it nears public launch sometime in the as-yet-to-be-determined future. 

Tellingly, we didn’t speak over VR or even over Zoom video, instead opting for a Zoom audio call. Zuckerberg says he doesn’t find Zoom meetings memorable or compelling. “I find that when I’m on a bunch of video calls, they all kind of blend together and I have a hard time remembering exactly which call something was said on or it’s just kind of harder to place it because there’s no real sense of space,” he notes. And though he admits that videoconferencing has its positives today — including higher video quality than you get in VR right now — he’s also confident that people will see the benefits of VR. 

“There are a lot of advantages to the presence that you get in virtual reality compared to the other modes of communication that we have. If we’re already there with the fidelity of experiences that are possible today, to me that just says, wow, in five years this is going to be clearly better on almost all of these fronts for a lot of the things that we do.” 

That VR future, however, still isn’t designed for kids, despite a growing number of children I know using the Oculus Quest at home. Zuckerberg doesn’t see a kid mode for Oculus VR being in the works anytime soon, either, but admits a large interest in VR for education overall.

Facebook is also working on more advanced AR glasses with wrist-worn neural interfaces, but that may still be many years away. Before that, an Oculus Quest Pro could bring more advanced sensors into Facebook’s VR ecosystem first. We also talked about the possibilities for what a step-up Oculus Quest Pro could bring next and what VR apps Zuckerberg spends time with at home. Below is our conversation, lightly edited for clarity.

It’s been five years since the Oculus Rift came out. Where do you see VR and AR tech for you now versus what it was even in 2019? Are things significantly changed? Or are there things that you wish were here but still aren’t?

It’s an incredibly exciting time for this. It’s pretty amazing to see how a lot of the aspects of the original vision, and what we hoped would play out here, are starting to fall into place. You know, it’s still a long-term journey. There’s still a ton that needs to get done over the next five-plus years to really deliver all the experiences that we want. But there are a lot of awesome pieces that are coming into place. And I’m excited to get a chance to talk about those today. 

At the same time, you’re right that with the pandemic and more people shifting toward being more remote more of the time, that’s just put even more importance on building technologies that give us a sense of presence, and that help us feel like we’re together and really get to connect naturally, whether that’s socially or professionally or for entertainment and playing games. That’s always been the promise of virtual and augmented reality. Unlike every other computing platform and type of screen that we’ve had to date, these platforms give you a sense of presence, like you’re right there with another person or in another place. That’s pretty magical. Every other communication tool that we’ve built up to this point is trying to approximate that, but virtual and augmented reality are the first ones that really deliver that sense of presence. And I know that is going to be increasingly important as the world, I imagine, will stay more remote as we come out of the pandemic.

You mentioned presence, and I think a lot about social [uses]. At the same time, I talk to some people and they don’t have any VR headsets. Other people I know are starting to actually buy them. I was curious where you see that right now. You don’t specifically mention sales numbers, but you’ve mentioned sales coming along, but not necessarily being as big as the Nintendo Switch. Do you think VR has achieved that social level you wanted?

We’re in the second generation of Quest now, and what I can say is that Quest 2 is doing quite well. It’s meaningfully outperforming even what we’d hoped for it. So that’s great. The Quest was where we really cracked the form factor and got it to be a wireless device that can do high-quality experiences. When you’re talking about virtual reality, in the sense of presence, there really is something that’s incredibly important about it being wireless. If you [have a] wire that’s wrapped around your neck or draped over your shoulder and it’s touching you, it really just breaks the whole illusion and sense of presence. It’s a big step forward in terms of quality of the experience, and it requires a lot of innovation to achieve that. What we’ve seen is that a lot of the other folks in the space haven’t been able to deliver that wireless experience yet, and I think that will likely continue to hold back some of those products. 

But we had the first-generation Quest, we improved on it with Quest 2. That’s going quite well. So what I look at is the trajectory of how these things are going, how is the next version after that going to go, and the next version after that. We have really exciting products in the road map for down the line that I just think are going to be really awesome. But you know, Quest 2 has been an inflection point for the adoption around this.

You mentioned some of the road map. And recently I heard a chat mentioning the existence of a Quest Pro. Is that something that would be for business? Or do you imagine there being possibly different levels of interest at different tiers for what the Oculus Quest could be?

This is certainly something that we’re working on. Basically, having a higher-end virtual reality experience. Traditionally, if you wanted to get a virtual reality device that had more power, the thing that you did was you wired [it] into a PC or some other computer, that’s one way to do it. But I think the trade-off on requiring the wire is too great in terms of the experience … what you trade off on immersion and being able to walk around — even if you’re sitting at a desk and doing productivity work. I don’t think you want that wire basically breaking the sense of presence. So even for Quest 2, we focused a lot on AirLink, which we just released. It’s the ability to now stream games from your PC, so you can take advantage of the power of the PC and still have a wireless experience, which is really important.

But there are other aspects that make virtual reality a higher-end experience as well, including putting more power in it in terms of different types of sensors and capabilities on the device. We do want to be able to support a wider range of use cases. I mean, it’s one of the things that’s been quite exciting with Quest 2 — seeing it broadened out. It’s still primarily gaming. But we’re starting to see the top few apps are social apps where people hang out together. We’re starting to see an increase in apps for creative production or productivity or people getting together to work. One of the things I’ve been pretty excited to see is this growth of fitness apps. So you see apps like FitXR and Supernatural, which are basically subscription services where you can take different classes doing boxing or dancing or different things. It’s almost like Peloton. It’s just kind of as easy to jump into, and you’re paying a subscription. Now you can do your workouts that way. 

From my perspective, it’s filling out the initial vision and hope that we had for VR about how there are going to be all these different use cases. It’s amazing for gaming, but it’s not only for gaming. Part of the question is if you were focused on building a higher-end device that could really max out further on some of those other use cases, in addition to doing the gaming pieces, there are some interesting questions about how you design. Now it’s not coming out anytime soon, but that’s certainly something that we’re excited about and having different products that basically can serve different use cases really well.

You mentioned fitness: It’s an area I definitely want to talk to you about because I’ve been using my [Quest 2] more for that. I see that it’s being positioned for that. I saw an ad in the New York Times talking about it as a fitness device, and I believe Facebook as a company is allowing expensing of it as a health and wellness device. Do you use it for fitness yourself?

I use all these different apps. I love Beat Saber. That’s one of my favorite things on Quest. I’ve certainly enjoyed FitXR as well. And I’m a big runner. So we don’t quite have that in VR yet. I am also a surfer and a foiler. I really want us to get a good experience where you can basically do the pumping part in VR, but we don’t quite have that yet. But I think in all these things, doing Beat Saber or FitXR or Supernatural, they’re real workouts. If I’m in Beat Saber, especially if I’m competing with one of my friends for half an hour or an hour, you definitely work up a sweat. You get tired by the end of that. So it’s pretty active. And I think it’s pretty clear why people really like it. 

Do you set up a space in your home that’s a dedicated VR zone where you do these things, and is there any time of day where you might do workouts with this?

One of the things about Quest, and it being wireless and stand-alone, is it is really portable, to be able to do it anywhere. So I have [it] in our living room and in our basement. To be honest, sometimes I’ll kind of go down there and there’s sort of bigger open spaces. But I’ll even just do it in our bedroom, where I probably have a more traditional, not particularly huge space to do it, but definitely enough. But again, I think this gets down to the form factor question. At the time we were getting into the experience, you had to be tethered to a desk or tethered to the same room as where your gaming PC was. [That] was just a bit more limiting for people and getting it to be more free-flowing is a very big advance in terms of letting people try it on in different places, making it easier to jump in.

That’s a big part of what we’re seeing here, and why we’re so dedicated to wireless as the form factor. All the things that we’re going to focus on, including the future Quest Pro work that we’re thinking about, that’s just going to be a killer part of it.

You also asked about routines. I have this one group of a bunch of my friends on a Messenger group thread. And it’s like our metaverse thread. Every weekend or so, someone pings the thread and is like, ‘Hey, do you want to play Onward?’ Or, ‘Do you want to play Population: One? Or do you want to play Arizona Sunshine?’ Those are a bunch of my favorite multiplayer games. That’s probably the closest thing that I have to a real ritual around this — kind of getting together with friends and going to do this. Over the last year, especially during the pandemic, when I couldn’t see a lot of these people in person, it was just really a neat thing to be able to do. [It] really drove home for me the value of being able to have those kinds of social and gaming experiences together.

I’m sort of getting into the same when you mentioned fitness. You mentioned sensors… it raises the question for me, do you think that there’s a chance of these fitness apps working more with watches and trackers? I know you’re working on wrist tech for neural inputs and AR. Does that open doorways for VR? And are you looking at more of a wellness direction for what fitness can do?

These are all really interesting questions. We are certainly working on the neural interfaces part and the wrist interface around that. Our hope is that eventually that works across virtual and augmented reality, and will be valuable across all these things. 

Getting back to your question around Quest Pro, there are a lot of sensors that would add different senses to the overall experience. We’ve talked a bit about things like eye tracking and face tracking, and you’re talking now about things like different health sensors, whether that’s heart rate monitoring or the different other kinds of fitness sensors that you might have on a fitness watch. The basic thing that these all have in common is that each of them takes additional compute power to power the thing. And the whole device needs to be tuned for that. So if you want to basically have a device over time that is just capable of all these things and is running an increasing number of sensors, you need to kind of get to higher- and higher-end devices. And then the question for us is going to be, well, how do we innovate on what that’s going to look like and be able to deliver something that’s a high-end product? 

And then also, how do we get it to be something that is really affordable for a very wide number of people, because our mission as a company is really to help connect everyone, right? 

Our approach to VR, is, rather than building a device and trying to sell it at a premium and make a bunch of money on the device, what we want to do is build a great experience and make it so that as many people as possible can experience this and can be part of this metaverse. And at the end of the day, we build experiences that are part of that and that will be the long-term business that we do. So I think the innovation on the sensor side, the compute side, to make sure that we can build devices that power these both at the high end, and devices that can be broadly available to everyone, that’s a big part of what we’re focused on over the next five years.

It sounds like affordability is a big part of that too, when you mentioned not climbing too high in price. Already the Quest 2 is reduced in price (from $399 to $299 today) and it’s gotten to a point where it’s game console-level, which is not something that other companies have been able to hit yet.

That’s right. I mean, getting to $299 on Quest 2 was a really big deal. That’s something we wanted to get to, the team worked really hard on that. I’m really proud of them. They did a lot of really hard work to be able to achieve that. And we wanted to see how that would affect accessibility for it. That’s been pretty good in terms of the results that we’ve seen there. But as you mentioned, at this point, even game consoles are more expensive than that. So I think there’s some ability for it to be a little more expensive.

But our bottom line on this is: We’re not approaching this from the perspective of, how do we charge people as much money as possible and make profit on the devices? We want to get as many people as possible to be able to experience virtual reality and be able to jump into the metaverse and then be able to have these social experiences within that. Then that’s really where where our bread and butter as a company is in terms of building those experiences. That’s also what our business is.

Speaking of social, you just launched a revamped version of social avatars. [Facebook] Horizon, which seems like Facebook’s metaverse, keeps approaching. Have you been spending time inside [Horizon]? I’ve had two demos in Horizon over the past couple of years. But I was curious if you’re spending time in there and whether that might be heading toward a launch?

Yeah, this is a big project for us because there needs to be a social fabric that goes across all of the different layers of virtual reality. That’s what we hope to do with Horizon. So part of it is we’re building this environment where individual creators can create worlds and you can hang out with your friends. Part of it is, we’re building out this avatar system that is going to get increasingly expressive on the one hand, and then if you want, also increasingly realistic. Although I think not everyone wants to be exactly realistic all the time, so you want to kind of offer both expressive and realistic. 

There are all these different services in this. But basically, that’s a big part of what we want to do around Horizon. And it also spans not just social use cases, it’s not just gaming. I think it’s also going to be work and collaboration and productivity, and that’s a big thing that we focused on. There are some interesting experiences in virtual reality now.

I have to say, one of the things that I’ve been excited about as we start thinking about what the policies are going to be around how employees start returning to the offices, and after the pandemic clears up, one of the things that I hope is that, going forward at Facebook, in addition to doing videoconferences and stuff like that, I want to basically have our culture be that a lot of our employees are holding meetings in VR, in something like Horizon. So that way, every employee of the company is kind of contributing to giving feedback, helping to tune and make those experiences better and better so that they can serve all these different use cases.

In the beginning, when we got started working on virtual reality, what we saw was virtual reality is really about this sense of presence, and therefore, it’s about social connection more than it’s about whatever the technology is. I would expect that as these things get built out more, whether it’s just use cases for hanging out and chatting, or playing different things together, or working together and collaborating, I would bet that those will be a lot of the biggest uses of this over the long term.

We’re very focused on just giving creators and developers the tools to build that with Horizon. It’s a very big priority for us. We’re not building it as just a single app or experience. We’re building it out as more of a platform that will enable people to build a lot of these different things over time. That’s why we’re building it methodically, and step by step. Maybe it’s taken a little longer than we would have thought to kind of have its first major, completely open release. But it’s a very important part of what we’re doing and the whole vision here. And I think it will play a big role toward helping to build out this broader metaverse that will go across all of virtual and augmented reality.

Do you see Horizon as a chance to rethink what the idea of what social media is for Facebook? Or do you also see … there are more Facebook elements coming into VR. Do you see that continuing?

It’s an interesting question. I certainly think that this is going to rethink what our perception of social experiences are. You asked about social media specifically, but I think social media is one category of social experiences, right? I don’t know if you’d consider, for example, WhatsApp to be social media in the same sense that you would say that Facebook or Twitter or YouTube are.

And so, similarly, I think that what you’re going to see with the metaverse and people interacting in virtual and augmented reality is it’s probably at least as different if not more from all these 2D-type interfaces, even though there will be some similarities. It is sort of an environment, an opportunity to kind of imagine what these social experiences can be in a completely different environment. This is a lot of what gets me really excited about this, is that I literally remember when I was a kid, in middle school, sitting in my math class, basically sketching in my notebook every day. I just kind of dreamed, while the teacher was going on and lecturing about something, about what I wanted to go home and build and code that night. And the tools didn’t exist yet to do this, but the ultimate thing that I really hoped to do one day was build out this kind of 3D immersive world where people can build different things. I feel like now that’s starting to become possible with all this technology. And I think that’s super exciting. 

So now, we’re literally able to start building and imagining some of these experiences that are like the holy grail of social experiences, because you’re going to be able to — with AR glasses in the future, when we’re having this conversation — you’ll be a hologram sitting on my couch next to me, rather than doing this over video or doing this over audio. Or in virtual reality, we can go into the same space. In a lot of ways meetings in VR today, or kind of hanging out, already feel more present and realistic than being on videoconference with someone because of the spatial audio. If someone’s to your right, you hear it coming from the right, you have a shared sense of the space, which you don’t when you’re on, say, a Zoom call for example, where everyone’s grid is a little bit different and all your meetings kind of look the same. I do think the social experiences here are going to be different, but pretty awesome. 

And I think getting a chance to build that from the ground up, not within sort of a box or platform that’s defined by other companies, who have their own sense of what a computer or a phone or something are, but really getting to design that whole experience from first principles around how people should be able to be present and connected with each other, is a lot of the most exciting work that we’re doing.

I think about that excitement — you bring up dreaming this as a kid, and we talked about using VR in the here and now, and what it’s becoming for people. I see a lot of people — I wanted to bring this up because my nephew wanted me to ask this too — I know a lot of people who are getting an Oculus Quest, and their kids are playing games in it. And it’s interesting, because I know there’s no Facebook account setup for under 13. Parents are doing stuff with them with it. But also I wonder how you feel about that, and if you see more of a role for it with kids or a kid mode. My 13-year-old nephew was asking me to ask you about if you’re going to be adding more things like that. Or a kid’s version of the headset at some point?

It’s a good question. And I imagine that is part of the full vision over time, we’ll have to address that more. But as you know, in order to use this, you sign in with your Facebook account. That way you can have all your friends there and have the kind of social experience that we’re trying to build. But you can’t have a Facebook account if you’re under 13. So I think it’s probably quite a ways off that we’d really build something like this. And there are also some pretty fundamental physical challenges with it. The device is designed for people who have a certain IPD (interpupillary distance) range, how far apart your eyes [are]. And different things like the weighting of the device are designed for people who have a certain amount of neck strength, for example. So not small kids, but at least people in their teens and adults. Those are things that I think will have to be overcome before you design even just hardware that I think really makes sense for younger kids to be wearing for an extended period of time.

But it is certainly interesting. I think over the long term, education is certainly going to be one of the really promising verticals here. We already see and I hear these stories all the time in higher ed. There was actually an experiment that was run, comparing heart surgeons with training in 3D, so that they can see the heart and see some of the things that they were doing, compared to people who had just been in lectures and experienced it in a more theoretical way. And my understanding is that the people who had the VR training generally performed better, which intuitively makes a lot of sense. So giving people the ability to do things hands-on and to experience them I think is going to beat being lectured to or just reading a book a lot of times in the future.

There are opportunities to build those kinds of educational experiences. Not just for the youngest kids, but even today, teens can use this, and people who are doing higher education can do this. There are even opportunities to do this in ways that are not traditionally what you’d think about as education. One view on communication technology is that they’re basically technologies around sharing a perspective. Some people describe books that way. Basically, books are a technology for sharing a perspective and trying to internalize someone else’s perspective. And there’s certainly film, and other things try to do that well. But in a lot of ways, I think virtual reality is the ultimate, because it literally lets you embody someone and walk in their shoes, and experience some of what they’re actually seeing and feeling around them. So I think that’s going to be pretty powerful for not just school-type learning, but culture and sharing each other’s experiences, and getting more empathy for what other people are experiencing around the world as well.

My kids are 12 and 8, and your kids are much younger. I don’t use VR that much with them at all. But I was curious if you ever thought of a moment when you might use VR with your kids?

I haven’t done that yet. Max is five now, and she sees me doing it and thinks that looks like I’m having a lot of fun. She has certainly asked if she can jump in. I told her when she’s older. But it’s an interesting question on all this stuff. The only other thing that I’d add, on top of all the challenges that we’ve talked about so far, is that part of the work that we do with younger kids — we work on Messenger Kids, for example, making it an experience that parents can really control — we do a lot to consult with experts to make sure that we’re doing this in a good way. I don’t think this is ever going to be something that we here at Facebook just decide, here’s how it should be for younger kids and therefore we’re going to go do it. This will be something that — this is not the top priority or near priority anytime soon, there’s a lot of other challenges that I think we need to solve to help expand virtual reality and help more people experience this — but I do think you’re highlighting what I think, you know, in kind of a 10- to 20-year future, I think people are going to want to use this in this way.

I think we’ll approach that by being more open with the community of educators and experts, and really taking their lead on what the right way is to approach this.

When you mention work in VR and aiming to get Facebook employees to work in VR, is that happening now? Do you find that you’re doing certain types of work in VR or are you setting up a sort of a routine for that with people right now?

Over the next several months, some more people are going to start going back into the offices, especially as vaccines ramp up. We’re trying to figure out what the new rhythm is going to be. That’s part of what I’m trying to figure out, exactly, how that’s going to all fit together. But for example, you can conceivably have a meeting that’s hosted in virtual reality, where some people who aren’t in virtual reality can videoconference in and be a part of the meeting, just like if you were in a physical meeting; you can have a screen, and people could be on that screen. I think being able to make it so as many people who are not together can feel like they’re present — and I think virtual reality can be a big part of that — that to me seems like a good direction for us to go in. And then given that this is such a big focus for our company, I really believe in dogfooding your own products. Which is I guess our technical term for eating your own dog food, which is basically saying if you’re in the middle of building out a product, what the best practice is is to use your product all the time. If we want to get better and better, then I do think we will be well served by having a lot of people inside the company, and outside, use it.

Some of the meetings that I’ve had in virtual reality so far are… it’s pretty good. It’s interesting and it’s different from video chat calls. Just to start off and be fair on some of the places where it’s not as good right now are, obviously if you’re on a video call, you can get a little higher resolution on the person’s face. We don’t quite have perfectly realistic avatars yet in VR in the way that we do if you’re on a Zoom call, for example. But there’s technology that we’re working on that will hopefully get there over the coming years. But then I think that there are all these things that are actually quite a lot better about meeting or being present in VR than even Zoom calls today. I mentioned this before, but a lot of how we as people process, even remember things, is through a shared sense of space. So if you’re sitting in a room with someone, if you’re on on my right, and we’re sitting on a couch, we have a shared memory where it’s like, all right, I remember that you were kind of sitting next to me, you’re on my right on the couch, and if you’re on my right, that means I’m on your left, so we kind of have a shared sense of what’s going on in the space, and all of our different memories — my visual memories of thinking of turning to my right and seeing you, my audio memory as I’m hearing the audio coming from the right — that stuff all ends up being pretty important in terms of imprinting memories, and feeling like this is a real experience where you’re present in a space together.

And you don’t get that on video calls today. I find that when I’m on a bunch of video calls, they all kind of blend together and I have a hard time remembering exactly which call something was said on, or it’s just kind of harder to place it because there’s no real sense of space. There’s certainly no shared sense of space. If you’re saying something and it’s not coming from my right or my left, and if you’re kind of in the upper right-hand corner on my Zoom screen, that doesn’t mean that I’m in any particular place on yours — there’s no shared sense of that at all. Even though the avatars aren’t quite fully defined yet — although we did just roll out a new avatar system, which is pretty good, and is certainly a big step in this direction — even without that piece kind of fully being in its final state yet, I still think there are a lot of advantages to the presence that you get in virtual reality compared to the other modes of communication that we have. If we’re already there with the fidelity of experiences that are possible today, to me that just says, wow, in five years this is going to be clearly better on almost all of these fronts for a lot of the things that we do.