It’s a normal Tuesday. I’m in a room surrounded by Google and Samsung reps, wearing what look like regular glasses. One of them came up to me and started speaking in Spanish. I don’t speak Spanish. As I float through the air, I see her words translated into English subtitles. When I read them, I see that she is explaining what I am seeing in real time.
I mutter insults. Everyone laughs.
This is my first experience with Android XR. Android XR is a new mixed reality OS designed for headsets and smart glasses, like the prototype I’m wearing. It’s Google’s big bet to deliver a new generation of augmented reality devices that embody all of our wildest dreams of what smart glasses could be.
Google is no stranger to augmented reality. Google Glass crashed and burned in the public eye more than a decade ago, was then repurposed for enterprise users, and was eventually retired. But things are different now. Apple has Vision Pro. Meta has Ray-Ban smart glasses, and their AI features are garnering positive buzz. That’s why Google is re-entering the fray with Android XR.
Google wants everyone to know that the time is finally right for XR, and it’s pointing to Gemini as its north star. Adding Gemini enables multimodal AI and natural language. This allows for richer interactions with the environment. In the demo, Google prompted Gemini to answer the title of a yellow book on the bookshelf behind me. I saw it a while ago, but I didn’t take any photos. Gemini thought for a moment before giving her answer. I went around and checked – it was correct.
Additionally, the platform supports mobile and tablet apps from the Play Store out of the box. Today’s release is designed to help developers build experiences. Although the general public won’t be able to buy products running Android XR right now, Samsung is expected to launch its long-rumored XR headset in 2025. The headset, named Project Moohan (meaning infinity in Korean), will be the first consumer product to ship with Android XR. Technically, they run the same software as the glasses I tried, but Project Moohan also supports VR and immersive content, content that isn’t suitable for smart glasses. It’s essentially a showcase of everything that’s possible. That’s why Google is embracing XR. XR stands for “augmented reality” and is an umbrella term that includes AR, VR, and mixed reality.
Samsung’s headset feels like a combination of the Meta Quest 3 and the Vision Pro. Unlike both, light stickers are optional, so you can choose to incorporate your worldview. It’s lightweight and doesn’t squeeze your face too much. I love that my ponytail slips into the top easily so I don’t have to redo my hair later. At first, the resolution isn’t as sharp as the Vision Pro until the headset automatically adjusts to my pupil distance.
At this point I start to feel a sense of deja vu. Learn how to pinch to select items and tap the side to launch the app launcher. It has an eye adjustment process that is very similar to Vision Pro’s. If you want, you can switch back to immersive mode and watch YouTube or Google TV in the mountains in the distance. You can open the app, resize it, and place it in different parts of the room. We’ve all done this before. This just happens to be Google-flavored.
Let me ask you, how do you hope to stand out?
You don’t get that chance until you say “Gemini.”
For skeptics, it’s easy to scoff at the idea that it’s Gemini above all else solving augmented reality puzzles. Generative AI is having a good time right now, but not necessarily for the better. Outside of conferences where technology evangelists gather, AI is often viewed with derision and suspicion. But inside a Project Moohan headset or wearing prototype smart glasses? Here’s a glimpse of why Google and Samsung believe Gemini is the killer app for XR.
For me, it’s the fact that I don’t have to be specific when asking for something. Usually, I get flustered when talking to my AI assistant. You’ll need to remember the wake word, clearly express your request, and possibly specify your preferred app.
“One thing I’m really confident about is that Gemini is not just different, it’s really great,” says Kihwan Kim, vice president of Samsung Electronics. When I mentioned this, he nodded in vehement agreement. For Kim, the ability to speak fluidly with Gemini and the fact that Gemini understands an individual’s context opens up dozens of different options for how each person interacts with XR. “That’s why I clearly see this headset is going to give us more insight into what (XR) should be.”
When you say “take me to JYP Entertainment in Seoul” on your Moohan headset, Google Maps will automatically open and show you the building. If your windows become cluttered, you can ask them to rearrange them. You don’t have to lift a finger. As I put on my prototype glasses, I watch and listen to Gemini’s long rambling text messages summarizing the main points. In other words, can I buy lemon, ginger, and olive oil at the store? From speaking English, I naturally learned to ask about the weather in New York in Japanese, and I was able to get an answer in both spoken and written Japanese. It’s done.
It’s not just my interaction with Gemini that sticks in my mind. It’s also how you build your experience on top of it. I asked Gemini how to get somewhere and got turn-by-turn text directions. When you look down, the text has changed to a zoomable map of your surroundings. It’s very easy to imagine yourself using something like that in real life.
But while headsets are great, they can be a tough sell to the average person. Personally, I like the glasses demo better, but it doesn’t have a specific timeline. (Google built the prototype, but is focused on working with other partners to bring the hardware to market.) There are still cultural cues that need to be established with either form factor. there is. Outside of Gemini, we need a robust ecosystem of apps and experiences for the general public, not just early adopters.
“It’s not going to be a single product. It’s Android,” said Shahram Izadi, Google’s vice president of AR and XR, adding that Google has a three-pronged strategy for Android XR. states. Gemini’s conversation experience is something else. Third, we believe that the future of XR is not a specific device. For example, a headset may just be a “temporary” device that you use for entertainment. Glasses could complement cell phones and smartwatches, allowing for unobtrusive notifications and information retrieval.
“The way I see it, these devices don’t replace each other. You’re going to use these devices all day long, so if Gemini and the generated AI experience are consistent across these form factors, people “They’re going to get more comfortable with having a computer on their face. It’s a gateway to more immersive devices,” Izadi says.
After hearing Kim and Izadi’s story, I’d like to believe that. However, I am also acutely aware that all of my experiences were tightly controlled. I wasn’t given the freedom to try and break things. I couldn’t take a photo of the headset or glasses. I was carefully guided through a pre-approved demo that Google and Samsung believed to be fully functional in every respect. I, and every other consumer, won’t fully believe it until we can play these things without guardrails.
But even knowing that, I can’t deny that for an hour I felt like Tony Stark with Gemini as Jarvis. For better or worse, this example has shaped many of our expectations about how XR and AI assistants work. I’ve tried dozens of headsets and smart glasses that promised to bring what you see in movies to reality, but they completely failed. It was my first time experiencing something relatively close.
