[Software] What is xrOS? The Apple VR headset's rumored software explained

[Mr.Hacker]

 

The Apple VR headset is getting close to its rumored arrival at WWDC 2023 on June 5 – and the mixed-reality wearable is expected be launched alongside an exciting new operating system, likely called xrOS.

What is xrOS? We may now be approaching iOS 17, iPadOS 16 and macOS 13 Ventura on Apple's other tech, but the Apple VR headset – rumored to be called the Apple Reality One – is expected to debut the first version of a new operating system that'll likely get regular updates just like its equivalents on iPhone, iPad and Mac.

The latest leaks suggest that Apple has settled on the xrOS name for its AR/VR headset, but a lot of questions remain. For example, what new things might xrOS allow developers (and us) to do in mixed reality compared to the likes of iOS? And will xrOS run ports of existing Apple apps like Freeform?

 

Here's everything we know so far about xrOS and the kinds of things it could allow Apple's mixed-reality headset to do in both augmented and virtual reality.

 

It looks likely that Apple will launch its new xrOS operating system, alongside its new AR/VR headset, at WWDC 2023 on June 5. If you're looking to tune in, the event's keynote is scheduled to kick off at 10am PT / 1pm ET / 6pm BST (or 3am ACT on June 6).

This doesn't necessarily mean that a final version of xrOS will be released on that day. A likely scenario is that Apple will launch an xrOS developer kit to allow software makers to develop apps and experiences for the new headset.

 

While not a typical Apple approach, this is something it has done previously for the Apple TV and other products. A full version of xrOS 1.0 could then follow when the headset hits shelves in late 2023.

The software's name now at least looks set in stone. As spotted by Parker Ortolani on Twitter on May 16, Apple trademarked the 'xrOS' name in its traditional 'SF Pro' typeface in New Zealand, via a shell company. 

We'd previously seen reports from Bloomberg  that 'xrOS' would be the name for Apple's mixed-reality operating system, but the timing of this discovery (and the font used) bolster the rumors that it'll be revealed at WWDC 2023.

 

A report from Apple leaker Mark Gurman on December 1, 2022, suggested that Apple had "recently changed the name of the operating system to “xrOS” from “realityOS," and that the name stands for "extended reality". This term covers both augmented reality (which overlays information on the real world) and virtual reality, a more sealed experience that we're familiar with on the likes of the Meta Quest 2.

While xrOS is expected to have an iOS-like familiarity – with apps, widgets and a homescreen – the fact that the Apple AR/VR headset will apparently run both AR and VR experiences, and also use gesture inputs, explains why a new operating system has been created and will likely be previewed for developers at WWDC.

WHAT IS XROS?
Apple's xrOS platform could take advantage of the AR/VR headset's unique hardware, which includes an array of chips, cameras and sensors. It's different from ARKit, the software that lets your iPhone or iPad run AR apps. Apple's xrOS is also expected to lean heavily on the design language seen on the iPhone, in order to help  fans feel at home.

According to Bloomberg's Gurman, xrOS "will have many of the same features as an iPhone and iPad but in a 3D environment". This means we can expect an iOS-like interface, complete with re-arrangeable apps, customizable widgets and a homescreen. Apple is apparently also creating an App Store for the headset.

See more
Stock apps on the AR/VR headset will apparently include Apple's Safari, Photos, Mail, Messages and Calendar apps, plus Apple TV Plus, Apple Music and Podcasts. App developers will also be able to take advantage of its health-tracking potential.

Gurman says that the headset experience will feel familiar to Apple fans – when you put it on, he claims that "the main interface will be nearly identical to that of the iPhone and iPad, featuring a home screen with a grid of icons that can be reorganized". 

But how will you type when wearing the Apple Reality Pro (as it's rumored to be called)? After all, there probably won't be any controllers.

Spacetop computer used in public

The Sightful Spacetop (above) gives us a glimpse of how the Apple AR/VR headset could work us a virtual Mac display. (Image credit: Sightful)
Instead, you'll apparently be able to type using a keyboard on an iPhone, Mac or iPad. There's also the slightly less appealing prospect of using the Siri voice assistant. Apple is rumored to be creating a system that lets you type in mid-air, but Gurman claims that this feature "is unlikely to be ready for the initial launch".

It's possible that you'll be able to connect the headset to a Mac, with the headset serving as the Mac's display. We've recently seen a glimpse of how this might work with the Spacetop (above), a laptop that connects to some NReal AR glasses to give you a massive 100-inch virtual display.

WHAT APPS WILL RUN ON XROS?
We've already mentioned that Apple's AR/VR headset will likely run some optimized versions of existing stock apps, including Safari, Photos, Mail, Messages, Contacts, Reminders, Maps and Calendar. 

But given that those apps aren't exactly crying out for a reinvention in AR or VR, they're likely to be sideshows to some of the more exciting offerings from both Apple and third-party developers. 

So what might those be? Here are some of the most interesting possibilities, based on the latest rumors and what we've seen on the likes of the Meta Quest Pro. 

1. Apple Fitness Plus
An AR fitness experience on the Litesport app

Apps like Litesport (above) give us a glimpse of AR fitness experiences that could arrive of Apple's headset. (Image credit: Litesport)
Assuming the Apple AR/VR headset is light and practical enough for workouts – which is something we can't say for the Apple AirPods Max headphones – then it definitely has some AR fitness potential.

According to a report from Bloomberg's Mark Gurman on April 18, Apple is planning to tap that potential with "a version of its Fitness+ service for the headset, which will let users exercise while watching an instructor in VR".

Of course, VR fitness experiences are nothing new, and we've certainly enjoyed some of the best Oculus Quest fitness games. An added AR component could make them even more powerful and motivating, with targets added to your real-world view.

2. Apple Freeform
The Freeform app on an iPad on an orange background

(Image credit: Apple)
We called Apple's Freeform, which gives you a blank canvas to brainstorm ideas with others, "one of its best software releases in years". And it could be taken to the next level with a version of AR or VR.

Sure enough, Bloomberg's aforementioned report claims that "Apple is developing a version of its Freeform collaboration app for the headset", which it apparently "sees as a major selling point for the product".

Okay, work-themed AR/VR work experiences might not sound thrilling and we certainly had misgivings after working for a whole week in VR with the Meta Quest Pro. But mixed-reality whiteboards also sound potentially fun, particularly if we get to play around with them in work time.

3. Apple TV Plus
A basketball team scoring in a NextVR stream

(Image credit: NextVR)
Because Apple's headset will have a VR flipside to its AR mode, it has huge potential for letting us watch TV and video on giant virtual screens, or in entirely new ways. This means that Apple TV Plus will also likely be pre-installed in xrOS.  

Another claim from that Bloomberg report on April 18 was that "one selling point for the headset will be viewing sports in an immersive way". This makes sense, given Apple already has deals for Major League Baseball and Major League Soccer on Apple TV Plus. 

And while they're only rumors, Apple has also considered bidding for Premier League soccer rights in the UK. Well, it'd be cheaper than a season ticket for Manchester United.

4. FaceTime
Joining a call through FaceTime links in macOS 12 Monterey

(Image credit: Apple)
While we haven't been blown away by our experiences with VR meetings in Horizon Workrooms on the Meta Quest, the Apple mixed-reality headset will apparently deliver a next-gen version of FaceTime – and the Reality Pro's hardware could take the whole experience up a notch,

With an earlier report from The Information suggesting that Apple's headset will have at least 12 cameras (possibly 14) to track your eyes, face, hands and body, it should do a decent job of creating a 3D version of you in virtual meeting rooms.

We still haven't really seen a major real-world benefit to VR video meets, even if you can do them from a virtual beach. But we're looking forward to trying it out, while crossing our virtual fingers that it works more consistently than today's non-VR FaceTime.

5. Adobe Substance 3D Modeler 

Adobe has already released some compelling demos, plus some beta software called Substance 3D Modeler (above), showing the potential of its creative apps in VR headsets. Will that software's list of compatible headsets soon include the Apple Reality Pro? It certainly seems possible.

The software effectively lets you design 3D objects using virtual clay in a VR playground. Quite how this would work with Apple's headset on xrOS isn't clear, given it's rumored to lack any kind of physical controllers. 

These kinds of design tools feel like a shoo-in for Apple's headset, given many of its users are already happy to shell out thousands on high-end Macs and MacBooks to use that kind of software in a 2D environment.

 

https://www.techradar.com/news/apple-xros