
Ivan Sutherland
A new report from The Information corroborates and expands upon an earlier Bloomberg report claiming that Apple is preparing to launch a high-end virtual reality headset as early as next year, citing unnamed people with knowledge of the product.
Among the new revelations is that the new headset will allegedly feature two 8K screens (one for each eye) and that Apple has considered a steep $3,000 price point.
The headset (which the report says is codenamed N301) will be able to display rich 3D graphics at that resolution, the report says, thanks to an ultrafast M1 chip successor and because Apple will liberally use an already-known VR technique that involves using eye-tracking to render objects in the user’s periphery at a lower fidelity than what the user is focusing on.
This method is somewhat like a more advanced version of the already widely used frustrum culling technique in video games, wherein only what is visible on-screen at a given time is rendered, while objects behind the player character aren’t drawn until the character begins to turn around. This is done in order to optimize performance and put as much computing power as possible toward what the player is looking at.
Further, the new story says the headset will feature swappable headbands akin to the Apple Watch wristbands, an outward display that could be used to show content to people near you or to check information when the headset is not on your head, and a mesh fabric similar to what we’ve seen in the company’s HomePod speakers.
At least one version of the headset that Apple is testing includes more than a dozen cameras used for everything from tracking hand movement to delivering a live feed of the space around the user for mixed and augmented reality experiences, instead of just fully immersive VR ones.
The headset also features a lidar sensor, which Apple included in some recent iPhone and iPad models. Lidar scans the space in front of the sensor to quickly generate a 3D map for the placement of 3D objects, plus accurate shadows, occlusion, and more.
Apple is still testing different input methods, including things worn on the user’s hands or fingers, reading body and hand movements with the included cameras and sensors, and even a digital crown-line knob on the side of the headset.
Is this the real deal?
For Apple to launch a VR headset this soon (or at all) would be a surprising move, to say the least.
The company has no developer APIs for VR beyond nominal support for the SteamVR SDK in its custom Metal graphics API, even as it has spent years building up very robust AR APIs. CEO Tim Cook has also been publicly dismissive of VR in the past, citing its tendency to isolate users from those around them, among other factors. He has repeatedly pointed to augmented reality as Apple’s future focus.
Apple has been building up tools for the creation of augmented reality content since 2017. That’s when the company started using the iPhone and iPad—and their AR-enabled rear camera arrays—as a spawning ground for AR developers who could create experiences for a future app marketplace for AR glasses. As a result of that groundwork, Apple’s AR glasses could see an immediate wave of high-quality apps on release.
It’s harder to picture that situation with VR, so one is left wondering why Apple would focus on VR instead of AR to start when the groundwork is there for AR. And Apple is way behind in VR. The company made a half-hearted foray into VR with barebones HTC Vive support within Final Cut Pro and some other tools alongside the launch of the iMac Pro, but little was heard of that feature again.
While Apple worked closely with a very small cadre of VR developers during that short period in 2018, there is little, if any, publicly visible work being done now on VR software for Apple platforms. Additionally, Apple has done nothing to encourage it.
The arguments in defense of the “Apple is building an expensive consumer VR headset” narrative include the fact that Apple’s recent silicon wins foreshadow much better graphics performance in mass-market mobile devices than has previously been seen—and who knows how fast those chips will be by the end of 2022.
Also, some of Apple’s biggest past successes came from approaching markets that bore some resemblance to today’s VR space.
These were markets that carried a lot of promise, but early players had made only modest headway due to business or design limitations. Consider the iPod, which swept through a world of mediocre-at-best MP3 players; the iPhone, which transformed a smartphone marketplace that had previously only had niche appeal; the Apple Watch, which was far from the first smartwatch wearable but which was certainly the most successful; and even the personal computer itself back in the early days of the company.
VR is in a similar state to some of those examples before Apple’s entry in their spaces. It has niche fans, and it has some modestly successful products. But it’s hard to imagine the VR landscape as we see it now morphing quickly or at all into anything mainstream.
And the prospect of Apple bringing VR to the masses is undermined by the supposed price point of this device. $3,000 is not a mass-market price, no matter how strong the experience is compared to competitors.
Without an equivalent to the iOS and iPadOS AR app ecosystem and APIs to build on, it’s very difficult to imagine strong software support at the launch of this rumored device compared to those cheaper competitors, which would also hamper adoption by consumers.
It therefore seems plausible that these reports may be accurate but missing one critical caveat: that this is actually a tool being made and marketed to developers to kickstart that mixed reality glasses software-support journey, akin to the Apple Silicon developer kit Apple sent out to devs after last year’s WWDC.
It’s also possible—though it would be unusual—that it’s both a niche product for very high-end consumers and a developer tool meant to get the mixed reality software flowing.
Both The Information and the author of the previous Bloomberg story have proven to be generally reliable when reporting on upcoming Apple products in the past, so there is likely some truth to this story. But it’s nonetheless hard to believe that what we’ve seen so far is the complete story. It just doesn’t make sense yet.
So yes, it looks likely that an Apple mixed reality headset is coming. That said, it’s best to wait for more information before jumping to too many conclusions on exactly what form it will take.