Why you should embrace the extended reality continuum

Check out all the on-demand sessions from the Intelligent Security Summit here.

The extended reality continuum

The reality-virtuality continuum describes a digital reality space with endpoints of reality and virtual reality. Mixed reality is the spectrum between the endpoints. From a contemporary perspective, there are clear boundaries between reality and MR, and reality and VR. The MR spectrum is naturally and unreconcilably fuzzy and without well-defined boundaries. Reality and VR are discrete states, but MR is a non-linear gradient.

Extended reality is the spectrum from MR up to and including VR, or from another perspective, it is the reality-virtuality continuum excluding reality. We can call this subset of the reality-virtuality continuum the extended reality continuum.

Technological advancements and improved experience design will reduce the significance of the differences between MR and VR, causing users (even sophisticated ones) to become unaware of the differences. Classifications, as defined by the XR continuum, will have meaning only for designers and developers.

The impact on experience design

It is interesting to imagine that the distinction between MR and VR may not always be as straightforward as it is today. We are seeing early instances of immersive digital experiences being fluid and not distinctly MR or VR. This duality raises questions about how being both MR and VR impacts the quality of user experience.

For the sake of exploration, let us assume hardware and software exist to support both high-quality MR and VR experiences. The form factor is immaterial, but to aid the imagination, think of Geordi La Forge’s visor (Star Trek: The Next Generation), any helmeted character in Star Wars (Darth Vader, the Mandalorian, stormtroopers), or an implanted device (contact lens or eye replacements) like in Black Mirror’s “The Entire History of You” (S1E3).

Several questions immediately arise when considering the impact of multidimensional extended reality on experience design. What are the benefits and detriments of an experience having multiple postures in the XR continuum? Can a single experience successfully go back and forth between MR and VR? Can a single experience successfully cover multiple stops in the MR spectrum? How do we design MR experiences that are actual augmented reality? Where in the XR continuum is the best experience for my users and their problem space?

Here are some of my early speculative thoughts. Designers should declare to be either MR or VR and stay true to that posture, but this thinking should be challenged and validated. Switching between MR and VR, if possible, will likely be through well-defined modalities, or shift so slowly that the user is unaware of any change. However, attempting to be both MR and VR within the same experience is probably a design trap.

For now, it is impossible to do anything more than speculate and experiment. However, we can advance educated hypotheses using past wisdom from interactive experience design.

The impact on hardware

Device hardware will take on this dichotomy too. The most recent wave of stand-out devices, like Oculus, HoloLens, Magic Leap and smartphones (iPhone and Android), have a strong disposition toward either MR or VR. Each device optimizes to enable one experience type or another, but not both, and for good reason. Supporting both creates many technical challenges and can dramatically affect production costs, in addition to the previously mentioned experience design challenges.

However, this is changing, and future device iterations will support both MR and VR experiences. Devices like those from ShadowLynx and Meta are screen-based, with passthrough camera capabilities. Passthrough means the device uses an exterior camera to capture what the screen obscures, allowing the user to “see through the screen.” These devices can support MR and VR experiences with high resolution.

The Magic Leap 2 (ML2) can dim the outer lens to create a not-quite-opaque view of reality. This feature is more about improving the visual quality of rendered content and less about a meaningful attempt to enable VR experiences. MR devices fall significantly short of supporting a quality VR experience due to a limited field of view and an inability to block out the physical environment completely.

It is easy to imagine (and hope) for a generation of XR devices — any device capable of supporting any experience along the XR continuum — and not devices solely dedicated to MR or VR. Unfortunately, this may take several years and likely require different form factors than we have today. Still, there will continue to be a market for dedicated MR and VR devices. As the technologies become commoditized, low-cost or solution-optimized hardware will continue to singularly support either VR or MR.

Converging points

Embrace XR as the general anchor point for discussing MR and VR technologies, as it best fits the general expressions of MR and VR. The term supports discussions with a broad audience and within the creator community. There is a cognitive benefit in simplification for all. The extended reality continuum is a grounding structure for technical or detailed discourse that designers and developers require. Settling on a simpler lexicon lets us focus on much more interesting things.

Simplifying and coalescing terminology helps us move beyond the “Can we do this?” phase of an emerging technology and into exploring what experiences the technology can enable. Designers and technologists need to prepare for the full spectrum of XR experiences. Now is the time to explore the XR continuum and establish the experience design principles which will define the future and success of the medium.

We are beginning what I find to be the most exciting and exuberant phase of any emerging technology.

Jarrett Webb is technology director at argodesign