Size of Industry

$125,200,000,000

What is it?

Extended Reality (XR) is an umbrella term encapsulating Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), and everything in between. Although AR and VR offer a wide range of revolutionary experiences, the same underlying technologies are powering XR. Qualcomm’s vision for XR is sleek headsets that will transform everyday consumer experiences and many market verticals from industrial manufacturing and healthcare to education and retail.

HOW does it work?

1. Sight

This is probably the first sense that comes to mind when thinking of a sensory experience in XR. AR technology primarily focuses on what can be seen, as its main function is to overlay computer-generated information onto the real world. Pokémon GO is a popular example of this technology, enhancing the experience of reality by combining it with a world of Pokémon.

VR, on the other hand, allows for a full immersive experience in digital worlds. Fictional settings, like seen in many video games, are popular VR retreats, created through 3D modeling, animation, and programming. Through the same technology, one can experience a place that mimics the real world, like GTA’s Vice City, which is based off the real city of Miami. With 360° cameras, one can even immerse in relatively accurate representations of the real world. This is popular in the real estate industry with 360° virtual tours. VR experiences can take you many places, only a headset away. There are several VR headset options from companies like Facebook (Oculus), Sony (PlayStation), and Google (Daydream). Headsets can use rotational or positional tracking. Rotational limits the user to a viewpoint and location, like in a 360° video, while positional allows the user to roam more freely in a simulation, creating an even deeper immersive experience.

MR combines AR and VR technology to allow for visual experiences that merge computer-generation with a real surrounding environment. You can experience a digital simulation, like you would in a VR application, but placed in your true surroundings, like in an AR application. Spatial mapping technology can be used to generate a 3D map of the real context to use in conjunction with a digital application. MR experiences are obtained through MR headsets such as the Magic Leap 1 and Microsoft’s HoloLens 2. Below are snippet examples of varying MR applications that Magic Leap created for oil and gas company, BP.

2. Hearing

Hearing is the auditory perception of sound. Immersive audio is a more advanced 3D version of audio. The same way a 360° videocamera can capture an experience from all directions, 360° audio provides non-linear sounds that mimic how we hear things in real life. It will match the location the sound is coming from and come from that direction. If you are in a VR simulation and a band is playing on the right, you will hear the music coming from that direction, and as you walk closer to the band, the volume will increase as it would in reality. This allows for more accurate and immersive sense perception in XR applications through the use of sound.

The combination of audio-visual sensations could arguably be the most popular. These experiences are seen in AR, VR, and MR. One example is through videos. While we have been exchanging videos with one another for a while now, 360° technology has allowed us to share even more immersive memories. The video below is an example of an audio-visual VR experience of the Tomorrowland music festival.

3. Touch

Touch is the sensation derived when coming in contact with an object. While you may not actually come in contact with real objects in an XR experience, wearable devices, such as gloves and bodysuits, are used to simulate the perception of touch in virtual environments. This is achieved with haptic technology and feeback through forces, vibrations, and motions. You may have experienced basic tactile haptics while using a steering wheel controller in a racing game. The wheel uses vibrations and resistance to try and emulate the real world experience of driving a car.

Wearable technology can allow you to understand textures in digital environments. Force feedback can be used to understand an objects size and weight. The specific mechanisms and technology used to emulate touch varies by manufacturer. For example, the HaptX Gloves uses its own patented microfluidic technology that combines tactile feedback, force feedback, and motion tracking. Tesla’s bodysuit, Teslasuit, uses complex haptic feedback along with features such as Electric Muscle Stimulation (EMS), climate control system, motion capture, and biometrics to allow for unique immersive sensations. Biometry is interesting as it uses realtime data to relay information such as heart rate. This allows for the incorporation of emotional and stress levels within an experience. Wearable devices have multiple applications from creating experiences to analyzing performance. They can be used in sport enhancement, workforce training, and public safety simulations.

4. Smell

Smell is the perception of odors. Simulating scents is relatively new in the XR space. There are a handful of companies dabbling with this technology, and FeelReal is one of them. Currently on pre-order, their multi-sensory mask is a device that will be able to snap on to several popular headsets. It incorporates some touch sensations through vibration, water mist, and micro heaters/coolers, but uniquely aims to allow for the sensation of smelling. It uses patented interchangeable scent cartridges that mimic hundreds of smells. It claims to create a range of odors from natural scents like coffee, relaxing scents like lavender, to realistic scents like burning rubber.

5. Taste

Taste is one of the most difficult senses to recreate as it is derived from a chemical reaction on the tongue. Scientists and entrepreneurs are racing to emulate this sense in the XR space. While still in the experimentation phase, electrotherapy may be used to achieve this. There is potential to place electrodes on the tongue to emulate the perception of flavor. Scientists can manipulate heating and cooling to achieve this, or even electric muscle stimulation to create the perception of hardness or chewiness.

However, taste is an interesting sense. It is one that is heavily influenced by many other senses. Robin Dando, professor of food science at Cornell, describes how “when we eat, we perceive not only just the taste and aroma of foods, we get sensory input from our surroundings — our eyes, ears, even our memories.” Designers can get creative and uniquely incorporate tasting experiences in XR, without the need to recreate them. Project Nourished has been playing with this idea, using smell to help allude to memories of taste. They create 3D printed “food” made from algae that have aromatic diffusers to release desired scents. The James Beard Foundation approached this differently with their recent debut of Aerobanquets RMX, an XR experience that uses art to enhance and redefine dining. Instead of emulating taste through smell, artist Mattia Casalegno, asked questions like “how can you give color to a flavor?” and “what shape is taste?” to create visual experiences to enhance the flavor of the food.

6. Bonus: “Sixth Sense”

A sixth sense is an intuition that gives awareness, usually not obtained in normal perception. Machine learning and artificial intelligence (AI) can play a role in XR experiences, acting as an advisory sense. Take the medical field for example, where AR is being used by professionals. During practice, doctors can visualize computer data, such as from MRI and CT scans, to analyze patients without having to go beneath the skin. When these devices become embedded with AI capabilities, backed by large amounts of data, doctors will be assisted in making instantaneous life-saving decisions, such as providing the correct medicine, or performing the correct procedure.

Use Case

Display

XR needs a disruptive revolution in display technology to show richer visual content, and to switch seamlessly between real and virtual worlds.

Common illumination

Making virtual objects in augmented worlds indistinguishable from real objects is a tremendous challenge, especially under diverse lighting conditions.

Power and thermal

Meeting the always-on, compute-intensive workloads of XR within the power and thermal constraints of sleek XR glasses is very challenging.

Connectivity

Seamless and ubiquitous connectivity to the internet and cloud services is required for XR to reach its full potential.

Download the VR and AR push connectivity limits presentation

Motion tracking

Intelligent on-device tracking of our head, hands, and eyes is required to interact intuitively with our XR glasses and create immersion.

1. XR in business

Businesses are reaping the benefits of XR tech already, with immersive technologies as part of try-before-you-buy experiences. For example, if you’re shopping online for a couch you can see how it actually fits into your living room. Some retailers are using XR tech to allow you to get a preview of the real thing. That leads to a reduction in returns and even one-ups the brick-and-mortar retail experience.

Manufacturing is getting a boost from XR, too. AR can show how new factory layouts will look before they’re set up, and maintenance crews will soon create plant walkthroughs that make machines that need servicing stand out in vibrant red or orange. XR hands-on training can give step-by-step virtual experiences in real-world facilities.

2. Top XR games

Star Wars: Squadrons is a VR game that taps right into your childhood dreams, letting you pilot an X-Wing or TIE Fighter with a VR headset. Other top XR titles include Trover Saves the Universe, Half-Life: Alyx, and No Man’s Sky. You’ve probably already played one of the top XR games in history, Pokémon Go, which puts friendly little “pocket monsters” in your living room and neighborhood.

3. Best XR devices

XR tech is making inroads into our homes and businesses in both expensive gear like the Oculus Rift, and bottom-dollar tech like Google Cardboard. That’s a piece of easy VR tech that straps your smartphone to your face. Other notables include Playstation’s PSVR, with popular titles like Astro Bot: Rescue Mission, Tetris Effect, and Beat Saber.

Today’s XR devices include VR headsets like HP’s Reverb G2 VR, laptops like the HP OMEN, and AR displays like the Microsoft HoloLens 2, which can track your hand and eye movements for a seamless blend between the virtual environment and the real world.

4. XR you’re using now

You’re almost certainly already using at least one XR application regularly: Google Maps. Street view is technically XR, as is the satellite view you can use during navigation. You also use XR every time you watch an NFL game and you see that bright yellow first-down line on the screen. That’s actually not there in real life. When you do a Zoom call and someone uses a virtual background or an I-am-not-a-cat filter, that’s XR, too.

5. Future XR applications

Get ready for XR to change your life deeply in the next couple of years. You’ll see it pop up in sports (analyzing your golf game and giving pointers, for example) and in healthcare (showing your path through the hospital to the X-ray department with blinking arrows). In education, virtual field trips will broaden young horizons in new, exciting ways. You’ll even navigate tomorrow’s brick-and-mortar stores with AR maps that guide you to the products you’re searching for.

Market

According to the new market research report "Extended Reality Market with COVID-19 Impact Analysis, by Technology (AR, VR, MR), Application (Consumer, Commercial, Enterprises, Healthcare, Aerospace and Defense), Offering, Device Type, and Region (North America, Europe, APAC) - Global Forecast to 2026", published by MarketsandMarkets™, is expected to grow from USD 33.0 billion in 2021 to USD 125.2 billion by 2026, at a CAGR of 30.6%. The market growth can be attributed to several factors, such as surging deployment of extended reality in education sector and industrial training, rising demand for extended reality devices and technology in global automotive industry, surging adoption of extended reality in healthcare sector, and increasing demand for extended reality in entertainment and gaming industry.

VR devices to account for the largest share of Extended Reality Market in 2020

On the basis of device type, the market is categorized into AR devices, VR devices, and MR devices. The growth of the VR devices segment of the market is expected to be led by HMD. These displays from companies such as Sony, Oculus, HTC, Samsung, and Google have received an overwhelming response from users. The key applications of HMD are gaming and entertainment. Gesture-tracking devices have been an integral part of VR hardware owing to their risen use in consumer applications.

Consumer application to hold the largest share of the market during forecast period

On the basis of applications, the market is categorized into consumer, commercial, enterprise (manufacturing), healthcare, aerospace& defense, energy, automotive, and others. The consumer segment of the market comprises gaming and entertainment applications wherein AR technology is used to create 3D visual objects in the real world. The flourishing gaming and sports and entertainment sectors fuel the use of AR technology in consumer applications. VR technology also offers remarkable results regarding visual effects when used in gaming and sports broadcasts. The demand for HMD is high in consumer applications.

North America is leading the market extended reality in 2020

North America held the largest share in the Extended Reality Market in 2020. Growing use of smartphones, increasing adoption of smart electronic devices, and surging demand for extended reality technology in various applications are the key factors fueling the growth of the Extended Reality Market in North America.

Some of the key companies operating in the Extended Reality Market are Microsoft (US), Sony (Japan), Oculus VR (Facebook) (US), HTC (Taiwan), Google (US), Samsung Electronics (South Korea), Apple (US), PTC (US), Seiko Epson (Japan), and so on.