“XR+AI” is unleashing its potential and unlocking more applications to welcome the new era

I remember that around 2016, when the VR virtual reality industry was in full swing, large VR headsets of various brands sprung up. The editor has also used VR to watch movies and play games. At that time, he happily thought this was the future. But within a year or two, the VR fire seemed to suddenly fade.

According to Chinajoy’s data, there were more than 70 VR/AR manufacturers exhibiting in 2016, and only 20 were left in 2017. Up to now, it seems that people have not seen amazing virtual reality products for a long time.

Is the virtual reality industry dying? In fact, this is not the case. According to the report of the 2019 World VR Industry Conference, from 2018 to 2023, the scale of China and the world’s VR industry will maintain a steady growth pace, and the current concept and technology of VR are also evolving. The current mainstream is already integrated XR (Extended Reality) of VR, AR, MR.

And XR is accumulating low-key energy, waiting for the opportunity to evolve the scale of the industry to a new level. This opportunity includes 5G and AI. In fact, the two are closely related. Regarding the empowerment of 5G on XR, the IT House has talked a lot before, but when it comes to AI, some friends may wonder. What intersection can XR and AI have? Today we will talk about it.

When distributed AI meets XR, there is a huge room for imagination
Let’s first think about the current pain points of XR. In fact, there are two most important ones: one is the bulky equipment, and the other is that there is too little to do besides watching videos and playing games, and it is playing games. It’s also stupid, with poor screens and stupid man-machine AI.

In the 5G era, due to the high speed and low latency of the network, many calculations and processing tasks can be completed by the cloud, which helps the thinner and lighter XR equipment. With the addition of AI, some more complex calculations It can also be achieved through collaboration between the XR device side and the cloud, so that the XR device can do more.

For example, with the assistance of AI, we use XR glasses to play games, the screen can be more refined, and the computer will not be so stupid when the man-machine is fighting. In addition, many things we have done on mobile phones can also be done in XR glasses. For example, wearing XR glasses to watch map navigation, and the route is displayed directly on the ground, which is an intuitive experience; and wearing XR glasses, you can Identify everything, such as telling us what a certain flower is, what is the name of a certain scenic spot, etc. These are all use cases that can be realized soon after XR and AI are combined.

So the question is, what kind of AI do we need to empower XR?
Speaking of AI, you may first think of various interesting applications on smartphones, such as face recognition, scene optimization when taking pictures, real-time voice translation, and so on.

At present, most of these applications appear on Android phones, and in the Android camp, most of these functions are provided by the AI ​​Engine of the Qualcomm Snapdragon mobile platform. Regarding AI Engine, IT House has introduced it many times before. For example, you can refer to “Unleash the Potential of 5G! Through Qualcomm, understand the importance of terminal-side AI” article to understand.

It is conceivable that on the XR platform, AI Engine will still play an important role. For example, Qualcomm incorporated the artificial intelligence engine AI Engine on the Snapdragon XR2 platform launched at the end of last year, and the AI ​​performance is 11 times higher than the previous generation.

So, is the AI ​​Engine on the XR platform simply “porting” the mobile phone? Obviously not. XR’s requirements for AI capabilities and power consumption restrictions are different from those on mobile phones. Moreover, even on the mobile phone side, the future direction of AI will be different from the present.

Why do you say that? In fact, we often say that the 5G era is like a big network, which includes TVs, cars, refrigerators, washing machines, air conditioners, drones, speakers, and even electric meters and water meters. , Also includes XR headsets.

In theory, these terminal devices should all have AI capabilities, or they are all intelligent. Only in this way can the 5G Internet of Everything truly construct a full-scenario smart experience.

The linkage of so many devices with each other requires powerful AI computing capabilities. What’s more, our requirements for “intelligence” will become higher and higher in the future, which can be achieved not only by the AI ​​Engine on the terminal.

What we call the AI ​​on the terminal is called terminal-side AI. As the name implies, it means that data collection, calculation, and decision-making are performed locally on the front-end equipment. The advantages are high privacy, high reliability and low latency. The disadvantage is AI Performance is limited.

The opposite of terminal-side AI is cloud AI. Its data training and processing are performed in the cloud, so large-scale AI operations can be performed, but there will be a delay compared to terminal-side AI.

In the era of 5G interconnection of everything, what we have to do is to integrate the advantages of cloud AI and terminal AI so that they can better collaborate. Thus, distributed AI came into being.

First of all, we must ensure that AI is distributed at the wireless edge, and at the same time, we must also allow a large number of terminals on the edge to reach the cloud with extremely low latency at any time.

Simply put, it is to let the cloud be close to the edge terminal, and store the big data training and delay-insensitive content and store in the cloud, and the specific execution of AI is handed over to the massive terminal of the wireless edge, both of which are at high speed and low latency. The 5G networks complement each other and collaborate with each other, making massive terminals smarter and smarter.

And Qualcomm is the pioneer and practitioner of distributed AI. They have efficient hardware based on Snapdragon heterogeneous computing capabilities, including CPU, GPU, DSP/HVX, as well as the most advanced neural network algorithms and software tools. On this basis, Qualcomm also launched its own distributed technology architecture. For example, one of the technologies is to train artificial intelligence systems on the terminal side through a distributed architecture of multiple terminals to form an edge cloud. Coupled with Qualcomm’s powerful 5G network, the terminal-side AI and cloud AI are seamlessly connected.

With the blessing of these technologies, Qualcomm can effectively combine edge cloud and terminal-side AI to promote the transformation of various terminals including XR in a positive and disruptive manner.

Specifically for XR, the XR terminal at this time can be enhanced in computing, storage, rendering, and content through the edge cloud to achieve the best immersive experience and more disruptive use cases, such as directly using XR devices Realistic large-scale 3A games, real-time online e-sports, and even video calls directly render the opponent’s 3D model in front of you, as if face-to-face communication, or in a more professional field, basketball players wearing XR glasses can simulate in real time Teammates on the field conduct tactical drills, or use XR to analyze the accuracy of athletes’ shooting postures in real time, etc., which brings huge imagination.

Linking upstream and downstream, distributed AI + unbounded XR transforms all walks of life
From the above analysis, it is not difficult to see that XR, AI, and 5G are actually closely connected. To sort out the relationship between each other, it should be like this: 5G’s high-speed and low-latency network makes distributed AI possible, and distributed AI drives masses IoT terminals are evolving in a more intelligent direction. Among them, XR is the most typical and first benefit terminal case. The combination of it and AI can give consumers and different industries a smarter, more interconnected, and more immersive future, and promote distance education. Changes in various industries such as training, medical treatment and equipment maintenance.

In this process, there must be a role that can integrate AI and XR, and then link the upstream and downstream of the industry chain to drive changes in various industries. Qualcomm is an active player in this role.

For example, the Snapdragon XR2 platform launched by Qualcomm brings distributed AI technology to the XR field, thereby helping device manufacturers and developers develop a rich variety of innovative devices and applications. Snapdragon XR2 is the world’s first XR platform that integrates both AI and 5G connections. Compared with the previous generation, the performance of its CPU and GPU are improved by 2 times, and the performance of AI is improved by 11 times. At the same time, it is also the world’s first to support seven channels. XR platform with parallel camera and dedicated processor for computer vision.

For example, in the education industry, in the physical geography class, when students wear XR glasses on the Snapdragon XR2 platform, the headset will perceive the environment around the students in real time, and then “transform” the surrounding rendering into a tropical rainforest environment based on the environmental characteristics. The students seemed to be in the rain forest, so they had a deeper understanding of various knowledge about the rain forest. Use cases like this are inseparable from the powerful AI computer visual perception capabilities and screen rendering capabilities, which require the cooperation of edge cloud computing and terminal AI to complete.

The same is true in the medical care industry. When a doctor wears an XR device equipped with the Snapdragon XR2 platform during an operation, he can see all the information provided by the XR glasses in the field of view, including the patient’s real-time vital sign data to patient-based examinations Numerically constructed 3D images of organs, etc., so that doctors do not have to divert their eyes as frequently as in the past, so that they can focus on surgery and increase the success rate of surgery.

Even the Snapdragon XR2’s head display can also be supported by powerful AI learning capabilities with line-of-sight object detection and classification functions to help doctors find parts that may need to be checked during the operation. This was unimaginable before.

In February today, Qualcomm launched the XR head-mounted display reference design based on the Snapdragon XR2 platform. This reference design can simplify complex technologies, help equipment manufacturers shorten the development cycle, reduce development difficulties, and launch corresponding terminal products faster.

And soon on September 16, the Snapdragon XR2 will be commercially available on the Oculus Quest 2 terminal. I believe that as the Snapdragon XR2 appears on more terminals, it can further stimulate the unlimited potential of XR devices. Promote the process of XR transforming all walks of life.

In addition to its technical and product efforts, Qualcomm is also actively establishing close cooperation with developers and other participants in the XR industry. They are optimizing the SDK of equipment manufacturers to help technicians make full use of the performance of the underlying chips such as the XR2 platform; Qualcomm also launched the Qualcomm XR Enterprise Program (XEP) to provide specialized training and guidance for outstanding XR industry developers and technicians. In addition, they also established the Qualcomm Development Network (QDN) developer community. In the community, developers We can learn more about Qualcomm’s technical routes and products in the XR field, and can directly access some development tools and resources to help them develop more creative applications on the XR platform.

“5G has great potential to transform industries and improve lives. AI technology will also further enhance XR services and applications, and push user experience to a new level. We believe that in the future, the in-depth integration of XR+AI+5G will drive a new round The industrial reforms that have helped many traditional industries to upgrade and transform, and transform entertainment, shopping and learning experience, will also have a profound impact on education, construction, industry, medical and other industries.”

At this year’s Qualcomm XR Ecological Partner Conference and the 2nd XR Innovative Application Challenge Award Ceremony, Qualcomm Global Vice President Hou Mingjuan said. This sentence is a good summary of what kind of industrial transformation power “XR+AI” will burst in the future, and how it will change our lives. And Qualcomm, relying on its leading technical strength and ability to integrate upstream and downstream resources in the industry, is pushing this era to us faster. I believe that in the future of intelligent interconnection of all things, XR will usher in its own era.

For more such interesting article like this, app/softwares, games, Gadget Reviews, comparisons, troubleshooting guides, listicles, and tips & tricks related to Windows, Android, iOS, and macOS, follow us on Google News, Facebook, Instagram, Twitter, YouTube, and Pinterest.