The car as a concept was by no means fully developed. There is still a lot that can be improved – which will be possible to improve with the introduction of new technology.
Researchers at Volkswagen in Germany believe that augmented or mixed reality (augmented or mixed reality) will be among the most important components of future mobility concepts.
“We believe mixed reality information is the most intuitive information we can provide to improve the user experience of our customers,” says Andrew Klein, head of the computer science team at Volkswagen Group Innovation at Blog post published by Microsoft.
What you see there, and what you have to process, is very close to what people see and process naturally, so it’s not abstract, says Klein.
Few steps from HUD today
Like many other companies, Volkswagen for some time has equipped cars with a head-up display (HUD), which displays basic information on the car’s windshield, in the driver’s field of view. Then the driver does not have to look down at the dashboard to see, for example, speed or direction instructions.
A few years ago, VW wanted to use Microsoft’s MR glasses, the Hololens, for a research project, but that was too bad. The glasses lost their grip on the movements and the holograms that were normally displayed disappeared.
In 2018, researchers at Volkswagen began collaborating with Microsoft Mixed Reality and AI Lab in Switzerland. Then find out what the problem is.
became “car sickness”
Hololens 2, the latest version of Hololens, primarily uses two sensors to measure motion – cameras that detect visible light and the IMU (Inertial Measurement Unit), which measures acceleration and rotational speed.
The problem is that the combination of these sensors aims to detect how a human is moving. In the car, this is perceived differently. Microsoft compares it to car or seasick.
The main reason is that the IMU recorded the movements, while the cameras were unable to do so. Then it all collapsed.
So the mixed reality and AI lab had to develop an algorithm that models the discrepancies between the sensors, which means Hololens can keep track. Testing of this in passenger cars has been limited due to Covid-19 restrictions in Switzerland, so one of the senior engineers on the project, Joshua Elsdon, is said to have taken a lot of buses, trams and lifts in Zurich to test the technology. Many were tested at home in his apartment.
The system was later tested aboard some small charter boats near Microsoft’s headquarters in Redmond, Washington, before it was ready for testing with Volkswagen. A two-way data communication between the vehicle and the glasses is included.
– Connected to a GPS that tracks the position of the vehicle. In this way, we can also place 3D elements, such as points of interest, on the outside of the car. This opens up entirely new possibilities, not only by showing holograms in the driver’s front field of view, but also anywhere else the user of the glasses has seen, says Michael Wittkamper, augmented reality specialist at Volkswagen.
Not for consumers yet
Microsoft launched a few months ago what the company calls a “mobile platform mode” for the Hololens 2, which was initially intended for large ships. But the company plans to continue developing the technology for use in elevators, trains, cars and other moving environments.
Perhaps the idea now is not that motorists should wear relatively large and expensive Hololens when driving. But the technology may become relevant if there are smaller and cheaper versions in the future.
“Microsoft is not only interested in commercial devices like the HoloLens, but in the long run also in devices that make more sense for consumers in everyday life,” says Mark Bolivis, director of Microsoft Mixed Reality and AI Lab in Zurich.
– With augmented reality glasses, you can travel around the world, and then the information can appear in context wherever it is relevant. You can communicate with other people, too, in 3D, unlike what’s possible on a small screen, Polleveys says.
“Web specialist. Lifelong zombie maven. Coffee ninja. Hipster-friendly analyst.”