Figure 8: Markers in a store
It’s not just Pokémon, we’ll all be safer and happier
Augmented reality (AR) is the merging of real-time data with what you can see in the real world. AR is not an offshoot of virtual reality (VR), or vice versa, and, in fact, it has little in common with VR. AR is found in head-up displays in automobiles and airplane cockpits, in helmets worn by first responders and fighter-pilots, in smartphones and tablets, and in smart glasses. Perhaps, one day, AR will even be found in the form of contact lenses.
However, AR did not just happen overnight, and, in some instances, it hasn’t happened yet, even though we can fully predict it will. AR has been and will be the beneficiary of semiconductor technology developments that began with Shockley and Kilby.
As transistors got smaller and less expensive, they also used less power and could run faster. That made the process available for additional circuits, such as memory and sensors. Small, low-cost sensors enabled the smartphone revolution. Small, super-dense SoCs, combined with very small sensors and displays, will enable AR smart glasses.
Transistors that were 10nm in 1971 have shrunk to 7nm today, and 5nm is on the horizon. By comparison, a virus is a giant thing at 250–400nm. Today, chips are being built with over 30 billion transistors in them — 30 billion, that’s more transistors in one chip than 3.5 times the number of people in the world.
AR is also not just one thing, it’s found in many types of things, and in various locations, as figure 1 illustrates (below).
AR is used in the military, in hospitals, in industry, and in warehouses. And yet, it will be the consumer who becomes the biggest user and beneficiary. Focusing on the user, leads us to smart glasses.
Smart glasses — glasses that you and I would wear, out in the open, everywhere — can’t be conspicuous, dorky, or awkward. They must be no different than ordinary prescription eyewear or sunglasses.
Smart glasses are challenging to build and there is no definition of what constitutes a pair of smart glasses. It can be as little as an audio headset that gives text and email feeds, a camera, or a high-resolution color overlaid display. The parts that can go into a pair of smart glasses are shown in Figure 4.
For consumers to adopt AR smart glasses, two challenges have to be overcome: They must be attractive and they must be technically sophisticated.
Looking at an image in front of you, while simultaneously looking at the world around you, puts a strain on your eyes as your brain tries to compensate for the accommodation distance of the competing information.
The joke is you don’t have to be cross-eyed to see AR — but it helps. Vergence is the simultaneous movement of both eyes in opposite directions in order to obtain or maintain single binocular vision.
In addition to looking in and out at the same time, you must know where you are and where you are looking.
Simultaneous Localization and Mapping (SLAM) creates a map, without markers, by finding edges and highly contrasted points in your field of view. Tied to GPS, the wearer’s location, inclination (pose), and possible destination can be determined. All that is necessary in order to anchor information to locations, such as a coffee shop or store of interest, in the display and compensate for the wearer’s movements.
If it’s all done correctly, the system generates dozens to hundreds of fiducials for reference points to construct the map.
As Figure 8 (top) illustrates, illumination is a big factor. Smart glasses need to be worn inside in various lighting conditions, and outside in the bright sunlight.
Not only do you have to be able to see the real world and the messages and directions in the screen, you must be able to hear messages, or maybe entertainment or a phone call.
Smart glasses must also be able to hear and recognized alarms and alerts, provide voice recognition, and be able to distinguish between yours and other people’s voices — who’s giving instructions? They also must be able to detect the direction of sound source. And, one of the big hopes and applications for AR is real-time translations.
You must be able of course to see the real world, and the system should be able to do facial recognition and then provide data about the person it sees. It must also be able to provide unreal things superimposed on the real world, such as masking out stuff like advertisements, garbage, etc.
You’ll also want to see fantasy characters and objects as well as directional arrows and signs, and the distance to a destination or estimated time of arrival.
AR smart glasses should provide translated text of signs you look at and be capable of making invisible, dangerous things like heat from a room, person, or object, and mist, spray, fumes, or gas appear, so you can avoid them. You also want to see things behind you, such as threats, tardy children, or pets, and dropped or forgotten items.
With all that capability, we will live safer, better lives. There will be an indisputable record of who said and did what, the honest witness. We will never get lost or wander into the wrong place. We will never be rude, forgetful, or without information.
- Will it be intrusive?
- Will it remove us more from family and friends than a smartphone?
- Will it overwhelm us with information, demands, and requests? Pleas?
- Will it be subversive mind control, telling us what to like? Think? Where to go?
- Will we lose our identity? Our privacy?
- What does a world look like where we know everything about everyone, and they us?
- How do we behave — better, or as sheep?
We will have to learn how to use AR just as we learned how to use smartphones, TV, and PCs.
The good news is, this isn’t just science-fiction or wishful thinking. New products are coming out that meet some of the ambitions of smart glasses.
I think by 2025 a large percentage of people will be wearing some type of AR device, and they won’t look stupid for doing so. They will have an advantage over people who don’t have smart glasses, and that will accelerate the market. I can’t wait to get mine.
Learn more about how AR is changing the word in my book, Augmented Reality: Where We Will All Live.
Jon Peddie Research is a technically oriented marketing, research, and management consulting firm. Based in Tiburon, California, JPR provides specialized services to companies in high-tech fields including graphics hardware development, multimedia for professional applications and consumer electronics, entertainment technology, high-end computing, and Internet access product development.
