Augmented Reality (AR): What Every Newbie Should Know
Augmented reality, the technology that combines elements from the real world with virtual elements, becomes more and more popular every year. The AR industry grows at a very rapid pace, attracting multi-million dollar investments. Many people expect AR to change the way we live the same way as it has already happened with smartphones and tablets. However, a few people know that augmented reality has been around for decades.
History of Augmented Reality
- In 1968, Harvard associate professor Ivan Sutherland developed and presented the first augmented reality and virtual reality head-mounted display (HMD). The system showed simple computer-generated graphics and was called “The Sword of Damocles.” Although the system was head-mounted, it wasn’t mobile but hung from the ceiling.
- In 1974, Myron Krueger presented a project called Videoplace. This solution combined video cameras with a projection system. It displayed shadows on the screen and made users feel like their environment was interactive.
- The term “Augmented Reality” was coined in 1990 by Tom Caudell, Boeing researcher.
- In 1992, Louis Rosenberg from USAF Armstrong’s Research Lab developed the first properly working AR solution. It was called Virtual Fixtures. A complex robotic system was aimed to balance an extremely slow 3D graphics processing speed. Virtual Fixtures were intended to improve productivity by overlaying information from sensors on a workspace.
The same year, one of the first AR systems was also used within a training program for US Air Force pilots. Shortly after, a team from Columbia University also developed their HMD with built-in trackers called Knowledge-based Augmented Reality for Maintenance Assistance (KARMA).
- In 1994, augmented reality was used in theater production. “Dancing in Cyberspace,” produced by Julie Martin, included virtual objects with acrobats dancing around them.
- In 1998, the 1st and Ten line computer system was used for the first time to display the virtual yellow down line during a live NFL game. Since then, virtual down markers have become a norm for all television channels that broadcast football games.
- In 1999, NASA integrated augmented reality in the X-38 spacecraft by using a hybrid artificial vision system. This solution was aimed to improve navigation.
The same year, naval scientists started working on BARS — Battlefield Augmented Reality System, which was a wearable solution.
- In 2000, Hirokazu Kato from the Nara Institute of Science and Technology presented ARToolkit, software that could combine actions from the real world and virtual objects.
- In 2009, ARToolkit became available in Adobe Flash.
- In 2013, Volkswagen presented its MARTA app which provides users with virtual repair assistance, enabling service workers to evaluate the repair process for a particular vehicle.
- In 2014, Google presented its Google Glass devices.
- In 2018, the global augmented and virtual reality market was worth $26.7 billion.
How Does Augmented reality Work?
The first crucial component of AR systems is computer vision. A computer should process information from a camera, “understanding” features of the real environment. Computer vision is a complex branch of computer science. Augmented reality requires the computer to understand the environment both in terms of 3D geometry and semantics: the system should not only detect positions of objects but also recognize these objects. Another component of augmented reality is realistically displayed digital content.
During the last few years, there were many developments in semantics thanks to the deep learning technology. Deep learning allows computers to recognize objects accurately, with no need to worry about the 3D geometry of the object. However, not all AR solutions actually need complex algorithms of recognition. Modern market of AR solutions relies heavily on the following three technologies.
- Recognition-based AR
Such solutions use cameras to identify objects or visual markers, such as QR codes or natural feature tracking markers. In this case, a device can display an overlay only when a certain marker is detected. Such systems can calculate not only the image of the marker but also its position and orientation. When the computer recognizes the marker, it gets replaced with a virtual 3D object. The analysis of the position and orientation of the marker also prevents users from seeing the virtual object from another angle or in more detail. If you rotate the marker, the virtual object will rotate as well.
SLAM stands for Simultaneous Localization and Mapping. This is the most effective approach that allows for rendering virtual objects over the real environment. Such systems simultaneously localize sensors, taking into account the surroundings, and map the features of the environment at the same time. SLAM allows developers to solve various problems associated with simulation. It includes various types of software and algorithms aimed for mapping and localization. Most AR development kits offer certain SLAM functionality.
- Location-based AR
Instead of collecting data from cameras, location-based AR systems use such sources of information as a GPS, accelerometer, velocity meter, or digital compass. Visualizations are only displayed based on the location. This type of augmented reality is often used in mobile applications, and this type of AR has made the technology very popular.
Benefits of augmented reality education, science, or entertainment are unquestionable. Games like Pokémon Go started a new trend in mobile gaming. However, this technology has been also widely adopted in many other industries, and the reason is that AR is beneficial for business. First, this technology is noticeable. It’s a great choice for companies that want to surprise their customers. In addition, AR apps quickly go viral and create countless opportunities for personalization.
Popular Applications of Augmented reality
- Education and Science
Interactivity makes AR solutions a perfect choice for educational or scientific purposes. Such an approach is especially effective when working with children, as their ability to learn to a large extent depends on their involvement in the learning process. For example, kids can interact with cartoon characters while learning numbers or alphabet.
However, AR is much more than a toy. It can help learn various practical skills, being a part of professional training programs. For example, SketchAR is an application that can teach you to draw. First, you have to draw just a few crossing lines to calibrate your drawing surface. After that, the program “projects” detailed sketches onto paper, showing them on the screen of your smartphone.
Health education students can learn anatomy using 3D holograms of human bodies. The Case Western Reserve and the Cleveland Clinic created such a program that is based on Microsoft HoloLens and allows students to study the human body in details with no need to go to a cadaveric lab.
- Travel Industry and Navigation
AR can expand capabilities of navigation apps, laying route suggestions and nearby objects over the real environment. Thus, such apps can serve as perfect guides, showing relevant information in an interactive manner. Developers of Clew went even further and used this opportunity to create an indoor navigation app for blind and visually impaired people.
Any tablet and smartphone can provide a shopping environment, no matter is it a brick-and-mortar or an online shop. For example, Harley-Davidson partnered with Marxent to create a shopping app for iPad users. This app allows users to create their custom bikes, trying different designs, lights, seats, and other options. IKEA used the same approach to create IKEA Place. With this app, customers can see how furniture will look in their house even before buying it.
- Modeling and Design
One of the main opportunities of AR for designers is that it can be used at early stages of development while perfectly demonstrating product functionality and form. An AR-focused company Augment created an app for Watermark Products, a well-known supplier of inflight products. This app allows designers to visualize product models at scale using tablets, with no need to create expensive prototypes. Clients can compare old and new products side-by-side, making an informed decision quickly.
- Maintenance and Repair
AR can help inexperienced people detect problems and perform repairs based on detailed step-by-step instructions. This approach allows companies to reduce downtime and associated costs. For example, Hyundai launched its AR owner’s manual when users scan various areas of their vehicle, they see 3D overlay images. The app also offers numerous video guides.
Main Trends of Development
After Apple presented its ARKit 2.0 at WWDC 2018, it became clear that mobile AR is as important as headset-based solutions, if not more important. Now developers have many tools for measuring, predictable tracking, and 3D object detection, along with a number of rendering improvements. The new format of AR objects, Universal Scene Description (USDZ) is now open to third-party software, making mobile apps the most promising area of the AR industry.
AR headsets don’t develop at such a rapid pace. There is no particular leader in this niche. Although Microsoft HoloLens remains the most popular solution, it still has certain usability issues. Nevertheless, HoloLens offers many opportunities for software developers, encouraging creativity.
The main priority for most developers is creating a natural user experience. Thus, companies put a lot of effort into improving gesture recognition while also creating more effective solutions for gaze direction tracking. In real life, we don’t need to turn our head to detect an object, instead, we can just look at it. Therefore, natural controls, seamless user experience, and minimization of tracking lags become the main goals for most AR developers.
While mobile games remain the most popular application of augmented reality, social AR also gains popularity, with Instagram and Facebook using AR to improve the user experience. Facebook has even launched its Facebook AR Studio that enables developers to create interactive solutions for the Facebook camera.
According to research, augmented reality apps are much more affordable than virtual reality apps, while also being three times more profitable. Thus, it makes sense to expect much more AR programs to appear in the nearest future, bringing benefits to businesses, and introducing new standards of user experience.
Where to Learn More About AR
Courses for Augmented reality
- A Beginner’s Guide to Augmented Reality with Unity
- Getting started with Augmented Reality
- Unity XR: How to Build AR and VR Apps Specialization
- Rapidly Build 12 iPhone/Android Augmented Reality (AR) Apps
- Build Location-Based Augmented Reality Apps with ARCore&ARKit
Books about Augmented reality
- Helen Papagiannis, Augmented Human: How Technology Is Shaping the New Reality
- Visualization and Engineering Design Graphics with Augmented Reality (Second Edition
- Gregory Kipper & Joseph Rampolla, Augmented Reality: An Emerging Technologies Guide to AR
- Robert Scoble & Shel Israel, The Fourth Transformation: How Augmented Reality & Artificial Intelligence Will Change Everything
- Dieter Schmalstieg & Tobias Hollerer, Augmented Reality: Principles and Practice (Usability)
Development Kits for creating own Augmented reality application