The Technicality of Augmented Reality

Augmented Reality

Augmented Reality is gaining significance in real-world applications. Investors and businesses are looking for the ways to reap the most out of the present AR technologies. In due course, some acknowledgment of technicality involved in AR systems and AR application development may help everyone interested in AR.

In the first part of Augmented Reality series, “Birth and Juvenility of Augmented Reality” we have come to know that what augmented reality is and how the technology evolved with the pace of time and reach at present status to yield sweet fruits of it.

Now, if we want to know how augmented reality works, we must have prior knowledge of the technicality involved in it. With awareness of technologies and tools taking part in the augmented reality application development, we can enable technologies to go into the mass and get instant adoption.

The technicality of AR involves mainly three major components to understand:

  • Hardware for AR
  • Software for AR
  • Algorithms of AR

Hardware for Augmented Reality

Augmented reality becomes possible with the evolution of its hardware components along with advancements in software and various algorithms to shape it in present status. There are numerous hardware components to list, but we will explore only chief hardware components used present days, and those are:

  • Display devices
  • Sensors
  • Input devices
  • Processors like computing devices ranging from desktops to mobiles

Display of AR

Rendering of AR demands various technologies and the combination of technologies such as optical projection systems, various types of monitors, modern handheld devices like smartphones and tablets and body attached or worn on the human body system.

We can list various display hardware in following ways:

Head Mounted Display (HMD):

  • Harness or Helmets loaded of sensors with six degrees of freedom
  • Eyeglasses loaded with cameras to intercept the real world and re-display it as AR through eyepieces
  • Head-Up Display (HUD) provides transparent display with data that saves users to stick with usual viewpoints
  • Contact Lenses consisting of IC (Integrated Circuit), LED, and antenna
  • Virtual Retina Display (VRD) is scanning display onto the human eye retina, and it seems floating display in the space
  • Eye Tap captures light rays going to the center of lens and substitutes it with synthetic light rays, which are computer-controlled

Spatial AR:

As its name suggests, SAR augments real-world objects in spaces without relying on display devices such as HMD, monitors, or handheld devices. For instance,

  • Shade Lamps
  • Mobile projectors
  • Virtual tablets
  • Smart projects

SAR display is becoming interestingly interactive and attractive, as it displays on any number of surfaces and settings and provides opportunities for great graphical visualization.


Tracking sensors & Networking Hardware need to work in combination to provide AR system with mobility. For example, modern smartphones and tablets have both. Mobiles consisting of:

  • Digital cameras and other optical sensors
  • Accelerometer
  • Gyroscopes
  • GPS hardware
  • Solid state compasses
  • RFID
  • Wi-Fi sensors
  • Native mobile connectivity hardware and sensors
  • Wired and wireless networking sensors and hardware

Input Devices

To make AR system interactive, various kinds of user inputs are mandatory and different types of input devices used such as

  • Keyboards for textual inputs.
  • Speech recognition systems like Siri, Cortana, Google Voice, and so on.
  • Gloves stylus, pointers, and other body wears with sensors to provide body gesture inputs.
  • Eye movement detection sensors and hardware.

Software & Algorithms for Augmented Reality

Find the best Augmented Reality Software should be capable of carrying image registration process where software is working independently from camera and camera images, and it drives real-world coordinates to accomplish the process.

How AR Software Works?

AR software can achieve augmented reality using two-step methods:

  1.   It detects Interest Points, fiduciary marker, and optical flows in camera images or videos.
  2.   Now, it restores the real-world coordinate system from the data collecting in the first step.

The First Step

#1 – Interest Point: The interesting point is well defined and has a well-defined position in image space. Image structure around the interesting point is rich in information content and simplify vision system. It also computed with a high degree of reproducibility.

It includes an attribute of scale to compute interest points from real-life images and with the scale of changes.

#2 – Fiduciary Marker: It is an object placed in the field of view of an imaginary. It uses as a point of reference, a measure, or it can place on the imaginary subject.

#3 – Optical Flow: it is a pattern of apparent motion of various objects, edges, and surfaces in a visual scene.

Moreover, the first step can use various feature detection methods such as:

1: Corner Detection: It is an intersection of two edges and used to extract certain features as well as infer the content of the image.

Moravec algorithm used in the corner detection where algorithm tests each pixel in the image to check the presence of corner. It defines the strength of corner using SSD (Sum of Squared Differences) between the patch and neighbors.

2: Blob Detection:

Blob is a region in the digital image which stands out by differing from the rests in terms of brightness, colors, and material.

3: Edge Detection:

It identifies points in a digital image where brightness sharply changes and display discontinuities. Edges are corresponding to discontinuities in depth, surface, material, and scene illumination. Thus, we can catch significant events and changes in properties of the world.

The Second Step:

It is a process of restoring real-world coordinates from the data, which we have obtained in the first step. In due course, we have to employ some methods such as:

  • SLAM (Simultaneous Localization and Mapping)
  • Structure from Motion methods including-Bundle Adjustment
  • Mathematical methods like-
    • Projective or Epipolar Geometry
    • Geometric algebra
    • Rotation representation with:
      • Exponential map
      • Kalman & particle filters
      • Non-linear optimization
      • Robust statistics

AR Programming Technologies

Augmented Reality Markup Language (ARML) has developed to define and interact with AR scenes. ARML consists of both XML syntax and ECMA scripts.

XML used to help in describing the location and appearance of virtual objects in AR scenes.

ECMA scripts binding permits dynamic access to properties of the objects in the virtual world.

ARML Object Model:

It was built on three main concepts:

  1. Features: Represents physical objects in AR scene.
  2. Virtual Assets: Represents virtual objects in AR scene.
  3. Anchor: Define spatial relationship between physical and virtual object in AR scene.

Anchors are four different types:

  • Geometries
  • Trackables
  • Relative to
  • Screen anchor

Augmented Reality SDKs

Just like other technologies, AR application development kits are available for rapid development process in the form of SDKs including:

  • CloudRidAR
  • Vuforia
  • ARToolKit
  • Catchoom CraftAR
  • Mobinett AR
  • Wikitude
  • Blippar
  • Layar
  • Meta
  • ARLab


Now, we are accustomed to various technologies, tools, methods, software and algorithms, AR programming language, and various AR application development SDKs.

However, technicality involves technical terms and may prove inadequate to understand how AR system works. Therefore, in the next part of the current AR series “How Augmented Reality Works” we are going to describe the entire process is highly comprehensive ways. Just wait for a week more!

Hemant Parmar is a veteran mobile app consultant. He is co-founder of the company. Thanks to his prolonged exposure to mobile application development projects for myriads of niches and industries, he is capable of providing high-end mobile app development consultancy. He is devoted to providing honest and transparent consultancy services for clienteles looking for righteous guidance to augment their niche services/products using the latest mobile technologies.

Leave a Reply

Your email address will not be published. Required fields are marked *