Research, Development, and Technology

Turner-Fairbank Highway Research Center

6300 Georgetown Pike

McLean, VA 22101-2296

PUBLICATION NO. FHWA-HRT-20-038 NOVEMBER 2020 

Leveraging Augmented Reality for 

Highway Construction
FOREWORD

Augmented reality (AR) is an immersive technology that combines computer-generated 

information with real-world imagery in real time. AR enhances the user’s perception of reality 

and enriches information content. In the context of highway construction, enriched content can 

help project managers and engineers deliver projects faster, safer, and with greater accuracy and 

efficiency. Navigating through the phases of a construction project allows managers to catch 

errors before construction and potentially improve design and construction details. Managers can 

also use AR tools for training, inspection, and stakeholder outreach. 

This study focused on documenting current AR technologies and applications, with an emphasis

on the state of the practice for using AR technologies in design, construction, and inspection 

applications for highways. This study included a literature review and interviews with 

researchers and vendors. This report is intended for State departments of transportation, the 

Federal Highway Administration, highway contractors and designers, and academic institutions 

involved in highway construction research. 

Cheryl Allen Richter, P.E., Ph.D. 

Director, Office of Infrastructure

Research and Development 

Notice

This document is disseminated under the sponsorship of the U.S. Department of Transportation 

(USDOT) in the interest of information exchange. The U.S. Government assumes no liability for 

the use of the information contained in this document. 

The U.S. Government does not endorse products or manufacturers. Trademarks or 

manufacturers’ names appear in this report only because they are considered essential to the 

objective of the document. 

Quality Assurance Statement

The Federal Highway Administration (FHWA) provides high-quality information to serve 

Government, industry, and the public in a manner that promotes public understanding. Standards 

and policies are used to ensure and maximize the quality, objectivity, utility, and integrity of its 

information. FHWA periodically reviews quality issues and adjusts its programs and processes to 

ensure continuous quality improvement.
TECHNICAL REPORT DOCUMENTATION PAGE

1. Report No.

FHWA-HRT-20-038

2. Government Accession No. 3. Recipient’s Catalog No.

4. Title and Subtitle

Leveraging Augmented Reality for Highway Construction

5. Report Date

November 2020

6. Performing Organization Code

7. Author(s)

Jaganath Mallela, Kevin Gilson, Paul Goodrum

(ORCID: 0000-0002-5656-1240), Jennifer Steen

8. Performing Organization Report No.

9. Performing Organization Name and Address

WSP USA Inc.

1015 Half Street SE, Suite 650

Washington, DC 20003

10. Work Unit No.

11. Contract or Grant No.

DTFH6117C00027

12. Sponsoring Agency Name and Address

Office of Infrastructure Research and Development

Federal Highway Administration

6300 Georgetown Pike

McLean, VA 22101

13. Type of Report and Period Covered

Final Report; October 2017–April 2020

14. Sponsoring Agency Code

HRDI-20

15. Supplementary Notes

Hoda Azari (HRDI-20; ORCID: 0000-0002-7340-0035) was the Contracting Officer’s Representative.

16. Abstract

Challenges in highway construction management and field operations include the lack of real-time and integrated

information, gaps between planned solutions and practical implementations, quality assurance, and effective

project communications. Three-dimensional (3D) model-based design and construction workflows are becoming

more common on highway projects, and the Federal Highway Administration (FHWA) is promoting these and

other innovations through its Every Day Counts program and Building Information Modeling efforts. This

increased use of 3D model-based workflows and rapid advancement in computer interface design and hardware

make augmented reality a tool for overcoming these challenges.

17. Key Words

Construction, augmented reality, AR, virtual

reality, VR, building information modeling, BIM,

Every Day Counts, EDC

18. Distribution Statement

No restrictions. This document is available to the public

through the National Technical Information Service,

Springfield, VA 22161.

http://www.ntis.gov

19. Security Classif. (of this report)

Unclassified

20. Security Classif. (of this page)

Unclassified

21. No. of Pages

100

22. Price

N/A

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized.
ii
iii

TABLE OF CONTENTS

CHAPTER 1. INTRODUCTION ................................................................................................ 1

Background and Significance of Work ................................................................................. 1

Research Objectives and Approach ...................................................................................... 1

Report Organization ............................................................................................................... 2

Overview of AR ....................................................................................................................... 3

AR Versus VR or MR ......................................................................................................... 3

Virtual Imagery and Scale in AR ........................................................................................ 4

Specific Opportunities for AR BIM in Construction .......................................................... 4

Collaboration in AR ............................................................................................................ 5

CHAPTER 2. AR TECHNOLOGY MARKET REVIEW ....................................................... 7

Classification of AR Technologies ......................................................................................... 7

AR Hardware ...................................................................................................................... 7

Display Characteristics ....................................................................................................... 8

User Discomfort with Display Technologies .................................................................... 11

Display Hardware ............................................................................................................. 12

Device Tracking—Sensing Movement Within the Real-World Environment ................. 13

Optical Tracking Configurations ...................................................................................... 14

Tracking Sensor Technologies .......................................................................................... 14

User-Controlled Positioning and Navigation .................................................................... 18

AR-Enabling Software and Applications............................................................................ 18

Computer-Aided Design and Drafting/BIM AR Applications ......................................... 18

Deploying AR Applications in the Field .......................................................................... 20

AR-Ready Products and Applications ................................................................................ 22

AR-Ready Viewing Devices––Handheld Tablet Devices ................................................ 22

AR-Ready Viewing Devices––HMDs .............................................................................. 23

AR-Ready Software Applications..................................................................................... 24

CHAPTER 3. CHARACTERISTICS OF AVAILABLE AR SYSTEMS ............................. 27

Introduction ........................................................................................................................... 27

Characteristics of AR Systems ............................................................................................. 27

AR Display Technologies ................................................................................................. 28

AR Information Display Options ...................................................................................... 29

Challenges for AR Systems in Construction .................................................................... 30

Data Support for AR Systems ........................................................................................... 32

BIM Model Workflows for the Construction Site ............................................................ 33

CHAPTER 4. STATE OF THE PRACTICE OF AR .............................................................. 35

Introduction ........................................................................................................................... 35

Current Applications of AR Systems In Highway Construction...................................... 35

Viewing Design Data in the Field with AR ...................................................................... 35

Bridge Lighting Design Application ................................................................................. 37

Autodesk BIM360 Docs Workflow .................................................................................. 38

AR Systems Supporting Inspection Activities .................................................................. 39

Tablet-Based AR Field Applications ................................................................................ 43

Conclusions ............................................................................................................................ 43
iv

CHAPTER 5. AR FOR TRANSPORTATION CONSTRUCTION WORKSHOPS ........... 45

Introduction ........................................................................................................................... 45

Workshop Highlights ............................................................................................................ 45

Workshop 1 ....................................................................................................................... 45

Workshop 2 ....................................................................................................................... 46

Results of the Polling ........................................................................................................ 46

Workshop Summaries .......................................................................................................... 46

Workshop 1 Summary ...................................................................................................... 46

Workshop 1 Results .......................................................................................................... 52

Plotting the Results of Workshop 1 .................................................................................. 53

Workshop 2 Summary ...................................................................................................... 55

Final Ranking of the AR Applications ................................................................................ 59

CHAPTER 6. AR FOR CONSTRUCTION MANAGEMENT, INCLUDING 

INSPECTION AND TRAINING .............................................................................................. 63

Introduction ........................................................................................................................... 63

Development of Data Flow and AR Technology Framework for the Use Cases ............ 63

Use Case Examples ............................................................................................................... 65

Use Case 1: AR Support of ROW Acquisition ................................................................... 65

Goals of the Application ................................................................................................... 65

Use Case............................................................................................................................ 66

AR Technology Framework ............................................................................................. 67

Use Case 2: Visual Variances ............................................................................................... 68

Goals of the Application ................................................................................................... 68

Use Case............................................................................................................................ 69

AR Technologies for This Application ............................................................................. 69

AR Technology Framework ............................................................................................. 70

Use Case 3: Nextgen Training and Certification ............................................................... 71

Goals of the Application ................................................................................................... 71

Use Case............................................................................................................................ 72

AR Technologies for This Application ............................................................................. 72

AR Technology Framework ............................................................................................. 73

Use Case 4: Inspector’s Toolkit ........................................................................................... 74

Goals of the Application ................................................................................................... 74

Use Case............................................................................................................................ 75

AR Technologies for This Application ............................................................................. 75

AR Technology Framework ............................................................................................. 76

Use Case 5: Automated Inspection ...................................................................................... 78

Goals of the Application ................................................................................................... 78

Use Case............................................................................................................................ 78

AR Technologies for This Application ............................................................................. 79

AR Technology Framework ............................................................................................. 79

CHAPTER 7. SUMMARY OF KEY OBSERVATIONS AND CONCLUSIONS ................ 83

Key Observations .................................................................................................................. 83

Conclusions ............................................................................................................................ 84

REFERENCES ............................................................................................................................ 87
v 

LIST OF FIGURES

Figure 1. Illustration. The reality–virtuality continuum. ................................................................ 3

Figure 2. Illustration. Typical human binocular FOV (120 degrees). .......................................... 10

Figure 3. Illustration. Current AR display FOV (typically 60 degrees). ...................................... 11

Figure 4. Photo. Handheld AR tablet device with GNSS tracking hardware. .............................. 22

Figure 5. Photo. User viewing outdoor site through an HMD. ..................................................... 23

Figure 6. Photo. 3D view overlaid on live video on a mobile device. .......................................... 25

Figure 7. Illustration. AR system architecture. ............................................................................. 27

Figure 8. Graphic. Display of design data overlaid on the video display of a tablet device. ....... 36

Figure 9. Photo. Display of 3D MEP overlaid on the video display of a tablet device. ............... 37

Figure 10. Graphic. Display of a 3D bridge lighting model in an AR HMD device. ................... 38

Figure 11. Graphic. Using an HMD to measure a target object. .................................................. 40

Figure 12. Graphic. 3D surfaces and points as captured by an AR HMD (Moreu 2018). ............ 41

Figure 13. Graphs. Results of postprocessing surfaces and points as captured by the AR 

HMD device (Moreu 2018). ................................................................................................ 41

Figure 14. Graphic. Surface and points captured in ramp grade measurements made in the 

field with HMD (Moreu 2018). ........................................................................................... 42

Figure 15. Graphic. Screen capture of an AR tool for measuring cracks in concrete 

structures (Moreu 2018). ..................................................................................................... 43

Figure 16. Graph. Overall polling results based on workshop 1. ................................................. 53

Figure 17. Graph. State and Federal respondents’ polling results based on workshop 1. ............ 54

Figure 18. Graph. Engineering and contractor respondents’ polling results based on 

workshop 1. ......................................................................................................................... 54

Figure 19. Graph. Research and developer respondents’ polling results based on 

workshop 1. ......................................................................................................................... 55

Figure 20. Graph. Overall polling results based on workshop 2. ................................................. 57

Figure 21. Graph. State and Federal respondents’ polling results based on workshop 2. ............ 57

Figure 22. Graph. Engineering and contractors respondents’ polling results based on 

workshop 2. ......................................................................................................................... 58

Figure 23. Graph. Research and developer respondents’ polling results based on 

workshop 2. ......................................................................................................................... 58

Figure 24. Graph. Overall polling results based on both workshops 1 and 2. .............................. 60

Figure 25. Graph. State and Federal respondents’ polling results based on both 

workshops 1 and 2. .............................................................................................................. 61

Figure 26. Graph. Engineering and contractor respondents’ polling results based on both 

workshops 1 and 2. .............................................................................................................. 61

Figure 27. Graph. Research and developer respondents’ polling results based on both 

workshops 1 and 2. .............................................................................................................. 62

Figure 28. Flowchart. AR technology process.............................................................................. 64

Figure 29. Screenshot. Mockup of interface for an AR ROW application. .................................. 66

Figure 30. Screenshot. Mockup of interface for AR visual variance application. ........................ 69

Figure 31. Photo. Mockup of interface for AR training application. ............................................ 73

Figure 32. Screenshot. Mockup of interface for the inspector’s toolkit application. .................... 75

Figure 33. Screenshot. Mockup of interface for automated inspection application. .................... 79
vi 

LIST OF TABLES

Table 1. AR display types and their advantages and disadvantages. .............................................. 8

Table 2. Tracking methods and their advantages and disadvantages. .......................................... 15

Table 3. Comparison of optical specifications and uses for HMD AR Devices........................... 24

Table 4. Comparison of HMD field measurements against county measurement. ...................... 42
vii 

LIST OF ABBREVIATIONS

2D two-dimensional 

3D three-dimensional 

4D four-dimensional 

ADA Americans with Disabilities Act

AEC architecture, engineering, and construction 

AI artificial intelligence

AR augmented reality

AU Autodesk University

BIM building information modeling

CADD computer-aided design and drafting

CAVE cave automatic virtual environment 

CIM civil integrated management

DOT department of transportation 

E&C engineering and construction 

EDC Every Day Counts 

FHWA Federal Highway Administration

FOV field of view

fps frames per second

GNSS global navigation satellite system

GPS Global Positioning System 

HMD head-mounted display 

IowaDOT Iowa Department of Transportation

iOS iPhoneTM Operating System 

IR infrared

LOD level of detail

MEMS microelectromechanical systems

MEP mechanical, electrical, and plumbing

MR mixed reality

NCHRP National Cooperative Highway Research Program 

NDOT Nevada Department of Transportation 

NEPA National Environmental Policy Act 

ODG Osterhout Design Group 

OJT on-the-job training 

OS operating system 

OST optical see-through 

QA quality assurance 

QC quality control 

QR Quick Response 

R&D research and development

ROI return on investment 

ROW right-of-way

SDK software development kit 

SLAM simultaneous localization and mapping 

STA State transportation agency
viii

TRB Transportation Research Board 

UDOT Utah Department of Transportation 

UI user interface

VIO visual-inertial odometry 

VR virtual reality

VST video see-through 
1 

CHAPTER 1. INTRODUCTION

BACKGROUND AND SIGNIFICANCE OF WORK

State departments of transportation (DOTs) are charged with providing a safe, efficient, 

accessible, and convenient transportation system that meets vital national interests and enhances 

the quality of life for users of State DOT facilities, today and into the future. To that end, 

highway projects must be delivered with a high standard of quality, with adaptable and scalable 

processes allowing for future innovations, and be effective at meeting public expectations and 

State DOT goals.

Three-dimensional (3D) model-based design and construction workflows are becoming more 

common on highway construction projects, and the Federal Highway Administration (FHWA) is 

promoting these workflows and other innovations through its Every Day Counts (EDC) program 

to identify and deploy innovative solutions to accelerate the delivery and cost effectiveness of

highway projects, enhance safety, and protect the environment (FHWA 2019). 

Challenges in highway construction management and field operations include the lack of 

real-time and integrated information, gaps between planned solutions and practical 

implementations, quality assurance (QA), and effective project communications. The increasing 

use of 3D model-based workflows and the rapid advancement in computer-interface design and 

hardware have made augmented reality (AR) a technology that can help overcome these 

challenges.

AR is an immersive technology that combines computer-generated information with the real 

environment in real time. AR enhances the user’s perception of reality and enriches information 

content—content that can help project managers and engineers deliver their projects faster, safer, 

and with greater accuracy and efficiency. The ability to navigate through all phases of a 

construction project enables managers to catch errors before construction and potentially 

improve design and construction details. Managers may also be able to use AR technologies for 

training, inspection, and stakeholder outreach. Considering its benefits and success in the 

entertainment and video-game industries, leveraging AR will likely be a significant advancement

in construction management of highway infrastructure assets. 

RESEARCH OBJECTIVES AND APPROACH

A key objective of this study was to identify the availability, accessibility, and reliability of using 

AR technologies for construction inspection and review, QA, training, and improved project 

management using real-time information in a real-world environment. AR has the potential to 

reduce construction cost, improve delivery time, and assist with overall management of 

construction projects. For this study, researchers documented potential advantages, limitations, 

shortcomings, and costs of using AR, as well as potential future challenges. 

Additional study objectives included documenting methodologies for managing the data flows

required to support AR and its integration into highway agency design and planning 

workflows—particularly how AR workflows are compatible with building information modeling 

(BIM) workflows already in use or being developed by transportation agencies. 
2 

A final objective was to identify current AR case studies that highlight and promote AR in 

construction management of highway infrastructure. The study identified and documented the 

implementation of emerging AR technologies currently used in construction and present best 

practices and lessons learned from successful AR use in other industries. 

While AR use in the architecture, engineering, and construction (AEC) industry is just 

beginning, formal documentation to catalog its potential advantages, limitations, and applications 

will help lay a foundation and identify promising future paths. 

REPORT ORGANIZATION

The first chapter of this report provides an overview and definition of AR, including the various 

terminologies currently in use, similarities and differences between AR and the more familiar 

virtual reality (VR), and the emerging overall descriptor, mixed reality (MR).

Chapter 2 includes the results of a market review of AR technologies. The chapter features 

descriptions of AR technologies, products, workflows, and software applications for highway 

applications—several of which are available today. The chapter also contains a discussion of 

implementing AR in 3D design and construction workflows. 

Chapter 3 outlines the characteristics of available AR systems, including displays, user interface 

(UI), sensing, and portability, and it describes the reliability, applicability, strengths, weaknesses, 

and challenges of available AR systems in construction applications. This chapter also outlines 

how AR technologies can be implemented in a BIM workflow supporting construction. 

Chapter 4 summarizes research on state-of-the-art AR applications in highway construction and 

related design and construction activities, describes highway construction scenarios and 

applications of AR tools, and outlines relevant technology and data workflow strategies. This 

chapter also reviews AR applications and workflows in construction-related fields, such as 

design, planning, and inspection. 

Chapter 5 reports the outcomes of the two workshops hosted by FHWA. The workshops 

provided a forum for discussion of AR tools and potential applications in highway construction. 

The goals of the workshops were to inform the research priorities and anticipate future directions 

and opportunities for AR in highway construction. The workshops involved technology and 

application developers, State DOT representatives, contractors, consultants, and other 

practitioners who explored how AR can be used to support construction-management functions 

during highway construction, ranging from construction inspection and review to QA, training, 

and improved project management through real-time information in a real-world environment. 

Five hypothetical use cases for AR applications that would positively impact highway 

construction were developed and ranked by potential impact and by feasibility of implementation 

as an outcome of the workshops. 

Chapter 6 summarizes the five case studies of potential highway construction AR applications 

developed from the outcomes of the two workshops and refined using the results of the study 

research. Each use case is outlined in a narrative format with a description of the AR technology 

and workflow used. The chapter also outlines an AR framework that delineates the specific data, 
3 

workflows, and technologies required to implement AR for these types of highway construction 

AR activities.

Chapter 7 summarizes the report with key observations derived from the study and includes 

some conclusions resulting from the research and associated workshops. 

OVERVIEW OF AR

AR Versus VR or MR

AR and VR are often part of the same conversation, and the difference between the two may be 

confusing to those who have not used the technologies. Azuma (1997) proposed a widely

accepted definition of AR that states AR must combine real and virtual elements, be interactive 

in real time, and be 3D registered. This definition does not require a specific output device nor 

does it limit AR to visual media (Azuma 1997). 

AR provides an intuitive and direct connection between the physical world and digital 

information. AR applications range from smartphone apps that annotate the starry sky to 

complex augmentation gear that delivers simulated military training scenarios to soldiers in the 

field. Implementing AR presents complex technical challenges, such as real-world 3D 

registration, real–virtual object occlusion, and devices that are wearable (Ren et al. 2016). 

AR is a general term applied to a variety of display technologies capable of overlaying 

(or combining) alphanumeric, symbolic, and graphical information with a user’s real-world view

to provide aligned, correlated, stabilized, contextual, and intelligent information that augments a 

user’s understanding of their real-world view. VR, by comparison, provides a complete 

replacement to the visual world experienced by the user (Aukstakalnis 2017).

MR is becoming more common in product literature and can be more confusing than AR and 

VR. Milgram and Kishino (1994) defined the relationship between AR, VR, and MR, and 

developed a diagram comparing them along an axis they call the reality–virtuality continuum, 

which is illustrated in figure 1. 

© 2019 WSP. 

Figure 1. Illustration. The reality–virtuality continuum. 

Full reality is at one end of this continuum, and complete VR is at the other end. The entire 

region, from full reality to VR, is considered MR. AR is a subset of MR and lies near the reality

end of the continuum. A significant portion of the MR experience is the real world, which is then 
4 

augmented by computer-generated contextual data. Milgram and Kishino (1994) created the term 

“augmented virtuality” to identify systems near the other end of the MR spectrum where the 

main experience is mostly computer generated. Variations in augmented virtuality include 

real-world imagery, such as adding texture-mapping video to virtual objects (Milgrim 1994). For 

construction applications, real-world perception provided by AR and its association with virtual 

data and imagery can be a powerful combination. 

Virtual Imagery and Scale in AR 

The virtual information displayed over the real-world image can take many forms, ranging from 

graphical annotation and conceptual information to accurate 3D representations of a proposed 

design or construction scenario. The latter most often comes to mind when considering 

applications for AR in construction; however, for inspection and training applications, more 

informational text or graphics will undoubtedly be part of the presentation. Displaying associated 

metadata or 3D-model attribute information beside relevant 3D elements or real-world objects 

will enhance the user’s understanding of the displayed imagery and environment. 

Most currently available AR applications and device platforms allow for displaying virtual 

3D-model imagery at different sizes, or they display varying model scales relative to the real 

world. For example, a 3D-building design can be displayed on a desktop at a scale like the view 

of a scale physical model, or the model can be displayed at true scale adjacent to or around the 

viewer and explored as if the user is in the proposed environment. This display at true scale can 

provide the user with a powerful sense of immersion and presence in the 3D model. The sense of 

reality and immersion is enhanced by the quality of the display technology (especially if the 

display is stereoscopic) and by the fidelity and quality of the 3D virtual imagery. 

Specific Opportunities for AR BIM in Construction

The use of 3D models and associated metadata for design and construction planning is 

commonly referred to as a subset of the BIM process. Georeferencing and visualizations of 3D or 

other BIM data overlaid onto a view of a construction site have three main use categories: 

• Visualizing what is not yet constructed (proposed). 

• Viewing buried elements or elements obstructed from view. 

• Viewing abstract informational data, such as alignment information, easements, site 

boundaries, or environmental boundaries (i.e., flood levels or sea level–rise data). 

There are many opportunities for visualizing information onsite because of the data-rich nature 

of BIM. BIM data can be geolocated directly on the construction site while using AR to 

communicate project information during construction. Overlaying BIM data onto existing site 

elements could benefit site inspectors or contractors checking construction progress. Overlaying 

3D models in AR can help confirm the best installation locations for construction components 

and locate materials, equipment, safety zones, and construction and project components to better 

prepare site workers. Hazardous work areas and critical emergency information highlighted in 

AR views can enhance onsite safety (Abboud 2014). 
5 

Collaboration in AR 

Collaboration involves two or more individuals working together to solve a problem or repair or 

assemble something. AR can support collaboration between team members involved in 

construction and engineering. Traditionally, AR allows field personnel to see engineering 

designs displayed over real-world images, which helps the construction team understand design 

goals and the engineering team understand constructability challenges. In the field, AR devices 

with built-in cameras can transmit an onsite team member’s view to remote team members with

images and annotations to help describe it in an AR-registered view. 

Several examples of this approach to collaboration were found in manufacturing and assembly 

operations where remote users could assist an onsite user by sending remote users virtual images 

or 3D models to match to real-world objects, sending them data about the objects, and annotating 

or sketching over real-world objects in their view (Welch 2016). There is significant potential for 

AR collaboration in construction training and inspection applications.

7 

CHAPTER 2. AR TECHNOLOGY MARKET REVIEW

This chapter provides detailed information on AR technologies and software applications, 

including technologies and applications that support AR implementation at a construction site. 

The review focuses on technologies and applications that support design, planning, and

construction of highways. 

CLASSIFICATION OF AR TECHNOLOGIES

This section provides an overview of technologies currently in use or likely to be used in the 

future to support AR applications for construction. The section includes descriptions of both 

hardware and software technologies that support AR. First is an overview of display types and 

characteristics, followed by an explanation of display hardware technologies. Next is a 

description of technologies for tracking the position and orientation of a device or user in the 

real-world environment and the user within the virtual environment relative to the real world. 

Also included is a description of currently available AR-enabling software and applications. 

AR Hardware

Display Types

A key component that enables AR systems is the display configuration, which allows the 

blending, or augmenting, of real-world imagery with virtual representation. This blending is 

typically achieved by using a see-through display. There are two categories of displays that 

achieve this blending: video see-through (VST) displays and optical see-through (OST) displays. 

Another category of augmented display described by Schmalstieg (2016) is spatial projection 

where virtual imagery is projected onto real-world objects, but the limitations and applications of 

this technology do not seem particularly suited for construction applications.

VST displays use digital video and a graphics processor to combine real-world imagery and 

computer-generated imagery, which is then presented to the user on the display. A VST 

head-mounted display (HMD) has three components that must be calibrated and tracked: the 

user’s eye(s), the display, and the camera (Schmalstieg 2016; Aukstakalnis 2017). When used 

with AR applications, tablets and smartphones can be considered VST devices because they 

capture real-world imagery through the camera on the device and combine that with virtual 

imagery, which is then displayed on the device’s screen. VST displays need not be present in the 

real-world environment being captured. The video imagery and virtual imagery can be combined 

and displayed in a remote location, similar to what occurs with video conferencing systems 

today. Remote AR is being used in the industrial sector for remote maintenance and 

collaboration. 

OST displays commonly use an optical element that is partially transmissive and partially 

reflective, such as a semitransparent mirror, to overlay a virtual image onto a real-world image. 

Users view the real world looking directly through monocular or binocular optical elements, 

which simultaneously display virtual imagery (Schmalstieg 2016; Aukstakalnis 2017). One 

challenge with OST devices is controlling the brightness of the virtual elements compared to 

real-world objects. Some devices on the market use advanced optics, such as a special optical 
8 

prism, to reflect and refract light outputs. Some devices use a separate image sensor to monitor 

environmental illumination so that the virtual display can be set relative to that brightness for 

better visibility. OST HMDs have the display set at a fixed distance from the eye. Each time an 

HMD is used, it must be calibrated to the user’s eye. Calibration can be performed quickly and 

automatically with devices that use eye trackers mounted inside the HMD. 

Table 1 lists advantages and disadvantages of some AR display types. 

Table 1. AR display types and their advantages and disadvantages. 

Visual Displays Advantages Disadvantages 

VST • Implementation is inexpensive 

and easy. 

• Reality is digitized. 

• Brightness and contrast of virtual 

objects are adjustable.

• Head movement is better 

tracked.

• Reality displays in low 

resolution. 

• Field of view is limited. 

• Distance adjustments can 

result in user disorientation. 

• Eye strain and fatigue are 

common occurrences. 

OST • Safely enables user to see with 

no power. 

• High-resolution display is 

included. 

• Simplicity allows for no frame 

delay. 

• Wide FOV benefits the user. 

• Reduction of brightness and 

contrast of virtual objects is 

poor. 

Display Characteristics

Various characteristics of displays can affect the quality of the AR imagery and the user’s 

experience, comfort, and perhaps most importantly, sense of immersion in both the real and 

virtual environments. 

Ocularity and Stereoscopy 

AR displays can be classified as either monoscopic or stereoscopic. Stereoscopic displays often 

produce more realistic and immersive experiences. Monocular devices present images only to 

one eye. A monocular HMD can be used for AR, but these types of devices are not as popular 

because they are not as immersive as biocular devices. Google GlassTM is an example of a 

monocular display. Biocular devices present the same image to both eyes concurrently, which 

maintains a monoscopic view. VR presentations of panoramic images on mobile devices, such as 

Google CardboardTM and Google DaydreamTM, are typically biocular views of the same

panorama presented to each eye. Binocular displays present separate images to each eye, 

resulting in a stereoscopic effect and a stronger sense of immersion. The tradeoff in using 

binocular displays is that images must be captured or rendered separately for each eye, thus 

increasing system requirements (Schmalstieg 2016). 
9 

Stereoscopy and Perception of Scale

Human vision is binocular and relies on disparities between each eye to perceive depth and the 

true scale of objects. VR and AR systems present images to each eye with the same geometry a 

user’s eyes would see in the real world; therefore, users experience an immersive and realistic 

view of the scene that represents depth and scale close to how a user would perceive objects in 

the real world (Howard 1995). 

Resolution and Refresh Rates

The resolution of the AR display has a direct correlation to the fidelity of the display image and 

is restricted by the type of display and optical system. VST display resolution is restricted by the 

type of video camera on the device used to capture real-world imagery. OST displays present a 

real-world view and will thus have higher perceived image quality because the real-world 

imagery is independent of display resolution. 

Conversely, the disparity in image quality can be detrimental. The refresh rate of the display, 

how rapidly individual video frames are displayed, influences how well the device renders 

scenes or objects in motion. For rendering fast motion without image lag (latency) or ghosting, 

which is when multiple versions of the same object are perceived, higher frame rates (120 Hz or 

higher) can be advantageous. For most AR devices, frame rates above 60 Hz are desired 

(Schmalstieg 2016). 

Field of View

A key consideration of any AR HMD is the field of view (FOV). FOV refers to the total angular 

size of the virtual image visible to both eyes, expressed in degrees (Aukstakalnis 2017). FOV, 

image resolution, and refresh rate are interrelated, as more pixels are needed for a wider FOV. 

More pixels, or higher resolution, to cover a wider FOV requires more system performance to 

maintain the same refresh rate. A limitation on many of the currently available OST devices is 

the FOV of the overlaid computer image due to technological constraints. Many current devices 

are limited to a viewable field of 60 degrees or less. 

The typical human binocular FOV is 120 degrees, which is combined from the left and right eye 

views, as shown in figure 2. 
10 

© 2019 WSP. 

Figure 2. Illustration. Typical human binocular FOV (120 degrees).

The typical FOV in a current AR display device is approximately 60 degrees, which is half of the 

binocular field, as shown in figure 3. This reduction in FOV limits what can be displayed by the 

AR device and can result in the viewer having to move his or her head back and forth to view the 

entire virtual scene, which can lead to fatigue and discomfort. 
11 

© 2019 WSP. 

Figure 3. Illustration. Current AR display FOV (typically 60 degrees). 

The FOV on VST devices is based on the camera rather than the display, and the camera’s FOV 

is often wider than that of the display. This difference can result in the displayed image 

appearing compressed with a slight fish-eye effect. Handheld AR devices, such as tablets or 

cellphones, typically have cameras with a larger FOV on the back of the phone than the angle of 

view presented on the display, which is noticeable when the device is held at arm’s length. 

HMDs benefit from a wide FOV, which can only be accomplished by either increasing the size 

of the HMD or by placing the display device or optics closer to the eye (Schmalstieg 2016). 

Occlusion

Occlusion refers to objects that should be hidden from view once the real-world image is fused 

with the virtual rendering (i.e., when a virtual object is placed behind or in front of a real object). 

Occlusion is an important visual cue to convey realistic scene structure. AR applications use a 

variety of rendering techniques to deal with occlusion; the success of these applications can 

depend on many factors and influence system performance. 

User Discomfort with Display Technologies 

One challenge that has faced the adoption of VR and AR display technologies is an effect 

alternately referred to as “VR motion sickness,” “cyber-sickness,” or “simulator sickness.” The 

effects vary between users with some people being more sensitive to the effects than others. One 

factor that contributes to this sensation is a mismatch between visual sensory input and vestibular 

input (i.e., there is a mismatch between the perceived movement and the physical body 

movement or motion sensory input from the inner ears). This effect is most pronounced in 
12 

HMDs, and the mismatch is due to visual computing systems not being able to provide updated 

imagery at a high enough frame rate to match the movement of the head and body. The delay in 

delivering updated frames is measured as latency and generally occurs with faster head 

movements. Research shows that at least 90 frames per second (fps) should be delivered to keep 

that discrepancy within tolerable levels (Buker 2015). Stereoscopic displays have two frames 

that would need to be delivered in at least 90 fps. Many currently available VR HMDs are 

designed to achieve 90 fps, but the computing system delivering the visual content to the device 

ultimately controls the frame rate (Hecht 2016). 

A second factor that contributes to VR motion sickness is the virtual movement of the user’s 

viewpoint in the virtual environment—an effect called vection. The problem with vection comes 

from the disparity between what the eyes and the vestibular system tell the user. The eyes show 

forward movement, but the vestibular system feels no such motion. This problem occurs more 

with VR systems because there is usually a need to move the user’s viewpoint as the user 

explores the virtual environment. One strategy sometimes used to reconcile the disparity

involves placing the user in a virtual element that moves with the user, such as a driving 

simulator, which typically includes interior elements of the simulated vehicle. Vection will most 

likely not be a critical issue with AR display systems because the user’s motion is tied to the 

real-world environment. 

A third factor that affects user comfort, primarily in stereoscopic displays, is vergence—the 

positioning of the eyes in opposite directions to fuse two images into a single binocular image. 

The eyes sense this fusing movement separately from their sense of accommodation (i.e., how 

much the eye adjusts its focal distance to bring an object into focus). The two cues can conflict if 

the actual distance of the object differs from the apparent distance—and the mismatch is larger 

when the actual focal point is on a screen close to the eye in a VR or AR HMD. Vergence can 

affect the user’s ability to place a virtual object into a real-world environment comfortably and at 

the correct distance (Hecht 2016). 

Display Hardware

AR displays can be roughly categorized by the distance of the viewing mechanism from the 

user’s eye. The three categories are head space, represented by HMDs; body space, which 

includes handheld displays, such as tablet or mobile devices; and world or environment space, 

including projected displays and immersive rooms, such as cave automatic virtual environments 

(CAVEs). Key considerations of HMDs include FOV, resolution, and refresh rates. The ideal 

HMD must have a wide FOV and be comfortable, unobtrusive, and lightweight.

HMDs

Visual 3D-display technologies are categorized into four classes: stereoscopic approaches, 

holographic displays, light-field displays, and volumetric displays. Stereoscopic approaches rely 

on sending separate images to each eye. This is the most commonly used approach in HMDs 

because binocular near-eye devices, or those with displays close to the viewer’s eyes will have to 

send separate images to each eye. Stereoscopic displays using glasses (polarized or active shutter 

glasses), such as projected 3D films, fall into this category as well. Holographic and light-field 

displays use light waves to generate and then recreate images in 3D space. Holographic devices 
13 

use coherent (laser) illumination, and light-field displays use incoherent light. Volumetric 

displays use multiple light sources projected at points in space to create a 3D image. Visual AR 

displays use one or more of these technologies to create or project virtual images in space that 

are perceived by the user from multiple viewing angles (Schmalstieg 2016). 

Handheld Display Technology 

Handheld devices, such as tablet computers and mobile phones, are currently the driving force 

behind AR in the gaming, entertainment, and AEC industries. Handheld displays are VST 

devices that use the back-facing cameras on the device to capture video of the real-world 

environment and display that image on the front screen. AR applications then align and render 

virtual data over the real-world imagery on the screen. 

A user typically needs to hold the handheld device close to eye-level at shoulder height and at 

arm’s length to capture the widest FOV. This position can be difficult for a user to maintain over 

long periods and could prove challenging in a construction environment. 

Stationary Displays 

Stationary displays that can be used in AEC environments include desktop configurations and 

possibly even immersive CAVEs with projectors. Desktop displays are composed of a desktop 

computer or laptop with a built-in webcam whose video image can be used to create a VST 

environment. Desktop AR systems will have uses in collaborative AR applications in addition to 

video conferencing. Users in office locations could potentially collaborate with users on 

construction sites through their desktop computers or by using CAVE facilities that allow 

viewing the real-world environment through the onsite user’s device, and communication can 

occur via the remote user’s VST device. It is possible that a large-scale immersive CAVE room 

could be used for collaborative AR applications, which could allow several people to 

communicate directly with remote users on construction sites through their AR device. 

Device Tracking—Sensing Movement Within the Real-World Environment 

In real time and with a high level of accuracy, AR devices need to follow a user’s movement in 

relation to position and orientation in the real-world environment. This process is referred to as 

tracking and is a significant challenge in the design of AR devices. The position of devices is 

typically measured in x, y, and z coordinates relative to real-world position in geodetic or 

project-based coordinates. Orientation, or the view direction of the real and virtual cameras, is 

measured in angles around the positional coordinates as angles in x, y, and z (i.e., Euler angles) 

or in camera-relative angles referred to as pitch, yaw, and roll. Changes in position and 

orientation represent movement in three axes of measurement or a total of six degrees of 

freedom.

The combined measurement of position and orientation is sometimes referred to as the device or 

camera pose. AR systems must follow the user’s position and orientation in real time and update 

the pose of the virtual camera to match that of the real-world camera in real time. Registration is 

the term used to describe the alignment of the virtual model that coordinates to the real-world 

position and orientation (Schmalstieg 2016). The positioning and orientation of the user in a 
14 

virtual world relative to the real world is referred to as localization, particularly in the robotics 

navigation field. 

Optical Tracking Configurations 

Optical tracking for AR devices, such as HMDs, can be divided into two categories based on the 

location of cameras and sensors. Outside–in tracking refers to systems in which cameras are 

placed in stationary locations and the user is tracked within a fixed environment. The external 

cameras track the user’s HMD, which has passive or active markers attached. This tracking 

method produces highly accurate position and orientation results. One limitation of outside–in 

tracking is the camera or sensor locations—the object cannot be tracked when it is out of the line

of sight. These types of systems are limited to the range of the cameras and the working space 

defined by their locations. Outside–in systems will most likely prove to be impractical for most 

construction sites due to this limitation to a configured environment and the need for installed 

devices in the environment. AR/VR devices, such as Oculus RiftTM, HTC ViveTM, and Sony 

PlayStation® VR use outside–in tracking. 

Inside–out tracking uses cameras or sensors placed on or inside the AR device. This approach 

allows for a greater sense of freedom because the user is not limited to a specific tracking space 

defined by fixed camera locations. The camera or sensors view the surrounding environment to 

determine how their position changes in relation to the real world. These systems are limited 

only by the size and performance characteristics of the sensing technologies they use. AR/VR 

devices, such as Microsoft HoloLensTM and DAQRI and Microsoft MR devices, such as the 

Samsung HMD Odyssey use inside–out tracking (Ishii 2010). 

Tracking Sensor Technologies

Sensors in AR systems help track the location and orientation of the user and the location of real 

objects and markers in the environment using real-world coordinates. Many technologies can be 

used to track position and orientation; each has its own performance characteristics, costs, ease 

of use, and mobility for the user. Tracking technologies include sensors that monitor satellite 

network position, such as global navigation satellite system (GNSS), or wireless networks, such 

as those used in mobile devices. Tracking technologies also include inertial sensors, optical 

sensors that include video and infrared (IR) sensors, simultaneous localization and mapping 

(SLAM) devices, and sensors that can perform 3D scanning of environments concurrently with 

the user’s change of viewing direction. 

Each of these tracking technologies have their advantages, but for outdoor environments, hybrid 

systems combining inertial sensors and optical tracking technology provide the most robust and 

accurate positioning for AR applications (Holloway 1997). At construction sites, highly accurate 

tracking is necessary, as well as long ranges, high bandwidth, and the ability to process large 

amounts of data. The challenges of tracking in an outdoor environment include sensitivity to 

static and dynamic errors and potentially less control over the environment. Construction sites 

are always changing, which may limit or even eliminate the ability to place static sensors or 

markers onsite.

Table 2 outlines some advantages and disadvantages of tracking methods found in AR devices. 
15 

Table 2. Tracking methods and their advantages and disadvantages. 

Tracking 

Method Advantage Disadvantage 

Satellite Position 

Tracking (GNSS) 

• Stable when enough satellites 

are available. 

• Available in a broad area. 

• Accuracy is improved with 

DGPS and/or CORS. 

• Absolute position is obtainable. 

• Unavailable inside buildings. 

• Only position is obtainable, 

orientation is not. 

• Vertical accuracy does not 

match horizontal accuracy. 

Wireless LAN • Available inside buildings. 

• Stable when enough stations are 

available. 

• Unnecessary installation when 

enough WLAN stations are 

already installed. 

• Absolute position is obtainable. 

• Low level of accuracy. 

• Only position is obtainable, 

orientation is not. 

Inertial sensors • Unnecessary to install in the 

environment. 

• Computational load is low. 

• Accuracy decreases over time 

because of accumulation of drift 

error. 

• Other methods need to be used 

concurrently to obtain absolute 

position or orientation. 

Vision sensors 

(marker-based)

• Inexpensive compared to most 

other tracking methods. 

• Scalable tracking system.

• Accurate and stable when 

enough markers are visible. 

• Necessary to install markers; 

positions and orientation must 

be measured in advance. 

• Available only when markers 

are visible. 

• Necessary to install a large 

number of markers when a broad 

area needs to be covered. 

Vision sensors 

(markerless or 

natural feature 

based)

• Installation not necessary while 

in the environment. 

• Less expensive for executing 

tracking. 

• Scalable tracking system.

• Accurate and stable when 

enough features are visible. 

• Trackable area can be extended 

online by using SLAM devices. 

• Require significant processing 

overhead. 

• Accuracy and stability depend 

on environment. 

• Unstable in a dynamic 

environment in which features 

are not static.

CORS = Continuously Operating Reference Stations; DGPS = differential global positioning system; WLAN = wireless local 

area network.
16 

Satellite Position Tracking

GNSS uses communication signals between satellites to provide precise location positioning 

anywhere on Earth. The United States’ satellite system is called the Global Positioning System 

(GPS). Higher accuracy can be obtained using networks of servers and base stations. The servers 

and base stations send correction signals through Wi-FiTM or radio waves to the GPS receiver on 

the device. Real-time kinematic or differential GPS satellite navigation and the National Oceanic 

and Atmospheric Administration network of Continuously Operating Reference Stations are 

technologies used to enhance the precision of position data derived from satellite-based 

positioning systems. Two drawbacks of GPS positioning include sensors that do not work well 

indoors and vertical precision (z) that can be less accurate than ground positioning (x and y). The 

accuracy and consistency of these systems are not currently ideal for the high-accuracy tracking 

required for AR devices in situations in which virtual imagery need to match real-world imagery

exactly. Virtual imagery matched using GNSS alone may move relative to real-world imagery 

displayed in real-time through the device (Pesyna et al. 2014). Mobile device–based AR 

applications in which exact registration of virtual and real-world imagery may not be required 

will most likely be based on GNSS tracking and positioning. 

Wireless Networks

Wireless networks include Wi-Fi, Bluetooth®, and mobile phone networks. These networks can 

be used to determine a user’s or device’s position. Wireless networks differ in bandwidths, 

latency, fluctuations, and availability. To support an AR device, a wireless network must have 

low latency, good mobility, and high bandwidth (Sharma 2014). Vertical precision can be an 

issue with these systems. The accuracy of systems, like GNSS or GPS, is often not ideal for the 

tracking requirements of AR devices. Combinations of wireless networks can be used to improve 

coverage, speed, and accuracy of user positioning. Combined wireless sensor networks and 

jobsite Wi-Fi networks are valuable to the construction industry. These combined networks will 

prove to be a critical and useful component on the jobsite by providing tracking and localization 

for AR devices as well as access to 3D models and other data associated with AR applications 

(Domdouzis 2004; DeWalt 2017). 

Inertial Sensors

Inertial sensors measure the motion of a device in space and can include gyroscopes and 

accelerometers. Gyroscopes measure rotational inertia while accelerometers measure linear 

inertial displacement. A magnetometer, or electronic compass, measures magnetic fields. The 

output of a magnetometer is used to determine absolute north, east, south, or west, like a 

compass. Gyroscopes and accelerometers are used in conjunction with magnetometers to 

determine both position and orientation (Collin 2013). Combining the three sensors allows for 

rapid and accurate position and orientation determination with minimal latency. Modern inertial 

measurement unit sensors are based on microelectromechanical systems (MEMS) technology. 

MEMS are being used more frequently in the development of AR hardware because of their 

small size, low cost, and low latency. 
17 

Vision Sensors and Visual Tracking 

Using vision or optical methods for tracking requires small digital cameras, which function as 

sensors. The camera captures specific features in the environment and uses those features to 

determine relative orientation and position. Many vision-based systems use IR scanning sensors 

and/or multiple cameras that allow capturing 3D real-world data. Tracking methods for vision 

sensors can be defined as either marker based or markerless based depending on whether 

markers, or graphical targets, are placed into the scene. These markers are placed in the scene 

and are typically objects designed to be easily detected, such as circular or square shapes. The 

position of the markers must be measured beforehand in the real-world position, and those 

locations are used by the system to register the virtual model as the locations are detected. Many 

off-the-shelf AR applications and toolkits use graphical markers for tracking. Marker-based 

tracking systems are not ideal for large-scale outdoor environments because the size of the 

markers varies too much between closeup and distant views. One study used markers in large 

indoor nuclear facilities, and the markers proved to be relatively successful for navigation and 

inspection tasks on the site (Ishii 2010). 

Markerless or natural-feature tracking uses real-world objects to compute the camera position 

and orientation and to align virtual objects in the scene. If a reference model is provided to the 

system before tracking, an approach referred to as model-based tracking, then the imagery from 

the camera is compared to the model and aligned to the real-world position. A few of the AR 

development kits on the market, such as VuforiaTM and VisionLibTM, support tracking using 

reference models (Schmalstieg 2015; Vuforia 2017; VisionLib 2020). Markerless tracking is 

more flexible and effective because there is no need for a prepared environment with markers. 

Model-free, markerless tracking uses a relative method of position calculation and can only place 

virtual objects spontaneously and not registered to real-world objects. AR systems can combine 

this camera-based, model-free visual tracking with inertial sensor data to more accurately and 

efficiently calculate movement and position, a process referred to as visual-inertial odometry 

(VIO). VIO algorithms are being built into many of the AR software platforms currently under 

development. A similar software process to VIO is SLAM, a process whereby the system 

concurrently maps the real-world environment in 3D and calculates the user’s position and 

orientation within that environment (Leutenegger 2014). 

Current VIO systems can maintain relative position with high accuracy and for considerable 

distances with access to real-world objects that can be tracked effectively. When a real-world 

position is provided to the system for accurate registration of the virtual model in the real-world 

environment, these systems maintain their position relative to the real-world, and hence the 

virtual world, very accurately. Many AR application developers are exploring hybrid solutions 

for AR devices that establish real-world coordinates with a marker first, then continue to track 

the user’s position and orientation with VIO tracking. A user first places a marker at a known 

location in the environment. An AR session begins when the user scans the marker with the AR 

device. The AR device registers the real-world view to the virtual model environment, then the 

system maintains position and orientation with the tracking process. Some case study examples 

of this type of implementation using the DAQRI and Microsoft HoloLens devices were 

presented at a recent industry-sponsored event by Chen (2017) and Jahangiri (2017). 
18 

User-Controlled Positioning and Navigation 

AR devices and applications use different methods for user navigation within the virtual 

environment. Tablet devices and phones typically support swiping and touch controls on the 

device face. Many AR device manufacturers have developed gestural controls that allow the user 

to navigate and control virtual elements with hand and finger motions. Intel® RealSenseTM and 

Leap MotionTM devices natively support these gestural navigation controls. The Microsoft

HoloLens supports audio-based commands and controls as well as gestures. Most devices 

support interaction with menus and virtual objects with a technique called gaze and dwell. A 

reticle in the user’s view shows where the user’s view is focused. When that view is fixed for a

certain amount of time on an object or menu (the “dwell” in the gaze and dwell technique), the 

object is selected, or the menu command is processed. 

The types of controls available on construction-focused AR applications will be a critical aspect 

of implementation and adoption due to safety and convenience concerns by users in an active 

and complex construction environment. 

AR-ENABLING SOFTWARE AND APPLICATIONS

AR software systems need proper framework, networks, data storage, and data access. Two 

important requirements for AR software are platform and user-interface abstraction, both of 

which allow reusability and extensibility of the interface. Platform abstraction is necessary for 

AR software to run on various systems and displays. Platforms contain an operating system (OS)

that is deployed on a device and built off software development kits (SDKs). SDKs are 

collections of multipurpose libraries, such as ARToolkitTM, Apple ARKitTM, Google ARCoreTM, 

and Microsoft Windows Mixed RealityTM, that are used to build software on independent 

platforms. Cross-platform compatibility allows developers to reuse source code for target 

systems. Reusability of software components is important because new AR applications and 

devices are constantly being developed. Platform independence avoids proprietary vendor 

software and promotes the adoption of new, more innovative hardware. AR UIs consist of real 

and virtual objects and need to be able to react in real time to changes in the user’s view and the 

real-world environment. There is currently no standard UI paradigm for AR interaction 

(Schmalstieg 2016). 

Computer-Aided Design and Drafting/BIM AR Applications

AR applications for construction will be developed around either the display of annotation and 

graphical information that enhances the understanding of real-world objects, or they will be 

developed around 3D design and construction models that will be overlaid in position in the view 

of the real-world environment. This is the direction many of the current AR applications have 

taken, and most are based on workflows and tools that are already in place in support of BIM 

applications for design and construction. These computer-aided design and drafting 

(CADD)/BIM tools will be an obvious starting point for the development of AR tools, and many 

AR applications will eventually become extensions of current BIM and 3D model workflows 

rather than discreet processes. 
19 

Nearly all major vendors for 3D design applications have implemented tools and workflows that 

support the deployment of 3D models to mobile devices for viewing in the field, and most 

support the built-in georeferencing capabilities of the devices. The platforms support the review 

and collection of data in the field through mobile devices, which can then be synchronized with 

project models and data in the design office. Many vendors have enabled deployment of these 

3D models to one or more of the AR HMD models through these mobile platforms. While most 

of the tools offer means for optimizing models and model display on mobile devices with more 

limited processing power and storage, this optimization will be even more important for AR 

devices and applications that typically require sufficient graphics performance to support 

real-time stereoscopic rendering of the 3D models. Current limitations of most AR HMDs 

include processing power, memory, data storage, and connectivity. 

Autodesk 360 Tools as an Example of Cloud-Based Project Information Sharing

Autodesk® developed a cloud-based sharing and collaboration framework for its design and 

construction planning tools that allows online access to project design information. The 

Autodesk 360 family of tools includes integration platforms and mobile-device-based viewing 

and data collection tools that can leverage data from all its applications. The Forge platform is 

Autodesk’s open-source development platform for third-party developers that supports access to 

all their CADD tools and platforms, as well as cloud-based collaboration and delivery of several 

data types via web-based applications. The BIM360 Docs tool is built on Forge and provides 

web and mobile-device access to most data types, including 3D model/BIM data.

At Autodesk University (AU) 2017 in Las Vegas, DAQRITM (developer of AR HMD devices) 

presented a tool under development in partnership with Autodesk on the Forge platform using 

BIM360 Docs to prepare and deliver a 3D model to the DAQRI device via a wireless connection. 

The preparation process included the ability to optimize the model by editing the extent of the 

model and hiding unnecessary objects to reduce the overall size so that it would run effectively 

on the DAQRI device. With the model loaded and the user onsite, a printed target image was 

used to register the virtual model to real-world coordinates. The user was then able to navigate 

around the site while the model remained registered to the real world. The user could access and 

display data embedded in the virtual model that was imported from the original design model 

(Chen 2017). 

Currently, only InfraWorks 360 for mobile devices supports geolocation within a project model 

through GNSS localization on enabled devices. Recent discussion on the Autodesk Knowledge 

Network has focused on geolocation being provided for mobile-based 360 tools. These 

mobile-based tools remain a powerful way of bringing virtual two-dimensional (2D)- and 

3D-model data to a construction site where that information can be compared to existing site 

conditions as evidenced by their adoption in 3D model-based workflows. This platform will most 

likely be the framework in which AR applications within the Autodesk family of products would 

be developed. At AU 2017, Autodesk was demonstrating a marker-based VIO-tracking AR 

model viewer running on a tablet device that was connected to an online version of the model 

through the Forge platform. Several presentations were made at AU that showed examples of 

users adopting some of these tools for project sites. 
20 

In one example, the Autodesk InfraWorks 360 on a mobile device was used to display a 3D 

design model for a highway widening and parking area project in Oregon, and the view of the 

model on the device roughly lined up with real-world views of the site (Iliyin 2017). 

Bentley Systems

Bentley Systems is advancing VR and AR through their CONNECT 3D modeling programs and 

through a recent partnership with Analytical Graphics, Inc. The CONNECT programs include 

LumenRT, OpenRoads, AECOSim Building Designer, OpenPlant, and Context Capture 

(Bentley Systems 2017). 

Bentley LumenRT is Bentley’s visualization and reality modeling software tool that allows the 

creation of immersive 3D virtual environments. Immersive LiveCubes can be created to share the 

VR models on desktop computers or export them to VR displays, such as Oculus, HTC Vive, 

and Samsung devices. OpenRoads allows users to develop 3D models of existing and proposed 

infrastructure, including roads, bridges, subsurface utilities, and buildings. Context Capture can 

generate millimeter-accurate 3D reality models from digital photographs. Bentley’s suite of tools 

allows designers to develop 3D virtual models and environments that can be used with AR 

devices. 

Trimble Systems/SketchUp

Trimble has developed survey- and reality-capture technologies, and Trimble has also developed 

several products intended to support mobile devices in the field and the deployment of 3D-model 

data to those devices. Trimble ConnectTM is a cloud-based data portal platform that supports 

model integration using all of Trimble’s 3D model formats (including SketchUpTM) as well as 

several of the more commonly used 3D design platforms from other vendors. Trimble Connect 

supports viewing and georeferencing of those models on several mobile-device platforms.

Trimble CatalystTM is a software subscription-based tool that supports enhanced GNSS 

positioning on mobile devices. Trimble states that, with a plug-and-play antenna device and the 

Catalyst software, submeter accuracy for positioning is possible in many locations. 

Synchro Software

Synchro ProTM is the viewing and collaboration tool that supports the deployment of Synchro 3D 

and four-dimensional (4D) models onto mobile devices for use in the field. Synchro Pro also 

supports a connection directly with AR devices, such as the Microsoft HoloLensTM, for AR 

viewing. At the time of this writing, Synchro has been acquired by Bentley Systems and will 

become part of the Bentley Connect Suite of tools. 

Deploying AR Applications in the Field

AR in construction will inevitably require bringing project design data into the field. Project 

design data will most likely be in the form of 3D digital models. These 3D digital models are 

typically large and complex. A workflow involving segmentation, optimization, streaming, or 

control over the level of detail (LOD) displayed or deployed to the field will be needed. This 

need for optimization will especially be true with AR applications where devices will have much 

higher performance requirements to maintain display frame rates and rendering quality.
21 

In addition to the use of a 3D cloud-based model distribution and sharing platform, such as those 

described earlier in this section, 3D model optimization schemes, whether through software 

tools, such as UmbraTM and Microsoft’s SimplygonTM, or manual methods by designers, will 

need to be used to simplify the 3D models for AR. 

This challenge of bringing large 3D models to the work site is similar to the challenge currently 

being faced by agencies and contractors transitioning to BIM and 3D model-based workflows 

(sometimes referred to in the highway design disciplines as Civil Integrated Management, or 

CIM) These model-based workflows are now being adopted by agencies and will be helpful in 

the adoption of AR tools for construction. AR could become an extension of this evolution of 3D 

tools in supporting construction. Many challenges faced by agencies will be the same as those 

encountered in the development of CIM as outlined in the National Cooperative Highway 

Research Program (NCHRP) Project 20-68A, Scan 13-02, Advances in Civil Integrated 

Management (CIM) (Adam 2015), and the associated publication NCHRP Report 831, Civil 

Integrated Management (CIM) for Departments of Transportation, Volume 1: Guidebook

(O’Brien 2016). 

The key CIM-related technologies identified in NCHRP reports that would support AR 

technologies include the following: 

• 3D engineered–models and e-construction processes. 

• GIS systems and geolocated data. 

• Mobile-device use in construction. 

• Electronic document management systems. 

The use of consistent geolocated coordinate systems for 3D geometry as part of a 3D-engineered 

model workflow is essential for AR. One case study in NCHRP 20-68A refers to the careful 

consideration of LOD in the 3D model, and that just enough detail is modeled to support the 

goals of the model in visualization and analysis. This approach will be critical in AR applications 

because device performance will be dependent on model size and complexity. Most devices 

currently available have limited onboard memory, storage, and processing power to support 

display of large and complex models. 

NCHRP Report 831 identified technology investments that would help agencies advance CIM, 

including required investment in information technology hardware and software. These same 

technology investments would benefit the advancement of construction-based AR. The CIM 

Guidebook authors examined the investment in software applications required to perform the 

necessary functions in the office and in the field. Investments can include database management 

systems, surveying and design software, and mobile applications for smartphones and tablets. 

There could also be additional costs for specialized equipment, depending on performance 

specifications. 

Mobile devices and wearable AR-viewing devices will require high-bandwidth Wi-Fi

connections for communication and model downloads. Construction site-based Wi-Fi networks 

for communication and localization will likely become more common on construction sites and 

an important supporting option for AR applications. 
22 

AR-READY PRODUCTS AND APPLICATIONS

There are relatively few AR products on the market with tools developed specifically for the 

construction market, but several commercial AR products and computer applications are 

available for the entertainment market. Some of the vendors in the AR space have started 

targeting construction applications and have developed marketing materials and prototypes 

directed toward the construction field. 

AR-Ready Viewing Devices––Handheld Tablet Devices

Trimble SiteVision GNSS

One AR device targeted directly for use on construction sites is the SiteVision GNSS tracking 

AR device, which will be supported through Trimble’s Connect platform (figure 4). The 

SiteVision GNSS tracking AR device is a VST handheld tablet, enabled with VIO tracking 

technology. The device can accurately overlay virtual 3D-model data based on position and 

orientation captured through the onboard GNSS tracking device in combination with Trimble’s 

Catalyst software. The Tango 3D capture technology tracks the position and view angle of the 

device relative to the real world so that the overlaid image does not appear to move over the 

video image. 

© 2019 Trimble. 

Figure 4. Photo. Handheld AR tablet device with GNSS tracking hardware.
23 

AR-Ready Viewing Devices––HMDs 

Two types of augmenting HMDs are available: binocular and monocular displays. Binocular 

displays are intended for general consumers but have branched out to a broader market for 

professionals (figure 5). Monocular displays are intended to aid technicians in completing 

specific tasks more efficiently.

© 2019 WSP. 

Figure 5. Photo. User viewing outdoor site through an HMD. 

Binocular and monocular HMDs can be directly compared by their image resolution, display 

frame rate in fps, and virtual image FOV in degrees. Table 3 compares currently available HMDs 

and lists the optical specifications and intended uses (based on manufacturer specifications).
24 

Table 3. Comparison of optical specifications and uses for HMD AR Devices. 

Device

Resolution 

(Pixels)

Frame 

Rate 

(fps)

FOV

(Degrees) Intended Uses

Binocular AR 

HMDs

— — — — 

Sony Smart 

Eyeglasses 

SED-E1

419 × 138 15 20 

Gaming, entertainment, AEC 

applications 

Epson Moverio 

BT-300 

1,280 × 720 30 23 Industrial and enterprise 

applications 

DAQRI Smart 

Glasses

1,360 × 1768 90 44 AEC applications 

ODG R-7 1,280 × 720 80 30 Surgeons, pilots, inspectors, 

construction 

ODG R-8 1,280 × 720 90 40 Surgeons, pilots, inspectors, 

construction 

ODG R-9 1,920 × 1080 60 50 Surgeons, pilots, inspectors, 

construction 

NVIS nVisor ST50 1,280 × 1024 60 50 Training, simulation, 

research

Microsoft 

HoloLens Gen2 

1,280 × 720 60 52 Gaming, entertainment, 

enterprise applications 

Magic Leap One 

Creator Edition

1,280 × 960 N/A 50 Gaming, entertainment, 

enterprise applications 

Monocular AR 

HMDs

— — — — 

Vuzix M100 Smart 

Glasses

420 × 240 N/A 15 Manufacturing and enterprise 

applications 

Google Glass 

Enterprise 

Edition 2 

640 × 360 N/A 15 Manufacturing, logistics, and 

healthcare

—No data available.

N/A = not applicable.

AR-Ready Software Applications 

While no AR applications specific to construction management could be found at the time of this 

research, several applications have been released that focus on the architectural design market. 

3D applications designed to facilitate viewing 3D design models on different devices have been 

on the market for a few years. Many of the developers of these applications are targeting AR 

devices that are becoming commercially available. This trend will continue to evolve, and as 

these applications and the supported devices become more stable and portable, the highway 

construction industry will see construction site–specific tools emerge. 
25 

Mobile Device–Based AR-Viewing Applications

Several off-the-shelf AR applications allow the user to display a 3D model over the live video 

feed of a mobile device. The application is provided a graphical target image, such as a plan, 

rendering, or photograph. The image is printed, and when the application on the mobile device 

recognizes the target image, the 3D model is displayed in alignment to the image as defined in 

the application. The view of the model is locked to the position of the target image in the video 

feed (figure 6). Many of these applications import standard 3D model formats, such as OBJ, 

DAE, or FBX®. 

© 2019 WSP. 

Figure 6. Photo. 3D view overlaid on live video on a mobile device. 

A photograph can be taken in the field and used as the target image for the AR application, 

which allows a 3D model to be embedded in a contextual view relatively accurately as though in 

a photo-simulation. The mobile device can then be pointed at the real-world scene from the same 

viewpoint, and the AR application will recognize the target image and superimpose the 3D 

model in the view. The simulation only works if the viewer remains stationary. No examples of 

applications specifically aimed at construction could be found, but these types of applications 

could be used to display a 3D model or graphical image of a project element or equipment when 

the device is pointed at a target image, potentially even a view of the real-world scene captured 

as a photograph. 
26 

Unity Support for HoloLens

The Unity development platform includes support for both the HoloLens HMD and the Windows 

10 Mixed Reality specification and devices. Many features and functionalities of the devices, 

including image recognition and tracking via Vuforia and gestural/audio navigation and controls, 

are supported in the gaming platform (Unity Technologies 2017). 

In a presentation at AU in 2017, the presenters demonstrated an implementation of the 

Unity/HoloLens platform for wall prefabrication and construction planning. The presentation 

covered strategies for deploying Autodesk Revit-based design models to the HoloLens and 

registering those models to real-world construction sites using a printed target image placed in 

the project site. The presentation showed a prototype live bidirectional connection between the 

HoloLens user viewing the model remotely on the construction site, and the deployed virtual 

model being displayed from the Revit design model in the design office (Jahangiri 2017). 

Trimble SketchUp Viewer for HoloLens 

The SketchUp 3D modeling platform now includes a separate 3D product called SketchUp

Viewer, which supports porting of SketchUp models to mobile devices and the Microsoft 

HoloLens HMD. Using the Viewer and the VR/AR extension to the SketchUp Desktop product, 

design models can be scaled and explored either in desktop mode or in full-scale immersive 

mode via the HoloLens. Scaling, navigation, and menu access are supported by HoloLens 

hand-gesture tools. A video demonstration shows a HoloLens user aligning the virtual model 

with a real-world scene to provide a registered and tracked AR view of the virtual model 

(Wheeler 2016).
27 

CHAPTER 3. CHARACTERISTICS OF AVAILABLE AR SYSTEMS

INTRODUCTION

AR is part of a spectrum of technologies that integrates digital information with the user’s 

environment in real time. AR is designed to augment a scene while a user maintains a sense of 

presence in the real world. The applications for using AR in construction are still evolving, but 

AR has the capability of augmenting traditional 2D drawings with digital 3D images to help 

facilitate construction inspection and review, QA, worker safety, training, and improved project 

management. A close companion to AR is VR, which involves a fully immersive environment in 

which a person’s senses are under control of a system, usually through an HMD. VR applications 

are also still being developed, but VR’s use for reviewing project design alternatives and 

stakeholder communication is showing strong potential, especially in collaborative 

environments. 

CHARACTERISTICS OF AR SYSTEMS

General AR system architecture can be described by the type of sensory input devices used, 

including any form of haptic input used for navigation or interaction with the scene and the 

display device (e.g., phone or tablet, eyeglasses, or head-mounted system, or HMD); tracking 

devices and software; the type of computing device (e.g., smartphone, tablet, onboard processor 

or remotely connected computing device); some form of media representation (e.g., textual/2D 

graphic and/or 3D imagery); and data-input mechanism (figure 15). 

© 2019 WSP. 

Figure 7. Illustration. AR system architecture. 
28 

AR Display Technologies 

Two categories of display types are found in AR systems: handheld devices, such as 

smartphones and tablets, and HMD devices, such as headsets or eyeglasses. The two display 

types offer different capabilities and provide unique challenges. The display types differ most in 

the way they display imagery to the user and in the way the devices track their position relative 

to the real world. 

Handheld Devices

These handheld devices are typically VST displays, which use the back-facing camera on the 

device to capture video of the real-world environment and display that image on the front screen. 

AR applications then register and render virtual data over the real-world imagery on the screen. 

With these displays, a user should hold the device above chest height close to eye level and at 

arm’s length to capture the widest FOV. Holding a device in this position can become difficult 

for a user over extended periods of time and could prove challenging in a construction-site 

environment because the user’s attention will be on the device and not on potential hazards at the 

construction site. 

Handheld devices typically use GNSS tracking to determine the initial user location within a few 

meters and then use the inertial movement of the device to change the view as the device is 

moved around the environment. Some vendors have demonstrated prototype applications that 

optically track the imagery in the video feed to support registration with the real-world view and 

to support tracking of user movement. Many commercially available AR applications use a 

marker-based positioning tool in which a target is placed in the real world and viewed by the 

video feed. This target is recognized by the software and used to register the user’s position in 

the real world relative to the virtual model.

One vendor has developed a unique solution that uses software and a powerful antenna to locate 

a mobile device to within a few centimeters. The device VIO sensors in combination with 

proprietary algorithms to help register the orientation of the device with the real world in real 

time. 

HMDs

HMDs are typically OST displays. A virtual image is presented to the viewer overlaid on the 

view out from the device and into the real world. The most commonly used approach in HMDs is 

a form of binocular near-eye displays that feed separate computer-generated images to each eye. 

Binocular near-eye displays offer more immersive and realistic experiences than handheld

devices because the real-world view is the direct view out, not a video feed, and the virtual view 

is typically stereoscopic. The scene changes as the user moves his or her head, creating a direct 

connection to the user scanning the real-world scene. Most HMDs on the market use a 

combination of inertial and visual tracking methods and are effective at tracking the real-world 

scene once a position for the user has been established. 

As with handheld devices, the initial positioning of the user with an HMD relies on GNSS 

localization or a marker in the field that the AR application recognizes and registers with the 

virtual model. AR application tools exist that allow 3D model elements from the real world to be 
29 

transferred to the device and used to register to the real world. These virtual model elements can 

be viewed over the real-world view then used to line up or register the elements from the model 

to the identical elements in the real world. 

AR Information Display Options 

There are several distinct categories of information display that AR can present to a user, each of 

which has unique applications on a construction site. 

Display What Is Not Yet Constructed

Incorporating AR into a BIM workflow allows users to see 3D design models in the real-world 

context. This is one of the most powerful opportunities with AR. The ability to overlay future 

construction on the real-world view allows users to do the following: 

• Compare design alternatives in context. 

• Check relationships between existing/future elements. 

• Examine site logistics and equipment movements. 

• Preview complex installation procedures. 

• Illustrate construction methods and sequencing. 

• Explore opportunities for training onsite. 

• Check traffic management plans/and temporary structures. 

Display What Was Intended to be Constructed

AR allows the user to overlay and compare the 3D design models (design intent) onto what was 

constructed in the field. Some potential opportunities this capability provides a user include the 

following: 

• Inspecting and validating sites.

• Checking for code/standards compliance. 

• Checking quantities and work progress. 

• Inspecting training opportunities. 

• Checking traffic management plans/and temporary structures. 

Display What Is Hidden from View

AR technology allows for the 3D display of existing elements not visible to the user in the 

real-world scene. This approach might provide challenges in display; for example, foreground 

objects need to be represented as transparent or cut-open for viewing what is behind them. This 

capability allows the user to visualize existing elements, such as buried utilities or structural 

components and elements obstructed from the current view. 

Display Abstract Information Aligned with Real-World Context 

AR technologies allow the alignment and display of abstract information that would typically 

only be available in a drawing plan view or virtual 3D model. These drawing elements can be 
30 

aligned and scaled in 3D to match visible real-world elements. Information that would prove 

useful on a construction site include the following: 

• Alignment information, easements, site boundaries, and right-of-way (ROW) boundaries. 

• Environmental boundaries, such as flood levels or sea level–rise data. 

• Archeological and historic sites. 

• Work-zone hazards. 

• Metadata tagged to associated real-world objects. 

AR could potentially support displays of other types of remote information, including video 

feeds from another user or documentation, instructions, or user guides that could be associated 

with real-world objects or activities. Abstract information geared toward the safety of the user 

could be displayed through AR; the system could monitor and display unsafe areas or risks, or 

the system could guide users safely through a complex construction site. 

Challenges for AR Systems in Construction

Construction sites, especially highway construction sites, are particularly challenging for AR 

systems because these sites are typically large, outdoors, and bright and often do not contain a 

variety of contextual information required for AR visual tracking systems. Buildings and 

structural sites contain a lot of 3D information that can be compared with virtual models to 

support localization and tracking for the AR user. Another issue will be the types of elements in 

a highway project. Elements of a highway project are typically large, smooth, and flat, and do 

not contain many fine details, which makes it much more difficult for 3D modeling systems to 

represent these projects in a virtual view where they will be easily recognizable by the user and 

the AR system. These 3D elements are much more difficult to simplify and optimize for display 

performance, which requires more forethought and preparation for their use in AR systems. 

AR systems provide many technological challenges, some of which apply to all AR applications 

and some of which are specific to construction applications. 

Performance and Portability

AR systems require significant processing power to support tracking processes and the real-time 

display of the virtual 3D model concurrently. AR devices must be standalone and portable to be 

used effectively in a construction environment. This means processing for tracking and display 

need to be onboard the device. Some systems are tethered to a separate wearable computing 

device, which reduces the necessary weight of an HMD. Larger 3D models, more accurate 

tracking, and increased display quality require more processing power. As AR systems evolve, 

there will inevitably be a tradeoff between the performance of the system and the size, weight, 

and comfort of the AR device. 

Portability of the 3D virtual model assets will be a challenge as well. 3D project models can be 

quite large, depending on the LOD and the area covered. The AR device must include onboard 

storage adequate for the model assets or use some form of connectivity solution that allows the 

model to be streamed in as required. One example of this was described in a presentation at AU

2017 in which the Microsoft HoloLens was used with Autodesk Revit. A remote Revit model 
31 

was converted in real time to a displayable format, displayed, and updated via a Wi-Fi

connection to the HoloLens device on the construction site (Jahangiri et al. 2017). 

Display Brightness

A key challenge with HMDs is the brightness of the overlaid virtual image onto the real-world 

view. In an OST AR device, the quality of the virtual image directly depends on the brightness of 

the real-world environment. Bright, outdoor scenes are the most difficult for the display systems 

to match. A typical highway construction site will be outdoors and bright, limiting the quality 

and usefulness of most current AR HMD devices.

Handheld devices are VST when used for AR, and the real-world display on the screen is 

controlled by the device. Device control allows the video display and virtual displays to be better 

matched in overall brightness, and it allows for better quality and greater realism of the displayed 

scene. In the next few years, handheld devices will most likely provide better opportunities for 

AR technologies on outdoor construction locations. Future HMD solutions based on VST

technology could be viable solutions for this display challenge. 

FOV

The available FOV presented to the user is a limiting factor of current devices. In handheld

devices, the display screen size and the video feed limit the FOV. If the FOV of the video feed 

does not closely match the FOV displayed on the screen, there is a disconnect with the 

real-world view. Additionally, the user must pan the device to view a large area, which could be 

prohibitive over long periods. 

Current HMD technology used in available devices is very limited in the FOV of the overlaid 

virtual model view, which is what the user sees. Most currently available devices display 

approximately 60 degrees of virtual view. To view a large area, the user must pan his or her head 

back and forth to fill in the scene. This can be distracting and tiring over a long period. 

As vendors improve the ability of devices to display larger FOVs, performance requirements for 

tracking and virtual model processing by the systems will increase. 

Occlusion

It is straightforward for AR systems to display virtual models on top or in front of the real-world 

image, regardless of the challenge of tracking and registering the two. In complex construction 

environments, there may be situations in which virtual elements in the model are behind 

real-world elements, which is called occlusion. The occlusion, or masking, of the hidden 

elements behind real-world elements is a complex problem for the AR system both in the 

calculation of the occlusion and in the display. When the occlusion is ignored or displayed 

poorly, the realism and immersive quality of the displayed scene are significantly affected. 

Current commercially available systems do not yet support occlusion rendering. 
32 

Safety

Safety issues on a construction site are of concern. Handheld devices must be held in front of the 

user, taking up one of the user’s hands and potentially blocking visibility of the site in front of 

the user. HMDs, even though they are hands-free tools, can limit the user’s peripheral view and 

block some of the audio, or site sounds, available to the user. As with any display device, some 

of the user’s focus may be on the device and not entirely on the user’s surroundings. These 

devices could be designed to be aware of safety issues and risks for the user because the device

would potentially know the user’s precise location on the site. 

Tracking Technologies 

GNSS tracking is the most straightforward method for localization in outdoor sites, but GNSS 

tracking requires visibility of the satellite network. GNSS provides reasonable precision for 

location but does not provide information on the orientation of the device. Solutions such as 

Trimble’s SiteVision device use a software/hardware combination that increases localization 

precision and provides orientation with GNSS as the primary tracking source. Tracking within a 

site can be enhanced with the addition of other position information, such as total stations or 

localized Wi-Fi networks. 

Most commercially available AR systems use a marker-based initialization of the scene to set 

position and orientation then use inertial and optical sensors to manage tracking and movement 

of the device. Optical tracking sensors require detail in the visible scene because the sensors 

calculate the movement of that detail in the optical imagery for tracking. A highway construction 

site will typically not include a lot of visual detail because they are typically flat. One solution to 

this lack of detail is to add more details to the real-world scene, such as markers or target images, 

or another type of 3D object. These systems could also be enhanced with the addition of Wi-Fi

nodes or position beacons. 

Data Support for AR Systems 

A key requirement for AR systems to be effective is the ability to register a virtual dataset to the 

real-world environment. Whether the AR system is tracking to a form of 2D information, such as 

a target or a position, or registering the real world to a virtual 3D scene, there are requirements 

for virtual scene data to be available to the AR system. This scene data will most likely be in the 

form of 3D digital models. These models are typically quite complex. Depending on the system 

used and the area of coverage, the models can include very large datasets. Current systems 

portable enough for construction applications typically need to carry the datasets onboard the AR 

device or the device must be tethered to the processing unit. 

AR technologies used in construction will require bringing project design data into the field. A 

workflow that involves segmentation, optimization, and streaming—control over LOD displayed 

or deployed to the field—will be needed. This workflow is especially important with AR 

applications in devices that have much higher performance requirements to maintain display 

frame rates and needed rendering quality. In addition to using a 3D cloud-based model 

distribution and sharing platform, 3D-model optimization schemes, whether through software or 

manual methods by designers, will need to be used to simplify the 3D models for AR workflows. 
33 

AR systems require front-end planning and setup to optimize these datasets for the AR system 

and application being used. This setup would typically involve filtering the dataset down to the 

most critical 3D elements for tracking and registration; the 3D elements need to be visualized in 

the AR system for the task being performed. 

Devices, such as tablets, smartphones, and some HMDs, allow wireless connections to an outside 

network. For these devices to work effectively in real time in the field, they would require a 

robust, high-bandwidth connection, most likely through a Wi-Fi network installation. Many 

construction sites now implement Wi-Fi networks to support communications and often to 

support a BIM workflow involving access to 3D datasets in the field. 

Software vendors have begun to develop cloud-based collaboration and distribution platforms to 

support the vendors’ design and construction support tools. Most vendors have demonstrated 

some support for AR devices and deployment of 3D models to the devices. AR-ready platforms 

include the following: 

• Autodesk: BIM360 Platform, Forge cloud-based developer’s platform. 

• Bentley: Bentley CONNECT Tools and Connection Client. 

• Trimble: Trimble Connect platform and Catalyst Tools. 

BIM Model Workflows for the Construction Site

AR applications for construction will most likely focus on either the display of annotation and 

graphical information that enhances the understanding of real-world objects or the 3D design and 

construction models that will be overlaid in position in the real-world environment view. Most 

AR applications are based on workflows and tools that are already in place in support of BIM 

applications for design and construction. These CADD/BIM tools will likely be a starting point 

for the development of AR tools, and many AR applications will eventually become extensions 

of current BIM and 3D model workflows rather than distinct processes. AR display of abstract or 

graphical information not georeferenced to the 3D design models will require new workflows 

and a different method for locating and registering data within the viewer’s real-world scene. 

The applications that would use this type of abstract virtual information need to be developed 

using the AR platform tools or AR hardware development tools currently available. 

Major vendors for 3D-design applications have implemented tools and workflows that support 

the deployment of 3D models to mobile devices for viewing in the field, and most support the 

built-in georeferencing capabilities of the devices. Most platforms support review and collection 

of data in the field through mobile devices, which can then be synchronized with project models 

and data in the design office. Most of these vendors have also enabled deployment of 3D models 

to one or more of the AR HMD models through mobile platforms. While most of the tools offer 

methods for optimizing models and model display on mobile devices with more limited 

processing power and storage, this optimization will be even more important for AR devices and 

applications that will typically require enough graphics performance to support real-time 

stereoscopic rendering of the 3D models. Processing power, available memory, data storage, and 

connectivity are limitations of most available AR HMDs.

35 

CHAPTER 4. STATE OF THE PRACTICE OF AR

INTRODUCTION

To provide a full picture of AR implementation challenges, potential costs and benefits, and 

technology use, further research was conducted into current AR implementations to document 

activities using AR technologies that support highway construction and related AEC 

applications. The research examined exploratory efforts into AR use for inspection and gathering 

of project data in the field using AR technologies. While very few instances of AR 

implementation that focused specifically on highway construction could be found during the 

investigation, several research studies and product prototype development activities were 

identified that were closely related to highway construction and inspection. These prototype 

development activities represent workflows and technology applications that would be an 

important aspect of the development of highway construction–specific applications. These 

activities are documented in this chapter. 

CURRENT APPLICATIONS OF AR SYSTEMS IN HIGHWAY CONSTRUCTION

The research team conducted an online survey of current research activities and a review of 

presentations and papers at highway and technology-related conferences. No project-level 

applications of AR systems were found in the highway construction area, but several research 

efforts and vendor product prototype studies were identified that were directed toward using AR 

systems in the highway construction area. Inspection and training applications of AR systems 

were also part of the research activities survey, and a few key research applications in these areas 

were identified and are included in this chapter. These examples do not show a complete 

beginning-to-end application using AR, but the studies demonstrate capabilities of AR 

technologies required to implement AR applications, such as the case studies described in 

chapter 6 of this report. These individual AR technologies would inevitably be part of a larger 

process required for the development of a commercial AR application intended for highway 

construction. 

Viewing Design Data in the Field with AR

Several examples of research were found where simplified, and sometimes relatively complex, 

3D-model data were ported to an AR device for display in the field. As described in chapter 3 of 

this report, this process is currently unique to the device and system platform being used. The 3D 

model data flows required are similar to BIM-to-field workflows being developed by 3D 

modeling and CADD software vendors. These workflows have been developed for the most 

common mobile-device platforms. For specific HMD devices, the workflow is unique to that 

device.

Trimble Connect is a platform that enables the deployment of 2D and 3D data to several different 

AR devices, including HMDs and mobile devices (Trimble 2018). In 2018, Utah Department of 

Transportation (UDOT) conducted a prototype study that explored the use of the Trimble 

Connect platform for porting and displaying simplified 2D and 3D design information onsite 

using the SiteVision mobile-device tool. In the UDOT study, the device was used to portray 2D 
36 

CADD information, such as ROW lines, edges of pavement, and other 2D design data, from 

Bentley MicroStation design files over a live video feed of the site. 

Researchers needed to separate the AR-viewable design data from CADD design files and then 

translate the design data to their correct proposed or existing elevation in 3D. This process is

separate from typical 2D CADD design processes and must be completed before porting to the 

AR device and bringing the device to the field, which then allows for those 2D CADD data to be 

displayed correctly in elevation relative to the real world, as shown in figure 8. The UDOT study 

also explored bringing in a few 3D design–model components. As shown in figure 9, some 

drainage pipes and other mechanical, electrical, and plumbing (MEP) elements were transferred 

as 3D model elements into the SiteVision device and then could be viewed overlaid on video of 

the real-world context. In this AR view (figure 9) the MEP components are shown as being 

underground using a 3D model of a cylindrical hole in the ground in front of the viewer. 

© 2019 Trimble. 

Figure 8. Graphic. Display of design data overlaid on the video display of a tablet device. 
37 

© 2019 Trimble. 

Figure 9. Photo. Display of 3D MEP overlaid on the video display of a tablet device. 

The UDOT study demonstrated the basic 3D-model data flows that are required of any AR 

application. Localization on this device is achieved by using basic GNSS tracking enhanced with 

proprietary Trimble sensing hardware and software, which are part of the SiteVision device. 

Trimble has focused its efforts on handheld, mobile devices for customized AR applications for 

construction primarily because of the display and tracking challenges of HMD devices. The 

Connect platform does, however, support data flows to the most common AR HMD devices. 

Bridge Lighting Design Application

A contractor conducted bridge lighting-design field tests using an HMD AR device and 

3D-model data ported from Autodesk 3DS Max into the Unity platform. The HMD software 

supported display and application development on top of the Unity platform. The 3D bridge 

model and lighting-design data were relatively high in detail and required some optimization and 

geometry reduction to get the model to port over and display on the HMD device, which must 

keep all data onboard for untethered field use.

Figure 10 shows a screen capture of an AR view of a bridge model with one of several design 

alternatives for custom lighting proposals overlaid onto the real-world view of the bridge. To 

enable these views, the user manually aligned the 3D model view with the real-world view in the 

field by using a virtual camera in the model that represented the onsite location. Once the view of 

the virtual model was close to the real-world bridge, the virtual model was nudged by the user in 

the display’s view until the model lined up as closely as possible to the real-world bridge in the 

viewer. Other studies have mentioned nudging the view to align virtual and real objects 

(Chen et al. 2018). The onboard tracking device maintained good registration after this alignment 

as the viewer moved. 
38 

© 2019 WSP. 

Figure 10. Graphic. Display of a 3D bridge lighting model in an AR HMD device. 

This example demonstrated another key advantage with some HMD devices: the use of built-in

audio commands to change view parameters. Audio controls help maintain hands-free operation 

by the user. The team that developed this bridge application mentioned the challenge of using 

see-through AR devices outdoors during the day. The bright light affects the ability of the virtual 

display of the device to be adequately visible over the real-world view. The device also had more 

difficulty tracking the position/orientation in bright light.

Autodesk BIM360 Docs Workflow 

Autodesk BIM360 Docs is an established platform for sharing data in the cloud and is used 

extensively on collaborative design and construction projects. An AU 2018 presentation by Chen 

et al. explored the results of a collaboration between DAQRI, Autodesk, and a contractor that 

leveraged the BIM360 Docs platform to support an AR application for validating design 

information in the field. The application was developed using the Autodesk Forge system, which 

is used to create data flows and interfaces within the 360 platforms. The application developed 

by the team accomplished several things that will be essential to AR workflows such as direct 

connection to a BIM 3D model workflow; deployment of 3D models from BIM360 directly to 

the AR device over Wi-Fi networks, and optimization of the 3D models within BIM360. The 

application used the DAQRI Smart Glasses wearable device. 

In this prototype application by Chen et al., 3D models were initially uploaded to the BIM360 

platform using workflows similar to traditional collaborative BIM process. The application 

demonstrated the ability to use multiple 3D-model types. One feature of the AR application 

allowed the user to select a model on the BIM360 platform and segment the model down to a 

manageable size using a 3D clipping box that could be changed in any dimension interactively. 

After the user segmented the model to a manageable size, the AR application prompted the user 

to insert a unique QR-coded target into a known location in the 3D model; the target is then used 
39 

in the field to initialize the position and view of the model with the HMD. The user logs the 

DAQRI headset into the BIM360 Docs platform via Wi-Fi, and the model(s) for the fieldwork is

selected and downloaded to the device. When the user is in the field, a printed version of the 

target is placed in the real-world scene that coincides with the virtual marker in the 3D model. 

The user initiates the field session by pointing the HMD at the target, after which the correct 

model is automatically loaded into the device for viewing. The model can be moved or nudged 

slightly to improve alignment in the view. The demonstration illustrates the concept that AR 

could eventually become a viewing extension to existing and future BIM 3D model workflows. 

The DAQRI device used in this application also exhibited a useful characteristic for interaction 

and navigation called gaze interaction. In gaze interaction, a reticle, or white dot, in the view is 

placed over menu items for a certain period and a selection is made, which opens interface 

menus, for example. This method allows for hands-free operation of the device and is not 

affected by background noise in the environment. 

AR Systems Supporting Inspection Activities 

Applications supporting onsite inspection will require digital capture or measurement of installed 

elements in the field; this is called inspection quantification. The process of inspection 

quantification is a key aspect of the construction-inspection process. Several studies identified 

field measurement using AR devices. One study applicable to highway construction was jointly 

conducted by the University of New Mexico, Los Alamos National Laboratory, and Los Alamos 

County (Moreu 2018). The study illustrated good results in collecting inspection data in the field 

using an AR HMD device. This study not only demonstrated the capture of linear and surface 

information in the field but also featured tests to evaluate the potential precision of the process of 

measurement in the field using the HMD device. In one test, a user measured a 12-inch target 

using the HoloLens from 12 separate locations on a 2-ft by 2-ft grid, thus providing various 

distances and angles to the target. Figure 11 illustrates the layout of viewer locations. 

Researchers found that the measurements were accurate within 1 to 2 percentage points, a 

precision the researchers felt was equivalent to the error of eye–hand movements with the 

device.
40 

© 2019 WSP. 

Figure 11. Graphic. Using an HMD to measure a target object.

In another application presented by Los Alamos County, users measured areas of newly installed 

concrete pads to calculate contractor payment amounts (Moreu 2018). The user captured points 

using a cursor within the AR device that is placed over real-world points, such as corners and

points along an edge, and uses finger gestures to activate the capture. Those points along with the 

surface of the existing condition are captured by sensors on the HMD, and then the points and 

surface data were downloaded and postprocessed in a 3D system in the design office to interpret 

the 3D locations of the points and the areas of the surfaces. Figure 12 shows 3D surfaces and 

points captured by the AR workflow process overlaid on the real-world view. 
41 

Source: LANL. 

Figure 12. Graphic. 3D surfaces and points as captured by an AR HMD (Moreu 2018). 

In addition to collecting data using an HMD, standard survey devices were used to collect a 

separate set of concrete pad measurements. The HMD measurements were then compared to the 

standard survey measurements (figure 12, figure 13, and table 4). The HMD measurements were 

found to be within 1 to 2 percent of the survey data—a differential the research team concluded 

to be within tolerances acceptable to contractors.

Source: LANL. 

Note: All axis measurements are in meters.

Figure 13. Graphs. Results of postprocessing surfaces and points as captured by the AR 

HMD device (Moreu 2018).
42 

Table 4. Comparison of HMD field measurements against county measurement. 

Area

County Measurement 

(ft2) 

HoloLens Average 

Measurement (ft2) 

Difference 

(ft2) 

Percent 

Difference

1 187.98 191.5 3.52 1.9 

2 147.67 149.0 1.33 0.9 

3 129.00 127.4 1.60 1.2 

Los Alamos County field tests also included a demonstration of measuring ramp slopes to 

evaluate potential requirements for compliance with the Americans with Disabilities Act (ADA). 

The user collected 3D points along the ramp using the HMD, and then the points and scanned 

surface were processed to determine point elevations and the slope of the measured ramp 

(figure 14).

Source: LANL. 

Figure 14. Graphic. Surface and points captured in ramp grade measurements made in the 

field with HMD (Moreu 2018). 

In a separate study, researchers at the Los Alamos National Laboratory demonstrated AR 

applications for inspecting the condition of concrete structures (Moreu 2018). The types of 

applications explored included measuring the deformation of structures after an event and the 

volume of damaged areas on structures. One application tested in the field was a crack 

measurement tool. In this prototype application, an inspector traced the line of a crack through 

the HoloLens device, and the device captured the drawn line and the scan of the surface of the 

concrete (figure 15). The user was able to add comments and metadata when using the 

application. 
43 

 Source: LANL.

Figure 15. Graphic. Screen capture of an AR tool for measuring cracks in concrete 

structures (Moreu 2018).

Tablet-Based AR Field Applications

Applications like those described earlier in this chapter, which illustrate using HMDs for 

collecting 3D field data, could not be found for tablet-based devices. Applications on tablet or 

smartphone devices would inherently be less precise when capturing points through the screen 

interface in the field. However, tablet devices will most likely be the platform on which AR 

applications will be developed in the near future because of their low cost, simplicity, and ability 

to be used outdoors. One negative aspect of using tablets and smartphone devices, however, is 

their need to be handheld, thus eliminating the opportunity for hands-free operation. Tablets are 

more abstracted from reality, like looking at a photograph, and less immersive than HMD

devices through which the viewer is seeing the world directly. Nevertheless, tablets and handheld

mobile devices have the advantage of allowing multiple users to concurrently see the same view 

on a single device.

CONCLUSIONS

While these examples do not represent comprehensive AR applications for construction, and are 

not all specific to highway construction, they do represent key technologies and workflows that 

would be essential to highway construction AR applications. The examples demonstrate that the 

required building blocks, or capabilities, of AR technologies exist. The investment needs to be 

made by technology developers to refine AR technologies and integrate them into larger 

construction-focused applications. Workflows and applications are used as examples to develop 

the case studies described in chapter 6 of this report.

45 

CHAPTER 5. AR FOR TRANSPORTATION CONSTRUCTION WORKSHOPS

INTRODUCTION

AR is widely recognized as a cutting-edge technology to augment human perception. This 

technology consists mostly of proof-of-concept prototypes in design and construction. This 

current state of technology presented the research team with the unique opportunity of engaging 

industry stakeholders to identify future needs of AR in transportation construction, which will 

help guide future AR development to maximize its impact.

Two workshops were conducted to explore AR’s application in transportation construction. The 

first workshop was held on May 9, 2018, at FHWA’s Turner-Fairbank Highway Research Center 

in McLean, VA. The second workshop was held on January 16, 2019, at the Transportation 

Research Board (TRB) Annual Meeting in Washington, DC. Both workshops involved executive 

and senior-level participants from State DOTs and FHWA with diverse backgrounds, 

senior-level representatives from private consultants and contractors, and representatives from 

AR hardware and software industries. 

The goals of the workshops were to inform the research priorities and anticipate future directions 

and opportunities for AR in highway construction. The workshops included technology and 

application developers, State DOT representatives, contractors, consultants, and other 

practitioners who explored how AR technology can be used to support construction-management 

functions during highway construction, ranging from construction inspection and review, QA, 

training, and improved project management through real-time information in a real-world 

environment. The workshops also informed participants on the current capability of AR 

technologies and related applications in MR, identified potential new applications for AR 

technology in transportation construction, and prioritized future applications of AR technology in 

transportation construction. Finally, the workshops involved interactive experiences for the 

participants that explored the capabilities of the different types of AR devices and facilitation of

meaningful dialog about future applications. 

WORKSHOP HIGHLIGHTS

Workshop 1 

Both workshops followed similar agendas. The first workshop began with an overview of AR 

technologies and characteristics. This first session served as a synopsis of chapters 3 and 4 of this 

report. The next session of the workshop included presentations by vendors of AR technologies 

and supporting software applications. The last session was comprised of facilitated discussions, 

or brainstorming sessions; all participants explored opportunities for and benefits of using AR 

technologies in highway construction and the associated challenges each participant group found 

in implementing the AR technologies. A short list of the technologies and opportunities was 

captured, and participants’ input on each case’s impact and feasibility for development was 

obtained through electronic polling. Workshop 1 produced five defined use cases of AR 

technologies in transportation construction, which are described in the Results section. 
46 

Workshop 2 

The second workshop was divided into two parts. Session 1 began with an overview of FHWA’s 

exploration of immersion environments. This overview was followed by a state-of-the-practice 

review completed by the research and published in chapter 4 of this report. The review was then 

followed by a case study of Nevada Department of Transportation’s (NDOT’s) use of virtual 

technologies to simulate construction phases during a project’s development. The session

concluded with a presentation of the five use cases of AR technology in transportation 

construction (which were identified in workshop 1) and electronic polling regarding each case’s

impact and feasibility for development. Session 2 of the workshop included several presentations 

that explored R&D work involving AR and VR by several DOTs and consultants. 

Results of the Polling

Based on polling data from both workshops, use cases were ranked in terms of their potential 

overall impact on transportation construction as well as the feasibility of developing each use 

case. This final ranking was as follows: 

1. AR SUPPORT OF ROW ACQUISITION. Provide project visualization to property 

owners to better understand a project’s impact and to show design options to the property 

owner. 

2. VISUAL VARIANCES. Tag variances in the field regarding schedule and quality issues 

to ensure all parties are viewing the same issue. 

3. INSPECTOR’S TOOLKIT. Use AR technology to verify that work is being installed 

properly by providing the correct information in the correct format for the inspector in the 

field. 

4. NEXTGEN TRAINING AND CERTIFICATION. Use AR technology to support 

training and certification for construction inspection. 

5. AUTOMATED INSPECTION. Use AR technology to support automated 

code/standard-compliance checking of installed items through machine learning. 

The following sections review the outcomes of each workshop and examine how the preceding 

rankings were achieved. 

WORKSHOP SUMMARIES

Workshop 1 Summary 

Session 1

Workshop 1, held at FHWA’s Turner-Fairbank Highway Research Center in McLean, VA, 

began with an overview of AR technologies and characteristics. The participants were 

encouraged to engage in the discussion of challenges and opportunities for AR technologies 

introduced in the presentation from their individual perspectives. No specific examples reflecting 

the use of AR in highway construction could be identified at this time, so examples from related 

industries were summarized and shared with the audience to elicit commentary and feedback. 

Key points of the presentation and discussion were transcribed and included in the following 

sections: 
47 

• AR is evolving. There are several emerging advanced visualization technologies, 

including VR, MR, and AR. Of these, VR is the most mature technology. There are 

primarily two forms of AR display types: handheld and HMD. Handheld displays are 

becoming increasingly more common due to advances in camera and display capabilities 

of smartphones. While AR has traditionally used marker-based tracking systems, GNSS 

and Wi-Fi networks and optical sensors are advancing AR tracking. 

• AR has limitations. Limited FOV, which is common among current HMDs, restricts AR 

technologies. AR devices are typically intended to be portable; therefore, their 

performance relies on the processing speed of the portable devices. While the 

performance of AR devices is being addressed by technology developers, the increasing 

desire of having AR devices interact with an environment rather than simply displaying 

information poses additional challenges to the processing performance of future AR 

systems. Another limitation of AR technology is difficulty when seeing AR displays in 

bright environments. These limitations, coupled with the challenges of AR tracking in 

outdoor environments, are significant when using AR in construction applications. 

• AR display options are varied. Among the AR prototypes that have been explored in 

construction, AR display options include displaying what is not constructed, displaying 

what is hidden from view, overlaying or tagging abstract information, using remote 

communication, and ensuring safety.

• AR can be used to influence both performance and behavior. AR technologies can 

reduce planning and construction time, errors, and rework. In terms of behavior, AR 

technologies can more effectively identify safety hazards. 

• AR-ready platforms already exist. Construction-specific platforms for AR devices

include Autodesk BIM 360 (Forge), Bentley CONNECT, and Trimble Connect. General 

AR platforms include Apple ARkit, Google ARCore, Windows Mixed Reality, AR 

Toolkit, and WebAR. 

• Future questions about AR remain. How much information does the user overlay? 

Does the user show specific details, or do they show the overall project? How does the 

user utilize AR to display uncertainty, especially regarding uncertainty in accuracy? 

(There are two types of accuracy: accuracy of the model and accuracy of the overlaid AR 

display.) Is there a way to verify the accuracy of the model? This verification should 

occur during the modeling process, not in the field. 

Session 2

Session 2 included presentations by vendors of AR technologies and supporting software 

applications. Vendors responded to an invitation to participate in the session, and each vendor 

gave an overview of the technologies and the potential application for highway construction. 

After the presentations, participants had the opportunity to test the technologies firsthand and 

discuss the tools directly with the vendors. 
48 

Session 3 

The last session of the workshop included facilitated discussions and brainstorming. All 

participants explored opportunities for and benefits of using AR technologies in highway 

construction and the associated challenges each participant group found with potentially

implementing AR technologies. A list of the technologies and opportunities was captured, and 

participants voted on the application they felt provided the most opportunity for improving the 

state of the practice from the perspective of their discipline.

Session 3 began with a series of brainstorming activities that encouraged participants to examine 

the potential application of AR technologies in transportation construction. The first 

brainstorming activity focused on different activities related to construction and how AR 

technologies could be applied to those activities. Discussions on potential AR construction 

applications were coordinated around four subjects: inspection applications, preconstruction, 

human resource development (training), and project management. The subcategories of 

applications are detailed in the next section. Within these four subject areas, there were a few 

trends that emerged in the brainstorming discussions. One trend focused on visualization of 

design information, especially information that would be easily accessible from existing 3D 

model workflows, including simplified 3D model components or abstract information, such as 

ROW or alignment information. Another trend that emerged were applications that would 

support construction-inspection activities, such as comparing as-built components with 

as-designed virtual information or with inspection criteria and specifications. A third trend was 

using AR for training, particularly for construction-inspection activities.

Use of AR to Support Inspection Activities. The brainstorming discussion focused on the use 

of AR to support the inspection process, which ranged from helping the inspector identify what 

needs to be inspected to providing the inspector with the information to complete the inspection. 

Specific applications of AR technology for assisting with inspection include the following: 

• AR-guided site inspection: 

o Use AR to locate where the inspector needs to go. 

o Use AR to provide the right information in the right format to the inspector and 

provide situational measurement (e.g., virtual checklists help the inspector verify the 

work is being installed properly and the right materials are being used). 

• Remote inspection: 

o Use AR technology to capture inspection data (visual) that can be viewed later. Video 

capture of inspection would be especially helpful when inspecting remote sites.

• 4D inspection: 

o Visualize just what the inspection needs, which will be based on the contractor’s

schedule. 

• Bridge inspection: 

o Use AR for validating of rebar installation, checking bridge deck thickness, and 

mapping of cracks on structures. 

• Earthwork: 

o Ensure ADA compliance by verifying that slope percentages comply with ADA 

specifications while also noting the x, y, and z coordinates of each ramp.
49 

• Automated inspection: 

o Integrate the use of AR technologies and machine learning to allow the AR 

application to measure if the system/component has been installed correctly. 

Use of AR in Preconstruction. The ability to visualize design details on a proposed site using 

AR technology helps to examine in exact context potential conflicts between design and actual 

site conditions, which can be difficult in VR or when using a 3D model. Specific applications of 

AR technology during preconstruction that were discussed include the following: 

• Constructability reviews:

o Use AR technology to visualize clash detections by overlaying proposed design 

changes within the context of the built environment. 

o Examine site layout including access planning, temporary resources, execution. 

o Provide the contractor with an AR model to allow the contractor to better understand 

the risks when developing cost estimate. 

o Use AR technology to visualize phasing/mobilization/initial start. 

o Use AR technology to visualize the boundaries of neighboring/simultaneous 

construction activities to help facilitate site coordination. 

o Apply AR technology to support automated code/standard compliance of proposed 

designs. 

o Use AR technology to support real-time site evaluation of safety hazards to help 

establish safe work zones. 

o Use AR technology to alert craft workers of active hazards (e.g., overhead loads or 

unsecured oxygen/acetylene tanks). 

o Use AR technology to capture the knowledge and virtually share lessons learned to 

field supervisors and construction crews. An idea was described by participants of 

AR having the capability to visually recognize the scene and identify relevant lessons 

learned. 

• Utility relocations/ROW acquisition: 

o Capture newly installed utilities (location, material, etc.) digitally for future work. 

o Provide visualization to the property owner to show options and to understand a 

project’s impact.

• Claims avoidance reviews: 

o Tag variances visually in schedule and quality so that both sides are viewing the same 

issue (if the property owner and contractor are looking at the same issues in real time, 

a claim may never be filed).

o Verify distances and installation with a digital recording of the project being built so 

that details can be reviewed later if needed.

• Stakeholder outreach (communication): 

o Streamline and assist with National Environmental Policy Act (NEPA) requirements 

and upholding other project commitments (e.g., assisting the public and other Federal 

partners to better understand a project’s impact, and visualizing technical alternatives 

and their impacts on NEPA commitments).

o Design visualization with project stakeholders (e.g., how the project will impact 

neighboring businesses).
50 

Human Resource Development (Training). With the current workforce shortage in 

transportation construction, discussions focused on the use of AR technologies to facilitate 

training. Topics included the following: 

• Support junior inspectors by having onsite senior inspectors observe a scene remotely, 

essentially sharing the view. 

• Provide virtual and intelligent training manuals and videos and animations for inspectors 

for training. 

• Supporting training and certification for construction inspection using AR technology. 

Project Management. Using AR technologies to monitor progress, track QA, and analyze 

traffic maintenance was explored in the brainstorming discussion. Topics included the following: 

• Progress monitoring/quantity tracking: 

o AR technologies support automated as-built models. 

o AR technologies support obstacle avoidance on heavy machinery. 

o AR technologies facilitate faster approval of pay estimates (the AR technology shows 

what is being claimed in the pay estimate has been installed [i.e., visual checkbox of 

installed items]).

• QA: 

o Overlay defect maps based on nondestructive evaluation data on the installed facility. 

• Asset management: 

o AR technologies support real-time availability of onsite construction assets. 

o Automated inventory of roadside assets using AR. 

• Traffic management/maintenance of traffic: 

o AR technologies support work-zone safety (highlighting current areas of hazards 

during construction). 

• Automated layout of onsite installations: 

o Systems that guide the contractors during installation (sequencing and location). 

Currently used in MEP, but there are similar complex systems in bridges, transit 

systems, lighting, toll buildings, toll gantries, and conduit. 

Challenges for AR Applications in Construction

During the third session, participants were asked to provide examples of potential challenges. 

The brainstorming session focused on two areas. The first area included technical challenges 

related to the AR system’s hardware and the 3D-model workflows required to support AR 

applications. The second area that emerged focused on the challenges within agencies when 

implementing AR systems, both culturally—the change in workflows and the acceptance of new 

and untried technologies—and financially, specifically the potential costs required for new 

computing systems and for the labor required to support more advanced 3D-modeling 

workflows. 

Technology-Related Challenges. Technical challenges for AR applications in construction 

include actual technical performance as well as the perception of the AR’s performance, which 

can be attributed to its limited use in transportation construction. Challenges were discussed 
51 

regarding the efforts required to develop AR systems and related applications for the 

transportation-construction field. Specific challenges include the following: 

• Model size versus AR system capabilities. Distilling a large project model down to a 

size that can be supported on a mobile device to support AR display can be a challenge. 

Some elements in the model can be simplified (e.g., bolts) without affecting the fidelity 

of the mode; however, the model may need to be adjusted to precisely fit the mobile 

device’s capability. Currently, this workflow between BIM and AR systems is a very 

tedious and nonautomatic process. 

• Research and development (R&D) needed for AR applications in transportation 

construction. Participants discussed challenges related to the level of R&D required to 

support the development of new technology, including AR technology for construction. 

One challenge focused on not knowing the anticipated return on investment (ROI) if AR 

technologies are successfully developed and implemented. This challenge could be 

addressed through State transportation agencies (STAs) exploring and examining the use 

of AR technology on a pilot project to support the assessment of value. Private industry 

might race to implement higher-risk technologies to gain market share, so, alternatively, a 

pilot project might not make the most sense for public agencies.

• System durability. The commercial use of AR systems in exterior working environments 

remains challenging. Part of the challenge relates to the comfort of using HMDs for 8 to 

10 continuous hours. Additional challenges include the poor quality of displays in direct 

sunlight and battery life over an 8- to 10-hour workday. Better understanding of how 

much display information is required to make AR systems effective could address the 

challenge of using AR systems in an outdoor working environment. There is also 

uncertainty regarding ADA requirements for the use of AR across a broad community. 

Questions also remain regarding the amount of technical support required as well as the 

ability of the devices to work independently without Internet connectivity. 

Implementation-Related Challenges. Participants identified that expertise in 3D modeling, 

required training, and information security are current challenges to AR implementation. 

Challenges to agency implementation include: 

• Existing use of 3D modeling. Not all State DOTs use 3D modeling and AR technologies 

effectively. Targeting one or two potential application areas to demonstrate the value of 

AR will help advance adoption. 

• Training and implementation. Participants identified challenges in the lack of 

successful implementation plans, training tools, and demonstrated enterprise approaches 

to successfully implement AR technologies. One strategy for successful implementation 

is to make the AR interface simple. Voice-activated AR systems are not necessarily 

intuitive.

• Security. Some agencies do not want models of existing structures on a cloud-based 

server. Future strategies should focus on data warehousing and how it can be used to 

support AR technologies. 
52 

Electronic Ranking of Application Concepts 

Workshop participants voted on the applications that they felt would have the most impact on 

highway construction processes from their professional perspective. The top ten application ideas 

were captured for further polling using electronic voting devices and software that could rank 

and plot voting results. The top ten applications included the following: 

• AR to support site planning and logistics. 

• AR support of automated code/standard compliance of proposed designs. 

• AR support of road safety audits and hazards and hazard recognition to the traveling 

public. 

• Visualization to the property owner to better understand a project’s impact and to show 

options to the property owner. 

• Visually tag variances in schedule and quality issues so that both sides are viewing the 

same issue. 

• Streamline/assist with NEPA requirements and upholding other project commitments. 

• AR to support training and certification for construction inspection. 

• Verification that work is being installed properly by having the right information in the 

right format for the inspector. 

• AR to facilitate faster approval of pay estimates. 

• Automated layout of systems. 

Participants were then provided electronic voting devices and were asked to vote on each 

application based on the following criteria: 

• What would be the impact (positive) of the application from their professional 

perspective?

• What would be the feasibility of developing and implementing the application from their 

professional perspective? 

Each application idea was ranked according to its impact and feasibility. Rankings ranged from 

1 (lowest) to 10 (highest) for each criterion. Results were tabulated and plotted by impact versus 

feasibility. Data were collected from 26 respondents; the breakdown of the voters by discipline 

was as follows: 

1. Federal: six voters (23 percent). 

2. State: six voters (23 percent). 

3. Technology Developer: two voters (35 percent). 

4. E&C Industry Firms: nine voters (11 percent). 

5. University: three voters (8 percent). 

Workshop 1 Results 

The top five applications ranked in order of potential impact (high to low) are listed with their 

overall impact and feasibility ratings (based on average of rankings from 1 to 10). 
53 

1. INSPECTOR’S TOOLKIT. Use AR technology to verify that work is being installed 

properly by providing the correct information in the correct format for the inspector in the 

field. 

Impact: 7.54; Feasibility: 3.70. 

2. NEXTGEN TRAINING AND CERTIFICATION. Use AR technology to support 

training and certification for construction inspection. 

Impact: 7.24; Feasibility: 4.73. 

3. AR SUPPORT OF ROW ACQUISITION. Provide project visualization to property 

owners to better understand a project’s impact and to show design options to the property 

owner. 

Impact: 7.12; Feasibility: 6.10. 

4. AUTOMATED INSPECTION. Use AR technology to support automated 

code/standard-compliance checking of installed items through machine learning. 

Impact: 7.08; Feasibility: 3.44. 

5. VISUAL VARIANCES. Visually tag variances in the field regarding schedule and 

quality issues to ensure all parties are viewing the same issue. 

Impact: 6.92; Feasibility: 4.54. 

Plotting the Results of Workshop 1 

Figure 16 shows distribution of the polling results for the original 10 application concepts from 

workshop 1. The results are plotted with impact on the industry versus feasibility of development 

and implementation of the proposed AR applications for the overall group of participants. 

© 2019 WSP. 

Figure 16. Graph. Overall polling results based on workshop 1. 

Figure 17 through figure 19 show distribution of the rankings regarding impact on the industry 

versus feasibility of development and implementation of the proposed AR applications based on 

the participants’ discipline. 
54 

© 2019 WSP. 

Figure 17. Graph. State and Federal respondents’ polling results based on workshop 1. 

© 2019 WSP. 

Figure 18. Graph. Engineering and contractor respondents’ polling results based on 

workshop 1. 
55 

© 2019 WSP. 

Figure 19. Graph. Research and developer respondents’ polling results based on

workshop 1. 

Workshop 2 Summary 

The second workshop at the 98th Annual Meeting of the Transportation Research Board (TRB) 

was intended to bring technology and application developers, State DOT representatives, 

contractors, consultants, and other practitioners together to explore how AR technologies can 

support construction-management functions during highway construction. Topics ranged from 

construction inspection and review, QA, training, and improved project management through 

real-time information in a real-world environment. Like the first workshop, the second workshop 

was designed to educate participants on the current capability of AR technologies and related 

applications in AR/VR/MR, identify potential new applications for AR in transportation 

construction, and prioritize future applications of AR in transportation construction. The 

workshop was designed as an interactive experience for the participants to explore the 

capabilities of the different ranges of AR devices and to engage in meaningful dialogue about 

future applications. 

Session 1 of the workshop began with an overview of FHWA’s exploration of immersion 

environments, which was followed by a state-of-practice review completed by the researchers

and included in chapter 4 of this report. These presentations were followed by a case study of 

NDOT’s use of virtual technologies to simulate construction phases during a project’s 

development. The session concluded with a presentation of the five application use cases of AR

technology in transportation construction identified in the May 2018 workshop, and electronic 

polling regarding each case’s impact to highway construction and feasibility for development 

and implementation. 
56 

Polling Results of the Use Cases

Participants were provided electronic voting devices as in the first workshop and were asked to 

vote on each application based on the following criteria: 

• What would be the impact (positive) of the application from their professional 

perspective?

• What would be the feasibility of developing and implementing the application from their 

professional perspective? 

Prior to polling on each use case, participants identified their primary affiliation, which included 

the following: 

1. Federal: 16 percent. 

2. State: 20 percent. 

3. Technology Developer: 8 percent. 

4. Engineering and Construction Industry Firms: 32 percent. 

5. University: 24 percent. 

Workshop participants voted on each application according to its impact and feasibility from 

their perspective. The uses cases are ranked below by the newer potential overall impact and 

feasibility rating with results from the TRB workshop only (based on average of rankings from 

1 to 10): 

1. AUTOMATED INSPECTION. Use AR technology to support automated 

code/standard-compliance checking of installed items through machine learning. 

Impact: 7.42; Feasibility: 3.61. 

2. VISUAL VARIANCES. Visually tag variances in the field regarding schedule and 

quality issues to ensure all parties are viewing the same issue. 

Impact: 7.00; Feasibility: 5.15. 

3. AR SUPPORT OF ROW ACQUISITION. Provide project visualization to property 

owners to better understand a project’s impact and to show design options to the property 

owner. 

Impact: 6.94; Feasibility: 5.67. 

4. NEXTGEN TRAINING AND CERTIFICATION. Use AR technology to support 

training and certification for construction inspection. 

Impact: 6.61; Feasibility: 5.70. 

5. INSPECTOR’S TOOLKIT. Use AR technology to verify that work is being installed 

properly by providing the correct information in the correct format for the inspector in the 

field.

Impact: 6.51; Feasibility: 4.55. 

Figure 20 shows the distribution of polling results regarding impact on the industry versus 

feasibility of development and implementation of the proposed AR applications for the overall 

group of participants in workshop 2.
57 

© 2019 WSP. 

Figure 20. Graph. Overall polling results based on workshop 2. 

Figure 21 through figure 23 show distribution of the rankings regarding impact on the industry 

versus feasibility of development and implementation of the proposed AR applications based on 

the participants’ discipline.

© 2019 WSP. 

Figure 21. Graph. State and Federal respondents’ polling results based on workshop 2. 
58 

© 2019 WSP. 

Figure 22. Graph. Engineering and contractors respondents’ polling results based on 

workshop 2. 

© 2019 WSP. 

Figure 23. Graph. Research and developer respondents’ polling results based on 

workshop 2. 

After polling was completed, participants explored hands-on demonstrations of AR systems 

provided by several invited technology developers already in attendance at the TRB Annual 

Meeting.
59 

Session 2

Session 2 included several presentations that explored R&D work involving AR. The 

presentations included the following: 

• Iowa Department of Transportation’s (IowaDOT’s) Use of Virtual Reality in 

Design. The presentation highlighted IowaDOT’s use of VR, including VR’s use in 

design reviews and influence on driver behavior. 

• UDOT’s Use of SiteVision (Trimble). SiteVision is an AR system designed for exterior 

environments. At this time, the system is in development and being beta tested; one pilot 

test was conducted on a UDOT project. 

• Hybrid Reality: Visual and Structural MR (Heriot-Watt University). Research at 

Heriot-Watt University has explored the use of MR systems to support both design 

visualizations and hazard-recognition training. 

• Use of Advanced Visualization Techniques for Explorations in Built 

Environments (University of Southern California). Research supported by the 

National Science Foundation highlighted the use of visualization technologies to explore 

human behavior during emergency events. 

• AR for Bridge Lighting. AR was used to explore different lighting options on existing 

bridges. 

FINAL RANKING OF THE AR APPLICATIONS

When the polling data from both workshops 1 and 2 were combined, 87 responses from industry 

experts provided the following final ranking: 

1. AR SUPPORT OF ROW ACQUISITION. Provide project visualization to property 

owners to better understand a project’s impact and to show design options to the property 

owner. 

Impact: 7.02; Feasibility: 5.84. 

2. VISUAL VARIANCES. Visually tag variances in the field regarding schedule and 

quality issues to ensure all parties are viewing the same issue. 

Impact: 6.97; Feasibility: 4.88. 

3. INSPECTOR’S TOOLKIT. Use AR technology to verify that work is being installed 

properly by providing the correct information in the correct format for the inspector in the 

field. 

Impact: 6.88; Feasibility: 4.27. 

4. NEXTGEN TRAINING AND CERTIFICATION. Use of augmented reality supported 

training and certification for construction inspection. 

Impact: 6.88; Feasibility: 5.27. 

5. AUTOMATED INSPECTION. Use of AR to support automated 

code/standard-compliance checking of installed items through machine learning. 

Impact: 6.71; Feasibility: 4.17. 

The distribution of the polling scores based on both workshops is shown in figure 24 through 

figure 27. 
60 

© 2019 WSP. 

Figure 24. Graph. Overall polling results based on both workshops 1 and 2. 

When the overall polling data are segregated by the demographics from both workshops 

(figure 24 through figure 26), differences by the type of respondent appear. Respondents from 

Federal and State DOTs indicate ROW visualization as the top impact followed by visual 

variances, nexgen training, automated inspection, and inspector’s toolkit (figure 24). While E&C

and R&D participants viewed other use cases as having a higher impact, both groups of 

participants gave visual variances a relatively low feasibility score (figure 26 and figure 27), 

which suggests the visual variance use case is an application requiring a relatively long-term 

time frame to develop. The ROW visualization use case had the highest feasibility score among

Federal, State, and E&C respondents (figure 24 and figure 25). While ROW visualization ranked 

relatively lower on the impact scale, it may still be an application worth exploring through a pilot 

effort to explore AR technology’s impact but also explore the general use of AR technology in a 

transportation-construction environment. The challenge of developing a georeferenced AR 

system that integrates with a BIM model among Federal, State, and E&C respondents was noted 

as a significant technical challenge in developing the ROW acquisition use case. This use case, 

however, ranked relatively high in terms of its feasibility among all respondents. 
61 

© 2019 WSP. 

Figure 25. Graph. State and Federal respondents’ polling results based on both 

workshops 1 and 2. 

© 2019 WSP. 

Figure 26. Graph. Engineering and contractor respondents’ polling results based on both 

workshops 1 and 2. 
62 

© 2019 WSP. 

Figure 27. Graph. Research and developer respondents’ polling results based on both 

workshops 1 and 2. 

The preceding use cases are described in more detail in chapter 6 of this report along with 

detailed descriptions of the potential data requirements and data flows, AR technologies, and 

user workflows that would be required to implement the activities.
63 

CHAPTER 6. AR FOR CONSTRUCTION MANAGEMENT, INCLUDING INSPECTION 

AND TRAINING

INTRODUCTION

Based on audience input and discussions in the workshops described in chapter 5 

(i.e., workshops 1 and 2) and the research results of this report, the top five AR application use 

cases identified in both workshops have been expanded and augmented in this chapter to include 

the following additional criteria: 

• A summary of the goals of the individual activity. 

• A detailed description of the use case. 

• AR technologies that would be required to support the application. 

• A potential data/technology framework for the application. 

DEVELOPMENT OF DATA FLOW AND AR TECHNOLOGY FRAMEWORK FOR 

THE USE CASES

The research team developed a framework to describe the AR data, workflows, and technologies 

required to implement the use cases. The framework is based on the characteristics of AR 

systems, as outlined in chapter 3. Figure 28 is a flowchart representing the developed framework. 

Each box, or stage of the workflow, represents a critical step that would potentially be unique to 

each activity using AR technology. Each stage will be different for different data types; the data 

type will be driven by the specific requirements for the onsite activity. 

Different data types will have different delivery requirements depending on size and complexity, 

and the data platform will depend on the AR technology being used. Details in the framework 

stages will vary depending on whether tablet- or HMD-based display technologies are the target 

platform for the AR application. Localization, the positioning and orientation of the displayed 

data relative to the real-world view, will depend on how accurately the data must be overlaid, or 

registered, in the real-world view to accomplish the given activity. The onsite delivery and 

collection of information will be driven by the activity requirements but will also depend on 

technologies applied, such as display type and accuracy of localization. 
64 

© 2019 WSP. 

Figure 28. Flowchart. AR technology process.

Key aspects of the technology framework process include the following: 

• Data Types/Sources (data requirements for specific activities): 

o 3D design/BIM data. 

o Other (e.g., text or unstructured data).

• Data-Delivery Platform: 

o Cloud/network based. 

o Manual transfer. 

• Data-Delivery and -Collection Devices:

o Mobile- or tablet-based. 

o HMD. 

o Other (e.g., smartphone and measuring/camera). 

• Localization Technology (driven by the accuracy of position and orientation 

required for the activity): 

o GNSS. 

o Marker-based. 

o Local Wi-Fi/emitters/beacons. 

• Information Delivered Onsite (requiring localization): 

o 3D information on existing conditions. 

o 3D design information. 

o 4D or temporal information. 

o Additional 3D information (e.g., ROW and temporary easement). 
65 

• Information Delivered Onsite (not requiring localization): 

o Information on existing conditions. 

o Design information. 

o Other information, (e.g., design criteria and specification requirements). 

• Information Collected Onsite (requiring localization): 

o Notation, documentation, images. 

o 3D design changes. 

o Other information, such as request for information and queries. 

• Information Collected Onsite (not requiring localization): 

o Notation, documentation, images. 

o Other information (e.g., worker headcount). 

USE CASE EXAMPLES

This section provides a detailed description of the five case studies and outline of the framework 

applicable to each based on the preceding technology framework description, and includes 

describing the AR data, workflows, and technologies that would be required to implement the 

use case applications as described.

USE CASE 1: AR SUPPORT OF ROW ACQUISITION

Provide project visualization to property owners to help better understand a project’s impact as 

well as to show design options to property owner. 

Goals of the Application 

The acquisition of ROW is a critical step in a project’s development. The acquisition often 

requires negotiating and purchasing private property to support a new route or realignment of an 

existing roadway. Traditionally, negotiating involves displaying the proposed ROW

requirements using 2D plan sets (figure 29), and, when possible, using flags and survey stakes on 

an actual property to show the extent of the ROW requirement. Staking the proposed ROW can 

be a visual estimation without having to use a survey crew, which can be expensive and subject 

to time delays. Using 2D plan sets to show the extent of the ROW requirements can be difficult 

for a private landowner to understand, which can lead to misunderstandings that can cause

significant disruption during construction. 
66 

© 2019 WSP. 

Figure 29. Screenshot. Mockup of interface for an AR ROW application.

Use Case

A fundamental step in acquiring ROW on transportation projects is paying the owner of the 

property just compensation. Two critical components of this process are the valuation of the 

property and the presentation of the offer to acquire the property. The process can involve 

multiple negotiations stemming from disagreements on the valuation of the property, which can 

be due to a lack of a common understanding of the proposed ROW boundaries. AR technology 

allows for the ability to overlay the proposed ROW acquisition in a 3D context of the actual 

property, which can then be easily viewed from multiple perspectives.

In addition to showing the boundaries of a proposed ROW, AR technology could provide other 

benefits, including the following: 

• Better understanding of proposed grade changes that may be involved along the 

boundaries of the ROW. 

• Improved community relations through enhanced communication of the project to project 

stakeholders. 

• Improved communication between a private landowner and the designer on different 

design options with varying ROW boundaries. 
67 

The ability of AR systems to provide the visualization of a ROW is feasible with current AR 

systems. The ability of AR systems to tailor information to a specific geolocation has been used 

in prior commercial applications, although the accuracy of these applications may not be 

sufficient to support negotiation and presentation of a proposed ROW. Highly accurate AR 

applications that can operate in an exterior environment are currently being developed by at least 

one technology developer. The use of tablet or HMD applications would support this application; 

tablet devices would have the advantage of being accessible to multiple users simultaneously. 

AR Technology Framework 

Data Types/Sources

Primary data types to best support this application include design-based (e.g., CADD) 

information, such as ROW lines, alignment centerlines and stationing, edge of construction lines 

(e.g., paving/grading, and simplified 3D models of design elements. Most of these elements 

would be part of a typical design dataset; the elements would need to be separated from other 

data. These types of data are typically 2D, except for alignment and corridor surface data that 

could be generated from civil design tools. 3D alignment and corridor surfaces would most likely 

need to be sorted out and reduced in complexity from the overall design model. 2D data are 

typically developed with a z-value, or elevation of zero. To display this information correctly in 

the field, the data would need to be translated to the correct elevation of the data for the proposed 

design. If the terrain or proposed line work involves complex terrain, the 2D data may need to be 

draped or converted to 3D using z-values from the terrain. This is a step that would need to 

happen before porting to the AR device. 

Data-Delivery Platform

Datasets required for this application would be relatively small depending on the complexity of 

the project and the area of coverage. The data required could be feasibly transferred between 

platforms using standard wireless connectivity or transferred directly using portable media, such 

as USB drives. If a construction site were set up for wireless communication, then it would be 

feasible to access data directly from the device in the field.

The ideal implementation would be to use existing BIM workflows and tools; AR would then 

become a display extension to the workflow. 

Data-Delivery and -Collection Devices 

This type of simplified abstract design information can be easily understood with a tablet or 

mobile device. The simplicity of using a tablet or smartphone device display is advantageous 

because the information can be shared with multiple users. The display of design information or 

project alternatives might be enhanced and would potentially be more immersive and easier to 

understand with the use of HMD devices, particularly for large 3D elements, such as structures 

or utilities. These elements would benefit from a stereoscopic display format because they would 

be rendered more realistically and true to scale. 
68 

Localization Technology 

Visualization of ROW lines, annotation, and other associated abstract CADD information may 

not require the same localization accuracy that displays of 3D models or design information 

would require. If accuracy within a few inches or feet is sufficient for the activity, then GNSS or 

local network systems could potentially be adequate. One vendor has demonstrated a system that 

would provide accuracy within a few centimeters using GNSS systems combined with inertial 

sensing within the mobile device. When more accuracy is required, a marker-based system with 

targets placed in the environment would most likely be needed. Several examples of 

marker-based applications currently exist on the market. 

Information Delivered Onsite

The primary information delivered to the user will be line work representing potential boundaries 

(and alternatives) for ROW, alignment centerlines and stationing, edge-of-construction lines 

(e.g., paving/grading), and simplified 3D models of design elements. Line work for other design 

information, as well as textual annotation that helps support communication, could also be 

delivered to the user. All 2D information will need to be converted to 3D so that it is displayed at 

the correct elevation that matches the existing conditions viewed through the device. Future AR 

systems will undoubtedly be developed so that they will display associated information, such as 

textual labels and annotation, correctly in the AR view adjacent to the associated 3D elements.

Information Collected Onsite

As planned, this application would communicate ROW effectively without the need to collect 

any information onsite. If alternative boundaries are displayed, then capabilities would exist to 

collect georeferenced notes or annotation as input, as well as to collect photos, general 

comments, or audio recordings. These capabilities would be relatively easy to add with a 

tablet-based application and potentially with some HMD-based systems. As shown in studies 

described in chapter 4 of the document, it is possible to collect 3D point, line, and surface data in 

the field to validate or measure existing conditions. This capability would most likely require an 

HMD-based AR system. The process has only been demonstrated in the field on one type of 

device.

USE CASE 2: VISUAL VARIANCES

Visually tag variances in the field regarding schedule and quality issues to ensure all parties are 

viewing the same issue.

Goals of the Application 

A challenge with inspection is communicating to the contractor and other project stakeholders 

when variances are found either in terms of discrepancies between design versus as-built 

conditions and the planned versus actual schedule of installed components. Variances are 

traditionally communicated through either verbal or written communication, which can lead to 

misunderstandings between different parties and the inspection team. Miscommunication can 

lead to delays in correcting the variance or additional errors in modifying installed items that 

may not be considered a variance. Furthermore, accurately documenting a variance when it is 
69 

observed by an inspector is challenging when documenting the variance’s location and its 

context on a jobsite. 

Use Case

Once an inspection uncovers an issue with an installed, the next step is to communicate the issue 

to all responsible parties. There are instances when inspectors may be able to share the discovery 

of the variance immediately so that affected parties can see the variance firsthand, but there are 

other instances when the variance may need to be viewed later. However, coordinating logistics 

between an inspection team and a contractor for simultaneous viewing can be challenging.

The ability to visually tag a reference through an AR interface so that another party can visually 

inspect the variance through another AR device would improve communication and likely 

expedite the necessary resolution to correct the variance. Furthermore, the ability for an 

individual on a jobsite to share the variance visually with parties in other locations could help 

facilitate the expertise of offsite personnel.

AR Technologies for This Application 

AR technologies’ ability to overlay images that identify objects in variation to a project’s 

contract documents is achievable in current AR systems (figure 30). 

© 2019 WSP.

Figure 30. Screenshot. Mockup of interface for AR visual variance application. 

However, the ability to use an AR interface to tag a variance once it is discovered and then store 

the variance and its georeferenced location so that other parties can then view the same variance 

requires capturing the information (i.e., images or 3D data) onsite and postprocessing of the data 

for future retrieval. An extension of this application could involve integrating the georeferenced 

information of the observed variance with the original design model so that the variance could be 

viewed in the context of the full system design. This approach would offer greater capability of 

identifying possible resolutions even after the inspection was conducted. However, this level of 

integration between an AR view and the design (i.e., BIM) model would require further 
70 

development, but integration could offer a significant improvement in not only resolution of 

variances but also aiding the development of an as-built model. The basic capability of capturing 

3D data onsite to support inspection and measuring variances was demonstrated in work 

performed by the Los Alamos team described in chapter 4. 

AR Technology Framework 

Data Types/Sources

The primary data type that best supports this application is design-based (e.g., CADD) 3D-model 

information, such as 3D models of design elements used in a BIM workflow. Most of these 

elements would be part of a typical 3D design dataset, but they would need to be separated from 

other data and potentially simplified in detail for display in the AR system. The display would 

need to maintain enough detail to adequately measure the variances being checked for at an 

adequate precision. This required precision may vary with different construction components 

(e.g., checking that a metal guardrail is installed based on the design criteria for barrier 

placement, plus or minus some measurement with design tolerances). Supporting information 

about these criteria for the placement of design elements would be useful for the inspection 

activity and the evaluation process. Such supporting information could be graphic or textual and 

preloaded ahead of time or included in a searchable database delivered to the inspector onsite 

and on the device. 

Data-Delivery Platforms 

Datasets required for this application could be relatively large depending on the complexity of 

the project and the area of coverage. Large datasets could be transferred between platforms using 

standard wireless connectivity or transferred directly using portable media, such as USB drives. 

If a construction site were set up for wireless communication, then it would be possible to access 

data directly from the device in the field. This is an application where preconfiguration of models 

ahead of time, similar to the demonstration described earlier by Autodesk and DAQRI using 

BIM360 Docs, would be helpful. 

Several working platforms have been described in this report, particularly those described as part 

of documented workflows earlier in chapter 4. As suggested previously, the ideal implementation 

would be to use existing BIM workflows and tools; AR would then become a display extension 

to the workflow. 

Data-Delivery and -Collection Devices

This type of 3D model information would be easily understood with a tablet or mobile device. 

The simplicity of a tablet or mobile-device display, especially with the ability to share 

information among multiple users, is advantageous. Displaying graphic or textual information is 

more easily achieved with a tablet device or possibly a mobile phone. HMD devices are currently 

limited in resolution and display quality, which is not conducive to reading or viewing detailed 

graphics. The display of design information or project alternatives might be enhanced and would 

potentially be more immersive and easier to understand by using HMD devices, particularly for 

large 3D elements, such as structures or utilities. If small variances need to be seen closely by the 
71 

inspector, this would be displayed more effectively by an HMD. These elements would benefit 

from a stereoscopic display format, which would render them more realistically and true to scale.

Localization Technology 

Visualization of variances with very detailed requirements for precision will require good 

localization and registration between virtual objects and real-world objects. Visualization of 

variances will most likely require a marker-based system with targets placed in the environment 

to initialize the position and orientation of the viewer. A more effective approach is for the AR 

system to match a known point in the virtual model to something in the real world that is known 

to be in the correct location, such as the end of a beam or column location. Several examples of 

marker-based applications have been documented in chapters 3 and 4. 

Information Delivered Onsite

The primary data type delivered to the user will be simplified 3D models of design elements and 

possibly models of existing elements to help with alignment. Additional information, such as text 

or graphics, could be delivered as well, but these would need to be brought up by a localized 

prompt or through a menu system. Future AR systems will be developed so that they will display 

associated information, such as text or graphics, correctly in the AR view adjacent to the 3D 

elements being displayed. 

Information Collected Onsite

As previously described in chapter 4, this type of application would be greatly enhanced by the 

ability to capture existing 3D objects digitally alongside the virtual model, which would not only 

create a virtual record of the inspection but support as-built documentation as well. One 

workflow might be to separate the capture process from the measurement process by bringing the 

onsite captured data back to the office and then measuring variances. This would reduce the need 

for bringing large amounts of information to the field. Studies described earlier in the document 

indicate that it is feasible to collect 3D point, line, and surface data in the field to measure 

as-built conditions. Scanning and capturing onsite 3D information in AR would most likely 

require an HMD-based AR system. Currently, the process has only been demonstrated in the 

field on one type of device, though scanning technology has been incorporated into mobile 

devices. However, there are other existing methods for capturing 3D data onsite, which could 

support virtual inspection after the site visit.

USE CASE 3: NEXTGEN TRAINING AND CERTIFICATION

Use AR technology to support training and certification for construction inspection. 

Goals of the Application 

Experienced personnel are leaving State DOTs through retirement and being replaced by 

less-experienced personnel encountering more rapid increases in responsibility earlier in their 

careers than their predecessors. In some State DOTs, retiring personnel are not being replaced at 

all. These personnel changes are impacting all divisions of DOT personnel, particularly those 

tasked with the construction of highway infrastructure. Research results in NCHRP Synthesis 
72 

450 found that among 40 State DOTs between 2000 and 2010, State-managed lane miles 

increased by an average of 4.10 percent while the number of full-time equivalent staff decreased 

by 9.70 percent (Taylor 2013). Compounding these challenges are recent demographics of STA 

construction staff, which indicated that the most frequent age range of construction staff was 

40–50 years old and that the average years of experience was 10–15. The data indicate that DOT 

construction staff will continue to experience a loss in knowledge and skills due to retirements. A 

challenge is how to efficiently develop the next generation of inspectors during their training and 

through certification. VR and AR offer the potential to explain complex spatial topics effectively 

and supplement traditional training materials.

Use Case

Workforce development involves the recruitment, training, and retention of a workforce 

population. Training is a critical step in the system, and it usually results in an individual 

achieving a public or industry-recognized certification. Inspectors complete a combination of 

formal classroom and on-the-job training (OJT) activities to learn engineering and construction 

fundamentals related to infrastructure systems. While most OJT requires hands-on learning, 

formal classroom training traditionally relies on lecture and 2D educational materials. A 

challenge in engineering and construction education involves understanding interactions of 

complex spatial systems (e.g., how structural systems forces function or how different 

components are assembled in construction). Future applications of AR technology could be used 

to provide 3D representations of technical concepts to facilitate the users understanding of the 

training material. A basic application approach could include AR applications designed to 

recognize 2D markers in textbooks or other types of 2D materials that would then provide a 3D 

representation of the concept.

AR Technologies for This Application

AR for education and training has been used throughout other industries, including healthcare, 

manufacturing, and aviation. Specific devices to support this application are numerous, including 

both handheld (figure 31) and HMD. There are minimal technical challenges to develop these 

applications because current off-the-shelf systems already exist. The more significant challenge, 

however, is to identify which training materials would benefit from 3D representations. The 

application could be an in-office or classroom tool, but the application could be brought to the 

field for onsite comparisons of real-world scenarios. An obstacle when developing this type of 

application is the temporary aspects of construction: construction sites change constantly, 

construction is eventually completed, and the AR content would not be current or accessible for 

long. 
73 

© 2019 WSP.

Figure 31. Photo. Mockup of interface for AR training application. 

AR Technology Framework 

Data Types/Sources

The primary data types that best support this application are design-based (e.g., CADD) 

3D-model information, such as 3D models of design elements used in a BIM workflow. Most of 

these design elements are part of a typical 3D design dataset, but they need to be separated from 

other data and potentially simplified in detail for display in the AR system. As these data are

prepared in advance based on the support of training materials, the models are optimized and 

segmented accordingly. Supporting information for the training exercises includes graphic or 

textual information preloaded ahead of time or included in a searchable database delivered to the 

training as needed on the device. 

Data-Delivery Platform

Datasets of the type required for this application would be relatively small, depending on the 

complexity of the subject matter and the area of coverage of the visualization. If the training is 

performed indoors and not onsite, it is likely that there would be access to standard wireless 

connectivity or the data could be transferred directly using portable media, such as USB drives. 

The information would be preloaded or delivered via browser connection in a similar fashion to

other digital educational or training materials.

Data-Delivery and -Collection Devices

The type of 3D model information used for this application can be easily understood with a tablet 

or mobile device. The simplicity of a tablet or mobile-device display, especially with the ability 

to share the information to multiple users, is advantageous. The display of graphic or textual 

information would be much more easily done with a tablet device or possibly a mobile phone. 

HMD devices are currently limited in display quality, which is not as conducive to reading or 

detailed graphics. The display of design information or project alternatives might be enhanced 
74 

and would potentially be more immersive and easier to understand by using HMD devices, 

particularly for large 3D elements, such as structures or utilities. If small variances need to be 

seen closely by the inspector, the variances would be displayed more effectively by an HMD. 

These 3D elements would benefit from a stereoscopic display format, which would render them 

more realistic and true to scale. 

Localization Technology 

If this type of application were used in an office or classroom, then there would be no need for 

registering a virtual model in a specific real-world location. The 3D representation would still be 

enhanced with accurate tracking in whatever environment the user is in. Markers would still be 

used to load the desired models and supporting materials. If an application were developed that 

included field demonstrations, then GNSS or marker-based initialization would be required to 

place the viewer properly in the scene.

Information Delivered Onsite

The primary information delivered to the user will be simplified 3D models of design elements, 

and possibly models of existing elements to help with understanding context. Potentially other 

information such as text or graphics would be delivered as well, and these would need to be 

brought up by some form of localized prompt or through a menu system. Future AR systems will 

be developed so that they will display associated information, such as text and annotation, 

correctly in the AR view adjacent to the 3D elements.

Information Collected Onsite

This application would be primarily developed to deliver information but also as part of a review 

of the trainee. When used for reviewing a trainee, the application could query the user for 

information or answers to questions. The application could potentially prompt the user to interact 

with the virtual model, asking for 3D input in the virtual scene. This functionality seems similar 

in technology requirements to the applications previously described in chapter 4 in which users 

enter 3D data in the field through an HMD device.

USE CASE 4: INSPECTOR’S TOOLKIT

Use AR technologies to verify that work is being installed properly by providing the correct 

information in the correct format for the inspector in the field. 

Goals of the Application 

Construction inspection requires processing textural and graphical data on behalf of an inspector 

to verify proper installation. Traditionally, inspectors have relied on a combination of training, 

cognition, and experience to make judgments to approve construction work. As projects have 

become more complex and the shortage of inspectors has increased, inspection of transportation 

construction components has become more challenging. The use of AR has the potential to assist 

new and experienced inspectors and to examine complex systems by offering the right 

information in the right format at the right time.
75 

Use Case

There are many aspects to inspection, ranging from measuring quantities for payment to 

providing the QA and/or quality control (QC) to accept work that meets contractual 

requirements. This AR application would focus on assisting inspectors in providing the QA/QC 

support. When inspecting a project, information must be obtained from plans, specifications, 

manuals, and policies. Currently, much of this documentation resides in electronic and hardcopy 

resources; ensuring an inspector is using the most current versions of these documents in the 

field can be challenging. If this information were available electronically and in a central 

repository, then some of the challenges would be alleviated and important information would be 

more easily accessible to inspectors in the field.

AR in the field could take many forms for an inspector. The required information could be 

accessed on demand through a series of menus and tabs, thus providing a similar interface to 

what tablets display now. In this case, the advantage of HMD-delivered AR over tablet-delivered 

AR is hands-free operations. The HMD device could potentially allow inspectors to measure 

field conditions for QA/QC. Part of the challenge of field inspection is knowing which critical 

items need to be examined and when. Future applications of AR that are georeferenced and 

linked to schedule milestones could intelligently tailor the information displayed to the user by 

providing relevant inspection parameters (figure 32). 

© 2019 WSP.

Figure 32. Screenshot. Mockup of interface for the inspector’s toolkit application.

AR Technologies for This Application 

The ability of AR technologies to provide information in graphical and textural detail to facilitate 

QA/QC inspection is practical with current AR applications. AR systems operate through a 

menu-driven environment that could be navigated by a user to access the desired information. 

The ability of AR systems to tailor information to geolocation has been used in prior commercial 

applications. However, the added synchronization with schedule information to further provide 

time-sensitive information is a new advancement. Developing this level of synchronization is 
76 

possible at the project level, although efficiently developing this at the enterprise level would 

likely require further efforts.

There are minimal technical challenges to develop these applications because current 

off-the-shelf systems already exist. The more significant challenge, however, is identifying 

which training materials would benefit from 3D images. 

AR Technology Framework 

Data Types/Sources

The primary data types that would best support this application would be design-based 

(e.g., CADD) 3D-model information such as 3D models of design elements used in a BIM 

workflow and textual design data (e.g., relevant specifications or design notes). Most of these 

elements would be a part of a typical 3D-design dataset; they would just need to be separated 

from other data and potentially simplified in detail for display in the AR system. Enough detail 

would need to be maintained to adequately display to the inspector what an inspector should be 

seeing in the field. This 3D and textual information could be relatively simple and abstract for 

some scenarios and very detailed for more complex construction scenarios. Supporting graphic 

or textual information about these design or installation elements and associated inspection 

criteria would be useful for the inspection activity and the evaluation process. The graphic or 

textual information could be preloaded or included in a searchable database delivered to the 

inspector onsite and on the device. 

Data-Delivery Platform 

Datasets of the type required for this application could be relatively large, depending on the 

complexity of the project, the area of coverage, and the amount of supporting graphic and textual 

information. This type of data could be feasibly transferred between platforms using standard 

wireless connectivity or transferred directly using portable media, such as USB drives. If a 

construction site were set up for wireless communication, then it would be possible to access 

data directly from the device in the field. This is an application where organization of models 

ahead of time, similar to the demonstration described earlier by Autodesk and DAQRI using 

BIM360 Docs, would be helpful. 

Several BIM platforms have been described in this report, particularly those described as part of 

documented workflows earlier in chapter 4. As suggested before, the ideal implementation would 

be to use existing BIM workflows and tools, with AR just becoming a display extension to the 

workflow. 

Data-Delivery and -Collection Devices

The type of 3D model and textual information required for this application can be easily 

understood with a tablet or mobile device. The simplicity of a tablet or mobile-device display, 

especially with the ability to share information among multiple users, is advantageous. 

Displaying graphic or textual information is more easily achieved with a tablet device or possibly 

a mobile phone. HMD devices are currently limited in resolution and display quality, which is 

not conducive to reading or viewing detailed graphics. The display of design information or 
77 

project alternatives might be enhanced and would potentially be more immersive and easier to 

understand by using HMD devices, particularly for large 3D elements, such as structures or 

utilities. If small variances need to be seen closely by the inspector, the variances would be 

displayed more effectively by an HMD. These variances would benefit from a stereoscopic 

display format, which would render them more realistic and truer to scale. An intelligent 

application that would deliver the right information at the right time based on context would be 

complex and require significant development. A less complex application is a menu-driven 

system that prompts the user for information.

Localization Technology 

Visualization of design components aligned accurately in the field would require good 

localization and registration between virtual objects and real-world objects. Visualization of 

design components will most likely require a marker-based system with targets placed in the 

environment to initialize the position and orientation of the viewer. A more effective approach is 

for the AR system to match a known point in the virtual model to something in the real world

that is known to be in the correct location, such as the end of a beam or column location. 

Displaying graphic or textual information, and possibly models of example design components, 

would not need as precise an alignment to the real-world view and would need to be accessible 

to the user at the requested or appropriate time. GNSS or marker-based localization would be 

sufficient for achieving this precision of alignment. 

Another potential requirement for an inspection application system is the integration with the 

construction schedule because the information an inspector needs not only varies by location but 

also by phases of construction. Different markers produced for specific components or timebased activities (e.g., steel beams, or an install sequence for the beams) could address this 

challenge. This requirement could also be addressed in the interface of the application. 

Information Delivered Onsite

Information delivered to the onsite user will include simplified 3D models of design elements 

and possibly models of existing elements to help with alignment. Additional information, such as 

text or graphics, could be delivered as well; this information would need to be brought up by a 

localized prompt or through a menu system. Future AR systems will be developed so that they 

will display associated information, such as text or graphics, correctly in the AR view along with 

the 3D elements being displayed. 

Information Collected Onsite

This application would be primarily developed to deliver information. Potentially as part of 

the inspection activity, however, the application could query the user for information 

(e.g., completion of an electronic checklist or comments on the subject being inspected). The 

application could potentially prompt the user to interact with the virtual model, asking for 3D 

input in the virtual scene. This functionality would require technology similar to that in 

applications previously described, in which users are entering 3D points while viewing 

real-world objects. 
78 

USE CASE 5: AUTOMATED INSPECTION

Use AR technology to support automated code/standard-compliance checking of installed items 

through machine learning. 

Goals of the Application 

Computer algorithms used to verify that design objects comply with industry and local building 

codes have been the subject of prolonged research and development efforts. The conventional 

practice of code-compliance checking in the construction industry has primarily been a manual 

process, which is costly, demanding of experienced personnel, and prone to error. A common 

approach to check for compliance automatically is to compare each object or system in a BIM to 

the constraints in a standard. The evolution of BIM and industry classes supports this 

development. The output is usually a list of noncompliant objects. Under this current approach, 

automated code-compliance checking is usually focused as a design effort rather than a 

construction inspection task, which would further ensure that not only are objects designed in 

accordance to building codes but they are also installed in compliance with those same codes. 

Use Case

There are numerous sections of Federal and State standard specifications that apply to 

transportation construction. These standards specify several spatial aspects related to design 

(e.g., bar spacing, member sizes, and object orientation). Verifying that all installed objects 

comply with applicable standard specifications can be challenging. If integrated with sensing 

data (e.g., laser scanning, or in the future, scanning by the onboard sensors), then AR technology 

could highlight installed components that are outside of code compliance. For example, in

transportation construction, AR technology could potentially highlight visually in the user’s view 

if the proper size, shape, and length of reinforcing steel has been installed at the correct spacing 

(figure 33). While current capabilities of AR systems cannot sense and measure the installation 

of components to the level of accuracy to determine whether components have been correctly 

installed in compliance to standards, other technologies, such as LiDAR and laser-scanning 

systems, can. These data could be collected and analyzed offline. The result of the analyses could 

be loaded to the AR system to help visually tag the components that are out of compliance.
79 

© 2019 WSP.

Figure 33. Screenshot. Mockup of interface for automated inspection application.

AR Technologies for This Application

Using 3D imaging technologies for automated code compliance has been developed to inspect 

installed highway components ranging from the flatness of concrete slabs, slopes, rebar spacing, 

and rebar diameter. However, integrating the results of these scans in AR is a novel concept. AR 

systems have been explored as tools to scan and perform the analyses internally while onsite, but 

the accuracy of scanners within current HMD devices was found to lack the accuracy to support 

the function adequately. One scenario for this application is that the inspector would draw from a 

library of components to compare against the installation. The comparison could be achieved 

manually and visually by the inspector, or by implementing artificial intelligence (AI), the 

device’s processor would be used to make the evaluation using the library component. In the 

future, with enough development and sophisticated AI, the device and processor could perform 

the compliance check automatically with some minimal input and direction by the inspector. 

AR Technology Framework 

Data Types/Sources

The primary data types that best support this application are design-based (e.g., CADD) 3D 

model information, such as 3D models of design elements used in a BIM workflow. Most of 

these elements would be part of a typical 3D design dataset; the elements would need to be 

separated from other data and potentially simplified in detail for display in the AR system. 

Virtual objects would need to have enough detail to adequately compare visual elements to 

real-world elements and with adequate precision. This required precision will vary with different 
80 

construction components. To be truly automated, the application would need to draw upon 

design criteria that would be applied to 3D information captured by the device in the field. This 

would be a truly challenging application to develop; it would require significant intelligence by 

the system. The design criteria could be a 3D library of design components or a series of 

abstracted dimensions and measurements.

Data-Delivery Platform

3D datasets required for this application could be relatively large depending on the complexity of 

the project and the area of coverage. These datasets could be transferred between platforms using 

standard wireless connectivity or transferred directly using portable media, such as USB drives. 

If a construction site were set up for wireless communication, then data could be accessed

directly from the device in the field. If the application were automated as previously described, 

then the platform would need to support the delivery of the library of information required to 

conduct the compliance analyses. In the future, this type of complex system would most likely 

need wireless access to process and support information in the cloud. 

Data-Delivery and -Collection Devices

This type of 3D model and textual information can be easily understood with a tablet or mobile 

device. The simplicity of a tablet or smartphone display, especially with the ability to share 

information among multiple users, is advantageous. Displaying graphic or textual information is 

more easily achieved with a tablet device or possibly a mobile phone. HMD devices are currently 

limited in resolution and display quality, which is not conducive to reading or viewing detailed 

graphics. The display of design information or project alternatives might be enhanced and would 

potentially be more immersive and easier to understand with the use of HMD devices, 

particularly for large 3D elements, such as structures or utilities. Currently, HMDs are the only 

AR devices that have been demonstrated to collect 3D information on a construction site, but the 

same scanning technology has been deployed on mobile devices. 

Localization Technology 

Comparison of virtual models with installed components would most likely require high 

precision and thus good localization and registration between virtual objects and the real-world 

objects. This type of application will most likely require a marker-based system with targets 

placed in the environment to initialize the position and orientation of the viewer. A more 

effective approach is for the AR system to match a known point in the virtual model to 

something in the real world that is known to be in the correct location, such as the end of a beam 

or column location. Several examples of marker-based applications have been described in this 

report in chapter 4. 

Information Delivered Onsite

As described above, one scenario for this application is that the user would draw from a library 

of 3D models of design elements to facilitate the evaluation of installed components. Other 

information, such as measurements, dimensions, and text or graphics, could be delivered as well; 

these would need to be displayed by a localized prompt or through a menu display. Future AR 
81 

systems will be developed so that they will display 3D components or associated information, 

such as text or graphics, correctly in the AR view adjacent to the 3D elements being displayed. 

Information Collected Onsite

As previously described, this type of application would be greatly enhanced by the ability to 

capture existing 3D objects digitally alongside the virtual model being displayed. This would not 

only create a virtual record of the comparison of objects but would support as-built 

documentation as well. One workflow that would accomplish this would be to separate the 

capture process from the compliance-check process by bringing the onsite captured data back to 

the office and then checking for compliance. This would reduce the need for bringing large 

amounts of information to the field. Studies described earlier in the document indicate that it is 

feasible to collect 3D point, line, and surface data in the field to measure as-built conditions. 

Scanning and capturing onsite 3D information with AR technologies would most likely require 

an HMD-based AR system. The process currently has only been demonstrated in the field on one 

type of device, though scanning technology has been incorporated into mobile devices as well. 

However, there are other existing methods for capturing 3D data onsite, which could support 

virtual inspection after the site visit.

83 

CHAPTER 7. SUMMARY OF KEY OBSERVATIONS AND CONCLUSIONS

This study focused on documenting current AR tools and applications and their implementation 

for supporting highway construction—especially for supporting construction inspection and 

review, QA, training, and improved project management through access to real-time information 

in a real-world environment. While the researchers were unable to identify mature applications 

currently ready to implement for construction, the researchers found several activities by 

academic researchers, agencies, and private industry that explore the opportunities and benefits 

that AR solutions can add to construction management. 

KEY OBSERVATIONS

The market review of AR technology identified several vendors offering AR solutions with 

products of varying sophistication and characteristics. AR technology is rapidly changing and 

improving in terms of hardware, applications, and workflows. Throughout this research project, 

several new AR technologies were released and existing technologies and devices have advanced 

and improved dramatically. Conversely, several devices documented early in the study have 

disappeared from the market, which is an indication of the industry’s instability and rapidly 

changing nature. A small portion of the devices found in the study focused on the AEC market, 

especially for interior architectural design and construction. The HMD devices with success in 

this market were untethered (mobile) and robustly supported standard 3D modeling-platforms. 

Tablet support for AR in the AEC market is typically based on software support of the onboard 

display and tracking technologies found in common mobile devices. 

Most of the current AR devices have very good tracking capabilities once registered to the 

real-world environment. Most devices and applications’ tracking capabilities use markers 

(or target images) in known locations in the environment. Once localized, the devices track the 

user’s movements and view with a combination of video imagery, 3D sensors, and inertial 

tracking. As the devices move within the environment, they maintain the registration of virtual to 

real-world imagery. With some devices and applications, 3D virtual–model elements are 

matched directly to their real-world counterparts to increase the accuracy of image registration 

and tracking. 

This study identified several challenges with AR visualization in outdoor and unstructured open 

area environments. For example, see-through devices such as HMDs have difficulty displaying 

virtual imagery over bright real-world scenes. The tracking sensors on AR devices require 

adequate detail to track the real world efficiently, which highway construction sites may lack as 

they are typically open, flat expanses. Other challenges, especially with HMD devices, include 

visual display performance, durability in a construction environment, onboard digital storage 

capacity, and access to onsite wireless communication. Using HMDs or tablet devices safely is in 

an ongoing concern. These challenges will affect when these devices can be leveraged for 

highway construction. 

Tablet devices appear to be making more rapid advances in the highway construction field 

because they have fewer limitations in outdoor display and viewing. The disadvantage of using 

tablets in the highway construction field centers on the accuracy of the tracking technology; the 
84 

registration of virtual to real-world imagery might not be precise. One vendor has focused on this 

market with a unique combination of a tablet device integrated with special hardware and 

software technology to provide higher precision tracking and better registration of virtual 3D 

data to the real-world view. 

A unique application of AR HMD devices that the research identified was the ability to leverage 

the 3D-scanning hardware used for AR tracking to capture existing 3D data in the field. The 

precision of the data depends on several factors, including available site survey information, and 

the scanning precision of the sensor hardware. On a typical highway construction site with good 

survey targets and a robust 3D model-based workflow, this capability will become an important 

aspect of AR implementation, especially for site inspections.

The commercially available devices identified in the research for AEC application were 

generally supported by 3D software applications used in design and construction. Workflows 

that support AR devices for this industry will be similar to 3D model-based workflows and BIM 

processes that are rapidly becoming more prevalent. FHWA promotes these 3D workflows 

through the EDC program. Challenges with data flow and 3D-model management for AR 

technologies will be similar to those challenges currently being addressed in BIM workflows. It 

is likely that the integration of AR devices into an existing BIM workflow will soon promote the 

early adoption of these AR devices. 

Virtually all the 3D modeling and CADD application vendors supporting the AEC industry are 

developing application workflows that support deploying 3D models to AR devices, and most 

are also developing cloud-based collaboration and model distribution platforms to facilitate 

managing 3D-model data and deploying it to construction sites and mobile AR devices. As 

communication technology continues to promote Wi-Fi access through cellular or satellite 

communications to remote parts of the country, it will also promote onsite deployment of 

BIM-based AR applications. 

CONCLUSIONS

The workshop findings, prototype case studies, and discussions on current technologies, 

opportunities, and challenges all point to potential future use of AR technology in highway 

construction. The individual AR frameworks described for each case study are currently 

available in the market or have been demonstrated in research or exploratory activities. Research 

activities have virtually demonstrated case study 1 identified earlier in this report. Activities 

described in all the case studies can be achieved at some level in the near future, but the activities 

require investment in specific hardware and software frameworks. The most important 

requirements include software development for managing the application (e.g., for the UI, data 

input/output, 3D-model management) and coordinating the AR hardware as an integrated 

working solution. The application and hardware should be seamless for users, allowing them to 

focus on the tasks that need to be accomplished. 

This study identified five use cases of AR technologies in transportation construction. These case 

studies are not the only potential applications of AR technologies in transportation construction, 

but the case studies can serve as starting points for transportation agencies and developers to 

focus AR development and implementation efforts in the near and long term. One challenge 
85 

raised by participants in both AR workshops is the lack of ROI to justify AR implementation. 

The lack of ROI is a traditional barrier to new technologies, but in this case, independent field 

trials have helped address the concern. These field trials have also helped further the 

development and implementation of technologies, such as 3D modeling and BIM. Multiple 

objective field trials that examine the benefits and costs of implementing AR technology in the 

highway construction industry based on empirical results will further alleviate the ROI 

challenge.

Just as construction is driving changes in technologies, such as survey, automated machine 

guidance, inspections, and QA using unmanned aerial systems, it will likely also drive the 

advancement of AR technologies. The EDC program promotes innovation (e.g., 3D-model-based 

workflows) and this innovation will provide opportunities for using AR technologies on highway 

construction projects.

87 

REFERENCES

Abboud, R. (2014). Architecture in an Age of Augmented Reality: Opportunities and Obstacles 

for Mobile AR in Design, Construction, and Post-Completion. The National Association 

of Women in Construction. New South Wales, Australia. Accessed May 22, 2020. 

https://www.academia.edu/14677741/Architecture_in_an_Age_of_Augmented_Reality_

Opportunities_and_Obstacles_for_Mobile_AR_in_Design_Construction_and_PostCompletion. 

Adam, J., Cawley, B. Petros, K., Brautigam, D., Burns, R., Burns, S., Kliewer, J., Lobbestael, J., 

Park, R. and Jahren, C. (2015). NCHRP Project 20-68A, Scan 13-02, Advances in Civil 

Integrated Management (CIM). Requested by American Association of State Highway 

and Transportation Officials. Washington, DC. Accessed June 4, 2019. 

http://onlinepubs.trb.org/onlinepubs/nchrp/docs/NCHRP20-68A_13-02.pdf. 

Allen, B., Hanley, T., Rokers, B, and Green, S.C. (2016). “Visual 3D motion acuity predicts 

discomfort in 3D stereoscopic environments.” Entertainment Computing 13: 1–9. 

Augment 2018. “The platform for 3D and augmented reality product visualization.” Augment. 

Accessed June 4, 2019. http://www.augment.com. 

Aukstakalnis, S. (2017). Practical Augmented Reality: A Guide to the Technologies, 

Applications, and Human Factors for AR and VR. Boston, MA: Addison-Wesley.

Autodesk. (2109). “BIM 360 Construction Management Software.” (website) Autodesk. 

Accessed June 4, 2019. https://www.autodesk.com/bim-360/?utm_source=adsk-team-eos. 

Azuma, R. (1997). “A Survey of Augmented Reality.” Presence: Teleoperators and Virtual 

Environments 6, no. 4: 355–385. 

Azuma, R. (1999). “The Challenge of Making Augmented Reality Work Outdoors.” In Mixed 

Reality: Merging Real and Virtual Worlds, edited by Yulchi Ohta and Hideyuki Tamura, 

379–390. New York: Springer-Verlag.

Bentley Systems. (2017). “Bentley Advances the Power, Accessibility, and Pervasiveness of 

Visualization and Virtual Reality for Infrastructure Projects.” (website) Bentley Systems. 

Accessed June 4, 2019. https://www.bentley.com/en/about-us/news/2017/october/09/pna06-microstation-lumenrt. 

Bosche, F. and Haas, C. (2008). “Automated 3D Data Collection (A3DDC) for 3D Building 

Information Modeling.” 2008 Proceedings of the 25th ISARC, Vilnius, Lithuania, 

279–285. 

Buker, T., Vincenzi, D., and Deaton, J.E. (2012). “The Effect of Apparent Latency on Simulator 

Sickness While Using a See-Through Helmet-Mounted Display: Reducing Apparent 

Latency with Predictive Compensation.” Human Factors, 54(2): 235–249. 
88 

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damini, E., and Ivkovic, M. (2011). 

“Augmented Reality Technologies, Systems, and Applications.” Multimedia Tools and 

Applications, 51(1): 341–377. 

Chen, P. (2017). Autodesk University 2017 Course FDC125400: “Visualization of Design Using 

Augmented Reality and Smart Wearable Devices.” (website) Autodesk. Accessed 

February 12, 2018. http://au.autodesk.com/au-online/classes-on-demand/classcatalog/classes/year-2017/bim-docs/fdc125400#chapter=0. 

Chen, P., Moffett, J., Negrete, J. and Tyner, D. (2018). Autodesk University 2018 Course 

CS226785: “Mixing Realities: A McCarthy, Autodesk, DAQRI Partnership.” (website)

Autodesk. Accessed June 4, 2019. https://www.autodesk.com/autodeskuniversity/class/Mixing-Realities-McCarthy-Autodesk-DAQRI-Partnership-2018. 

Collin, J., Davidson, P., Kirkko-Jaakkola, M., and Leppäkoski, H. (2018). “Inertial Sensors and 

their Applications.” In Handbook of Signal Processing Systems, edited by S. 

Bhattacharyya, E. Deprettere, R. Leupers, and J. Takala, 66–96. Cham, Switzerland: 

Springer. 

DAQRI. (2019). “DAQRI Wearable Service Line.” (website) DAQRI. Accessed June 4, 2019. 

https://daqri.com/wearable-service-line. 

Developer Apple. (2019). “ARKit 3.” (website) Apple Inc. Accessed June 4, 2019. 

https://developer.apple.com/arkit. 

Developer DART. (2018). “DART.” (website) Georgia Tech. Accessed June 4, 2019. 

http://ael.gatech.edu/dart/aboutdart.htm. 

Developer Google. (2019). “ARCore Overview.” (website) Google Developers. Google.

Accessed June 4, 2019. https://developers.google.com/ar/discover. 

Developer Microsoft. (2019). “Microsoft Mixed Reality.” (website) Microsoft. Accessed June 4, 

2019. https://developer.microsoft.com/en-us/windows/mixed-reality. 

DeWalt. (2017). “DeWalt Launches Jobsite Wi-Fi Access Points: Built for Construction.” 

(website) PR Newswire. Accessed June 4, 2019. https://www.prnewswire.com/newsreleases/dewalt-launches-jobsite-wifi-access-points-300564683.html. 

Domdouzis, K., Anumba, C., and Thorpe, A. (2004). “Wireless Sensor Networking in the 

Construction Industry - Prospects and Problems.” 20th Annual ARCOM Conference, 1–3 

September 2004, Heriot-Watt University. Association of Researchers in Construction 

Management, 2: 1107–1120. 

Epson. (2019). “Moverio Augmented Reality Smart Glasses.” (website) Epson. Accessed June 4, 

2019. https://epson.com/moverio-augmented-reality. 

FHWA. (2019). “On-ramp to Innovation: Every Day Counts.” (website) FHWA. Accessed June 

4, 2019. https://www.fhwa.dot.gov/innovation/everydaycounts. 
89 

Google Glass 2019. (2019). “Discover Glass Enterprise Edition” (website) Google. Accessed 

June 4, 2019. https://www.google.com/glass/start. 

Hecht, J. (2016). "Optical Dreams, Virtual Reality." Optics and Photonics News, 27(6): 24–31. 

Holloway, R. (1997) “Registration Error Analysis for Augmented Reality.” Presence: 

Teleoperators & Virtual Environments, 6, MIT Press: 413–432. 

Howard, I., and Rogers, B. (1995). Binocular Vision and Stereopsis. New York: Oxford 

University Press.

Iliyn, D. (2017). Autodesk 2017 Course CI123968: “The Augmented and Virtual Reality 

Brought to Life Through InfraWorks.” (website) Autodesk. Accessed June 4, 2019. 

http://au.autodesk.com/au-online/classes-on-demand/class-catalog/classes/year2017/infraworks/ci123968#chapter=0. 

Intel. (2018). “Intel RealSense Technology, Observe the World in 3D.” (website) Intel. Accessed 

June 4, 2019. https://www.intel.com/content/www/us/en/architecture-andtechnology/realsense-overview.html. 

Ishii, H. (2010). “Augmented Reality: Fundamentals and Nuclear-Related Applications.” 

International Journal of Nuclear Safety and Simulation. 1(4): 1–12. 

Jahangiri, B., Keane, D. and Tang, W. (2017). “Augmented Reality: Practical Applications for 

Real World Problems.” (website) Autodesk. Accessed June 4, 2019. 

https://www.autodesk.com/autodesk-university/class/Augmented-Reality-PracticalApplications-Real-World-Problems-2017. 

Kalloc Studios. (2017). “Fuzor.” (website) Kalloch Studios. Accessed February 26, 2018. 

https://www.kalloctech.com. 

Lang, P., Kusej, A., Pinz, A., and Brasseur, G. (2002). “Inertial Tracking for Mobile Augmented 

Reality.” Conference Record - IEEE Instrumentation and Measurement Technology 

Conference, 1583–1587. 

Leap Motion Blog. (2019). “Reach into the Future of Virtual and Augmented Reality.” (website)

Leap Motion, Inc. Accessed June 4, 2019. https://www.leapmotion.com. 

Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., Furgale, P. (2014). “Keyframe-based Visual 

Inertial Odometry using Nonlinear Optimization,” International Journal of Robotics 

Research, 34(3): 314–334. 

MacIntyre, B., Gandy, M., Dow, S., and Bolter, J. (2004). “DART: A Toolkit for Rapid Design 

Exploration of Augmented Reality Experiences.” Proceedings of the 17th Annual ACM 

Symposium on User Interface Software and Technology, 197–206. 

Magic Leap, Inc. (2019). “Encounter the Next Era of Computing.” (website) Magic Leap, Inc., 

Accessed June 4, 2019. https://www.magicleap.com. 
90 

Manurl, F. and Sanna, A. (2016). “A Survey on Applications of Augmented Reality.” Advances 

in Computer Science: An International Journal, 5(1) 19: 18–27. 

Meehan, M., Razzaque, S., Whitton, M., and Brooks, F., Jr. (2003). “Effect of Latency on 

Presences in Stressful Virtual Environments.” Proceedings of the IEEE Virtual Reality, 

141–150. 

Mekni, M. and Lemieux, A. (2014). “Augmented Reality: Applications, Challenges and Future 

Trends.” Applied Computational Science, 205–214. 

Microsoft. (2019). “HoloLens 2 – A New Vision for Computing.” (website) Microsoft. Accessed 

June 4, 2019. https://www.microsoft.com/en-us/hololens/hardware. 

Milgram, P. and Kishino, F. (1994). “A Taxonomy of Mixed Reality Visual Displays.” ICICE 

Transactions on Information and Systems, Vol. E-77-D, no. 12: 1321–1329. 

Moreu, F., Injante, M., Maharjan, D., Shelton, P., Glisic, G., and Mascarenas, D. (2018). TRB 

Straight to Recording for All: Augmented Reality for Structural Inspections. (website)

Transportation Research Board. Accessed March 19, 2019. 

http://www.trb.org/ElectronicSessions/Blurbs/178486.aspx. 

Mykola, V. and Gleb, B. (2017). “Best Tools for Building Augmented Reality Mobile Apps.” 

(website) Ruby Garage. Accessed June 4, 2019. https://rubygarage.org/blog/best-toolsfor-building-augmented-reality-mobile-apps. 

National Geodetic Survey. (2019). “Continuously Operating Reference Station (CORS).” 

(website) NGS. Accessed June 4, 2019. https://www.ngs.noaa.gov. 

NVIS. (2020). “NVIS Products.” (website) NVIS. Accessed April 1, 2020. 

https://www.nvisinc.com/product.html. 

O’Brien, W., Sankaran, B., Leite, F., Khwaja, N., De Sande Palma, I., Goodrum, P., 

Molenaar, K., Nevett, G. and Johnson, J. (2016). Civil Integrated Management (CIM) for 

Departments of Transportation, Volume 1: Guidebook. National Cooperative Highway 

Research Program Report 831. (website) The National Academies of Science, 

Engineering, and Medicine. The National Academies Press. Accessed June 4, 2019. 

http://nap.edu/23697. 

Papadakis, G., Mania, K. and Koutroulis, E. (2011). “A System to Measure, Control, and 

Minimize End-to-End Head Tracking Latency in Immersive Simulations.” Proceedings 

of the 10th International Conference on Virtual Reality Continuum and Its Applications 

in Industry, 581–584. 

Papagiannakis, G., Singh, G., and Magnenat-Thalmann, N. (2008). “A Survey of Mobile and 

Wireless Technologies for Augmented Reality Systems.” Computer Animation and 

Virtual Worlds, 19(1) 3–22, John Wiley & Sons, Inc. Chichester, UK. 
91 

Pesyna, K. Jr., Heath, R., Jr., and Humphreys, T. (2014). “Centimeter Positioning with a 

Smartphone-Quality GNSS Antenna,” Conference Paper. ION GNSS+. The University of 

Texas at Austin.

Rabbi, I. and Ullah, S. (2016). “A Survey on Augmented Reality Challenges and Tracking.” 

ACTA Graphica, 24(1–2), 29–46. 

Raskar, R. (2004). Spatial Augmented Reality, Keynote, “Symposium on Virtual Reality” 

(SVR).

Ren, D., Goldschwendt, T., Chang, Y., and Höllerer, T. (2016). “Evaluating Wide-Field-OfView Augmented Reality with Mixed Reality Simulation” IEEE Virtual Reality 

Conference 2016. Greenville, SC.

Roukounaki, K. (2015). “Top 5 Tools for Augmented Reality in Mobile Apps.” (website)

Developer Economics. Accessed June 4, 2019. 

https://www.developereconomics.com/top-5-tools-for-augmented-reality-in-mobile-apps. 

Schall, G., Wagner, D., Reitmayr, G., Taichmann, E., Wieser, M., Schmalstieg, D., and

Hofmann-Wellenhof, B. (2009). “Global Pose Estimation using Multi-Sensor Fusion for 

Outdoor Augmented Reality.” Science and Technology Proceedings - IEEE 2009 8th 

IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2009. 

Orlando, FL.

Schmalstieg, D. and Höllerer, T. (2016). Augmented Reality, Principles and Practice (Usability). 

Boston, MA: Addison-Wesley.

Sharma, P. (2014) “Augmented Reality: Its Applications and Use of Wireless Technologies.” 

International Journal of Information and Computation Technology. 4(3), pp. 231–238

SketchUp (Trimble). (2017). “SketchUp Viewer.” (website) Trimble Systems. Accessed April 1, 

2020. https://www.SketchUp.com/products/SketchUp-viewer. 

Sony. (2019). “Developed customized AR solutions, Specifications.” (website) Sony. Accessed 

June 4, 2019. https://developer.sony.com/develop/smarteyeglass-sed-e1. 

Taylor, R.B.T., and Maloney, W.F. (consultants) (2013). NCHRP Synthesis 450: Forecasting 

Highway Construction Staffing Requirements: A Synthesis of Highway Practice. 

Transportation Research Board: National Academies of Sciences, Engineering, and 

Medicine. Washington, DC: The National Academies Press.

TechViz. (2017). “TechViz AR.” (website) TechViz. Accessed June 4, 2019. 

https://www.techviz.net/techviz-vr-software. 

Trimble. (2018). “Trimble Connect.” (website) Trimble, Inc. Accessed June 4, 2019. 

https://connect.trimble.com. 
92 

Unity Technologies. (2017). “Unity for Mobile AR.” (website) Unity Technologies. Accessed 

June 4, 2019. https://unity3d.com/solutions/mobile-ar. 

Van Krevelen, D.W.F., and Poelman, R. (2010). “A Survey of Augmented Reality Technologies, 

Applications, and Limitations.” The International Journal of Virtual Reality 9(2): 1–20. 

VisionLib Product Portal. (2020). “Augmented Reality Tracking Library for Industries.” 

(website) Visometry GmbH. Accessed March 31, 2020. https://www.visionlib.com. 

Vuforia Developer Portal. (2017). “Innovate with Industrial Augmented Reality.” (website) PTC 

Inc. Accessed June 4, 2019. https://www.vuforia.com. 

VUZIX. (2019). “VUZIX Products.” (website) Vuzix. Accessed June 4, 2019. 

https://www.vuzix.com/Products. 

Waking App. (2017). “WakingApp AR Studio.” (website) Waking App Realities Ltd. Accessed 

June 4, 2019. https://www.wakingapp.com. 

Wang, X. and Dunston, P. (2007). “Design, Strategies, and Issues Towards an Augmented 

Reality-Based Construction Training Platform.” ITcon, 12, 363–380. 

WebVR. (2017). “Bringing Virtual Reality to the Web.” (website) WebVR. Accessed June 4, 

2019. https://webvr.info. 

Welch, G. and Foxlin, E. (2002). “Motion Tracking: No silver bullet, but a respectable arsenal.” 

IEEE Computer Graphics and Applications, 22(6): 24–38. 

Welch, G. (2016). “Highlights of ‘Immersive Sciences’ Research in the U.S.A.: 

Augmented/Virtual Reality and Human Surrogates,” VRS, 21(2).

Wheeler, H. (2016). “Trimble Launches SketchUp Viewer Mixed Reality Solution for Microsoft 

HoloLens.” (website) Engineering.com. Accessed June 4, 2019. 

https://www.engineering.com/DesignSoftware/DesignSoftwareArticles/ArticleID/13757/

Trimble-Launches-SketchUpSketchUp-Viewer-Mixed-Reality-Solution-for-MicrosoftHoloLens.aspx. 

HRDI-20/11-20(WEB)E Recycled

Recyclable