Intelligent Digital Twins and Augmented Reality in Inspection and Maintenance
Authors: Akbari Farid 
Published: December 2021

Abstract

Objective/Scope
Simplify and semi-automate the creation of Intelligent Digital Twins available on an augmented hybrid solution (on/offline) based on the latest mobile solutions and wearable technologies (Smart Glasses) communicating over Bluetooth. The main objective was to implement an interactive environment of digital twins through Augmented Reality (AR) to improve decision-making, enhance performance and optimize efficiencies in Inspection and Maintenance.

Methods, Procedures, Process
A breakthrough algorithm enabled to semi-automate the creation of 3D models by providing a platform of Intelligent Digital Twins to connect to various disparate systems and use Artificial Intelligence-driven augmentation techniques to contextualize and enrich the model with Tag numbers from the asset register. An application has been developed for Smart Glasses by transforming the Digital Twins into sectioned files to enable a performant AR experience and an on/offline syncing functionality with the mobile solution over Bluetooth. Furthermore, the 3D model was embedded within the mobile application for visual support during Inspection and Maintenance.

Results, Observations, Conclusions
As a result, the following products have been developed to improve efficiency and productivity by leveraging innovative technologies:
Application for Intelligent Digital Twins: Single Source of Truth for managing and maintaining Intelligent Digital Twins using breakthrough technology to semi-automate the creation of 3D models from Point Cloud data. A Mark-Up Tool and a Contextualization Editor were developed on the promise of Digital Twins to introduce a communication, collaboration, and planning tool. Furthermore, the 3D model has been embedded within the mobile application environment for visual support on the field tablets. Application for Smart Glasses (AR Headset): An application has been developed for the latest AR Headset to enable an Augmented Reality experience and a Mark-Up tool with Bluetooth communication to the mobile application as well as to support a hybrid on/offline syncing functionality as well as data capturing on the fly. With its basic functionality of hands-free video streaming and communication functionality, the AR Headset enabled a Remote Pre-Commissioning, Certification, and Audit with vendors and independent certifiers during the pandemic. Application Programming Interfaces (API): API interfaces were established to various source systems like the Computerized Maintenance Management System, Inspection Data Management System, Drawing System, and the Process Information System to integrate and visualize the big data within the platform of Intelligent Digital Twins.

Novel/Additive Information
The use of automation and artificial intelligence (AI) driven technologies for creating Intelligent Digital Twins is a key enabler for global scalability and accelerates creating the company's digital backbone during the digital transformation. Enhancing this technology with an Augmented Reality experience, including mark-up annotations, provides a solid basis for data insights, data-driven decision making, and performance optimization.