# Augmented Reality-Assisted Quality Control Based on Asset Administration Shells for Concrete Elements

Mario Wolf, Jan Luca Siewert, Oliver Vogt, Detlef Gerhard

► To cite this version:

Mario Wolf, Jan Luca Siewert, Oliver Vogt, Detlef Gerhard. Augmented Reality-Assisted Quality Control Based on Asset Administration Shells for Concrete Elements. 19th IFIP International Conference on Product Lifecycle Management (PLM), Jul 2022, Grenoble, France. pp.358-367, 10.1007/978-3-031-25182-5_35. hal-05420936

HAL Id: hal-05420936

https://inria.hal.science/hal-05420936v1

Submitted on 17 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Augmented Reality-Assisted Quality Control
Based on Asset Administration Shells
for Concrete Elements

Mario Wolf✉, Jan Luca Siewert, Oliver Vogt, and Detlef Gerhard

Digital Engineering Chair, Ruhr University Bochum, 44801 Bochum, Germany
mario.wolf@ruhr-uni-bochum.de

**Abstract.** The implementation of the Digital Twin concept in the form of Asset Administration Shell (AAS) enables smart flow production even in fields that currently do not incorporate such methods, such as in the fabrication of precast concrete elements. The authors present an approach to implement architecture, engineering and construction-specific AAS submodels for flow production using Building Information Modeling (BIM) data and further enhance these with a new stepwise, task-based quality control process utilizing smart devices as a human machine interface to incorporate Augmented Reality visualization. The concept briefly describes the general production process for concrete wall elements, in which the detailed process of creating quality control tasks is embedded, as well as the meta-models for those components communicating using AAS. Utilizing both product type-specific and product instance-specific data, knowledge about the process, the product's state in the process, and the physical setup enable a reliable and integrated quality control system with clear data flows, automatic documentation, and improved traceability. Finally, a prototypical implementation shows the integration between BIM-Software, the created AAS templates and submodels, and a prototypical quality control setup with smart device support in the proposed process.

**Keywords:** Augmented reality · Asset Administration Shell · BIM integration

# 1 Introduction

To enable a fully digitized flow production of precast concrete elements, data from the production process and the current state of the precast element being produced must be continuously recorded, compiled, and made available. The data and communication structure for these functionalities were developed in previous work in the form of the widely used representation as so-called Asset Administration Shells (AAS). Based on the continuously recorded data, the current production status of the precast concrete element can be monitored and tracked with ease.

The underlying AAS approach generally enables machine-to-machine communica-
tion and envisaged using the data and communication options in the event of production-
related deviations to check further use elsewhere in the structure or make automatic
corrections in the production process.
Ultimately, agile control of the planning and production of the precast concrete element is made possible by the context-sensitive management shells. For the people in the production environment, the application of the AAS concept results in the possibility of retrieving information on processes, products, machines, and efficiencies ad hoc via suitable end devices. Based on the information collected from product planning and design in Building Information Modeling (BIM) software and the digitally supported production process, individual sub-process can be coordinated more precisely, and people can be supported in their work, while results can be fed back in a process-safe manner.

This paper focuses on the support potential through Augmented Reality (AR) applications using multiple visualization methods and data supply through the developed data representations and interaction mechanisms based on the AAS concept. In particular, the purely smart device-based AR, in which image acquisition, registration, overlay, and display take place entirely on a mobile device, is to be compared with the use of stationary imaging devices, external calculation of the overlay, and a subsequent visualization via a smart device or as a projection in stationary installations.

The problems in the development of AR solutions are manifold and must therefore be considered from different dimensions, which concern both the implementation of AR itself (image acquisition, registration, overlay, display), as well as the implementation alternatives in each individual use case. Likewise, environmental specifics, device protection classes, privacy, and similar concerns in real production environments must be considered.

After an overview of related works, the contribution describes in detail the concept and the specified AAS models used throughout the process as well as the integration into existing BIM workflows. The prototypical implementation is then used to validate the described data models by describing an AAS export from Autodesk Revit, as well as a simplified AR application based on ARKit and a custom registration setup. The work closes with an outlook on the next steps and further work.

# 2 Related Work

## 2.1 Quality Control and Digital Twin Approaches in Civil Engineering

Several approaches have been presented to combine BIM and quality control measures, such as the QC approach to testing the compliance of the final products with quality standards [11]. To support workers in the construction phase, Puri and Turkan suggest additional, textual information to help to interpret the information provided in tolerance specifications [12].

An additional current problem with BIM's concept and implementation in the industry is the focus on the design and construction phases, while lacking guidelines for later life cycle phases and the efficient use of existing data [8], especially in the context of off-site construction. Lee et al. show an approach to combine factory-side quality control with on-site control after transportation [7].

A digital twin is a virtual representation of components or structures that goes beyond a BIM model. According to Boje, Guerriero et al. [4], a digital twin consists of three main components: a physical component, a virtual representation, and a data link. In their literature review, Shahzad et al. [16] investigate concrete requirements for digital
twins in civil engineering. A key finding is the dynamic enrichment and possibility of communication between digital twins. Akanmu et al. [1] explore the use of digital twins in cyber-physical systems and present use cases for improving productivity, health, and safety. They outline key technologies and barriers to implementation. Currently, many ideas for building digital twins and specific solutions exist in the construction industry. However, there is a lack of flexible data models and interfaces, as they already exist in the stationary industry.

This is where the digital twin in form of the AAS concept offers the flexibility to connect to the current state of the art in BIM while enabling flow production, product instance-specific data storage, and quality control, as shown by the authors in [6, 19].

## 2.2 Augmented Reality in Civil Engineering and Quality Control

Today, AR is widely used for various tasks in engineering, including quality control [14]. In such an AR quality control process, the technology is used to give workers visual guidance throughout the task. This is regularly attributed to lowering the time required for such tasks [2] and ensuring the quality of the required process and lowering the number of mistakes made by workers [15]. Complex tasks can be broken down into smaller, step-by-step instructions that closely guide a worker with visual aid. For such tasks, manual tools to create and manage the associated data exist, making content creation easy-to-use and straightforward [17]. For mass production applications, however, it is necessary to integrate the AR systems into the existing IT infrastructure and existing data models [9, 12].

The fundamental idea behind AR is to overlay virtual information over the real world [3]. Milgram et al. [10] differentiate between optical-see-through systems, where virtual content is shown on a half-transparent display between the users' eyes and the real world, and video-see-through systems, where both the virtual information and an image of the real world are observed on a screen, and systems where the virtual information is directly projected onto the real world. Additionally, the information can be observed through head-mounted displays, on mobile devices like smartphones and tablets, or through stationary displays, where a static camera observes the environment.

For this approach, the authors additionally differentiate between mobile AR, which is shown on a device the operator moves around, and stationary AR, where a fixed monitor displays an overlay on top of the image of a static camera [18]. Such fixed setups are often used in quality control settings, where workstands are often static, easing tracking and other AR challenges, as shown in [2].

AR can be used during the design, construction, and operation phases. Common areas include design assessment, progress monitoring, defect detection, quality management, assembly instructions, and maintenance [5]. [13] show an example of quality control based on BIM models. Here, the model is displayed on a smartphone, allowing the user to move around the building, view information on specific parts of the building, reviewing schedules, and adding notes to BIM objects. However, no registration between the building and the model takes place.
## 3 Concept

At the end of the design phase, a state-of-the-art BIM model is rich in information about the overall building structure, and each individually planned concrete element. To reach the aim of a fully digitized production of precast concrete elements, each element, and machine in the factory should have its digital twin, describing all relevant aspects of the physical asset. Each individual digital twin is able to reference BIM data and production systems and exchange necessary process information.

The AAS in this scenario is the functional representation of a digital twin and can be divided into different submodels, which in turn describe a certain aspect of the concrete element with a variety of properties. The concept developed in this work is based on the structure of the AAS already designed by the authors in previous publications, as stated earlier.

Figure 1 shows the interconnections between BIM, AAS, and the new process for creating quality control support information, which can be visualized using augmented reality. Based on the BIM data, information about each element type, element instance, and the associations between instances can be transferred to individual AAS with their specialized submodels. For an integrated quality control process, geometric features of elements (edges or faces) in the BIM model can be marked as relevant for future quality control steps. These custom properties yield the basis for the association between individual feature points of elements and the later task definitions, that when fully filled out, make up the AR-supported quality control workplan.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Concept overview</figcaption></figure>

After being exported from the BIM tool in an open exchange format (like JSON or AASX), the administration shells and the defined QC submodel are made available through the AAS registry. From there they are accessed by the AR QC tool. When a submodel as specified below is found, it is interpreted as a QC workplan and is shown to the operator in AR. After the worksplan has been completed, the results are evaluated and sent back to the registry, where they are stored in the submodel for later reference.
## 3.1 Meta-model for Quality Control Sub-models with AR Support

As described earlier, the AAS of a concrete element consists of multiple submodels each describing a specific aspect. In the following, the submodels *DetailedDesign* and *QualityControl* are described in further detail.

In the *DetailedDesign* submodel, the geometry representation of the concrete element is described in the *GeometryElement*. For visualization and other related tasks, like object detection, a simplified, tessellated version of the geometry is saved in a file and referenced in this submodel element. For its simplicity and widespread adoption, the Wavefront OBJ format was chosen for this.

To precisely reference and associate the faces and edges in the quality control steps, these are also saved as explicit submodel element collections. The *GeometryElement* holds a collection of vertices, which are described as cartesian coordinates with an x, y, and z coordinate. These vertices are then in turn referenced by the individual *Geometry-Faces*, which describe a polygon in 3D space, and *GeometryEdges* which are described by a series of vertices. As these are described as *SubmodelElementCollections*, they are referable across the whole administration shell. Both *GeometryEdges* and *Geometry-Faces* do not store the actual vertices used to describe the face and edge, but only store a reference to the local identifier of vertices in the specific *GeometryElement* (Fig. 2).

<figure><img src="image_4.png" /><figcaption>Fig. 2. UML class diagram of the AAS for concrete elements (left) and detail design (right)</figcaption></figure>

The *QualityControl* submodel closely follows the metamodel of quality control for concrete elements outlined in. It consists of a series of inspection tasks that are described as co-managed entities. A task is further broken down into individual actions, which either are actions to measure the flatness of a certain face of the geometry, the length in a certain dimension, or an angle between two edges. The faces and edges are stored as local references to the edges and faces stored in the aforementioned *DetailedGeometry* submodel, on which the *QualityControl* submodel depends. After an action is completed, an *InspectionResult* is created. There, a datetime and a reference to the working operator are stored together with the corresponding measurement result. Those can be described by either a length for dimensions and flatness checking or the measured angle for angle checking actions. An associated *EvaluationTask* then compares the given result either to default evaluation criteria defined for all tasks in the *QualityControl* submodel or specific evaluation criteria that may be attached to a specific *GeometryElement*. Each
checking action results in a passed, rework-required, or failed evaluation based on the measurement and the evaluation criteria. The final evaluation result of the complete inspection task is determined by the worst evaluation of the individual checking actions.

The quality control process is limited to checking dimensions, flatness, and angles between edges. However, the concept can be expanded to other dimensioning and tolerancing criteria as long as they can be described by a relationship between the individual vertices, edges, and faces of the concrete module.

The approach breaks down the quality control activity into individual tasks, which are then further described by a series of actions. This stepwise approach is well suited to be supported by an AR application. An AR interface can show a suitable, guiding overlay and input mechanism by referencing the individual edges and faces in each *InspectionAction*. Because the edges and faces reference local vertex coordinates, this information can be attached to the physical concrete element as long as a suitable tracking approach maps world coordinates captured by the AR system to the local coordinates for the model. For that to work, both model-tracking based on the tessellated OBJ geometry and edge model tracking based on the vertex information can be used (Fig. 3).

<figure><img src="image_5.png" /><figcaption>Fig. 3. UML class diagram of the AAS for quality control</figcaption></figure>

## 3.2 BIM Integration

As shown in Fig. 1, the integration of the concept into BIM requires some additional information about each quality control relevant feature. Each element in BIM has its own set of properties, which are divided into type properties and instance properties. For example, an instance of a wall of a certain type always has certain defined properties, which are supplemented by instance properties that only apply to this specific wall. This is the basis for the creation of type- and instance-specific AAS.
The proposed quality control process requires the quality control features of an element to be uniquely identifiable and offer geometric information with their respective measuring points and criteria. For this reason, the integration should provide user-defined properties to add a “marker” to geometric features in BIM, so that further down the toolchain and within the AAS submodels, every feature can be referenced correctly.

In addition to the new data points, the integration should also provide a tessellated 3D model of the elements for visualization purposes.

An inspection task can be created from these measuring points and instructions, which, in addition to the then defined measuring points and instructions, also contain the target dimensions and permissible deviations already stored in BIM and the automatically created AAS.

# 4 Prototypical Implementation

## 4.1 Autodesk Revit Plugin for BIM Integration

Autodesk Revit 2022 was chosen to be the BIM solution that the BIM integration is conceptionally based on. The data model behind each BIM system is highly specific, so the concept might need to be adjusted to fit other BIM systems. For demonstration purposes, a BIM model of a single-family house was created in Revit 2022, with the accompanying prototype plugin being based on the Revit API C# template.

The plugin offers the user the possibility to choose the elements to be exported in the 3D model of the building in Revit. After the selection has been confirmed, the element IDs of the selection are passed to the plugin. The instance properties are queried via the unique ID of the element. One of these properties is the ID of the element type. Since many properties are stored in the element type in Revit, these are also queried and stored as type properties for the element. To export material properties of an element, all materials of the element must be queried first, since a wall can consist of masonry and insulation material, for example. Thus, the query of the materials in turn provides a list of IDs of all materials. Those are used to retrieve the material-specific general, thermal and physical properties. Other properties are retrieved identically to populate the AAS for the elements and their types. For each property, its name, its unique Revit ID, its data format, and its value are stored. This information is relevant for the creation of a property in the AAS.

It must be considered that the values of the properties are exported in the internal Revit standard units. A conversion to the required unit is therefore necessary and is performed by the plugin. In addition to the properties, the 3D geometry of the element is relevant for quality control. For this reason, the Edges and Faces and a tessellated 3D model of the element are required. For each element, the faces and edges can be retrieved via the API, where Revit offers the option to tesselate each face. The tessellation provides the point information and the normal, which lets the plugin generate an OBJ file format compatible export. At last, user-specific properties are created to identify elements and geometric features to be included in creating an AR-supported inspection task. Those properties are exported to the corresponding submodel of each element's AAS.
## 4.2 Laboratory Experiment and Validation

To validate the designed data model, a simple tablet-based AR application has been implemented using an Apple iPad and the ARKit framework. The application loads the AAS and allows the user to browse through the various submodels and properties. In particular, the application recognizes the defined semantic identifier of the special quality control and detailed design submodel elements and shows a specialized view for each of them. For the quality control assistance, the application launches an AR view shown in Fig. 4.

A sidebar displays the various inspection actions that need to be performed by the user. Upon selecting a specific action, the edge gets highlighted on the model of the concrete element. Additionally, this model can be placed in AR and overlayed over the concrete element. For this visualization, the referenced vertices, edges, and faces are extracted from the detailed design submodel.

Results and measurements can be input by the worker and saved in the result sub-model element. The result is then automatically evaluated based on the defined evaluation criteria.

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** The prototype of the AR assistance system showing the edge that needs to be measured and allows the operator to input measured values.</figcaption></figure>

The system was implemented using the ARKit 5 SDK on a 6th generation iPad Mini. ARKit only supports model tracking based on captured point clouds. To work around this limitation, the authors plan to use a separate, static object tracking setup that is responsible for calculating the transformation from world space to model space. Registration between the tablet and the world can then happen through a simple image marker that is attached to the camera setup. For the validation, only a scaled, 3D-printed model of the concrete element was used. Nevertheless, this demonstrator shows that the data model is suitable both for automatic exports from BIM systems, to create overlay information in an AR guidance system, and to save and document performed quality control steps.

In the next phase, the system is evaluated in an industrial setting. Additionally, an outside-in model-tracking system will be deployed for more reliable and flexible AR overlay information. There, a static camera system will capture images of the production
line and calculate the specific pose of the concrete element automatically based on the
provided tessellated geometry.

**5 Conclusion**

The goal of the paper at hand was to create a foundation to set up a quality control
workplan with AR content, which is associated with the digital features of a BIM model
and anchored to the equivalent physical features later in the production and quality
assurance process of precast concrete elements. The flexible nature of the AAS concept
enables the enhancement of the previously discussed AAS for precast elements with new
submodels for specialized content such as stepwise AR instructions and associations
between digital and physical features.

While the AR implementation is currently a laboratory-only demonstrator, the
general feasibility of the IT process in its entirety has been proven.

In future work, the authors plan to validate the approach on both regular precast
concrete elements, as well as more specialized high-performance concrete elements, in
real manufacturing environments.

**Funding.** This research is funded under Grant No. 423963709 by the German Research Foundation as part of the priority project "SPP2187 - Adaptive modularized constructions made in flux".

**References**

1. Akanmu, A.A., Anumba, C.J., Ogunseiju, O.O.: Towards next generation cyber-physical systems and digital twins for construction. J. Inf. Technol. Constr. **26**, 505–525 (2021). https://doi.org/10.36680/j.itcon.2021.027

2. Alves, J.B., Marques, B., Dias, P., Santos, B.S.: Using augmented reality for industrial quality assurance: a shop floor user study. Int. J. Adv. Manuf. Technol. **115**(1–2), 105–116 (2021). https://doi.org/10.1007/s00170-021-07049-8

3. Azuma, R.T.: A survey of augmented reality. Presence: Teleoperators Virtual Environ. **6**(4), 355–385 (1997)

4. Boje, C., Guerriero, A., Kubicki, S., Rezgui, Y.: Towards a semantic construction digital twin: directions for future research. Autom. Constr. **114**, 103179 (2020). https://doi.org/10.1016/j.autcon.2020.103179

5. Chen, K., Xue, F.: The renaissance of augmented reality in construction: history, present status and future directions. Smart Sustain. Built Environ. (2020). https://doi.org/10.1108/SASBE-08-2020-0124

6. Kosse, S., Vogt, O., Wolf, M., König, M., Gerhard, D.: Digital twin framework for enabling serial construction. Front. Built Environ. **8**, 864722 (2022). https://doi.org/10.3389/fbuil.2022.864722

7. Lee, S., Kwon, S., Jeong, M., Hasan, S., Kim, A.: Automated On-Site quality inspection and reporting technology for Off-Site Construction(OSC)-based precast concrete members. In: Proceedings of the 37th International Symposium on Automation and Robotics in Construction (ISARC). Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). International Association for Automation and Robotics in Construction (IAARC) (2020). https://doi.org/10.22260/ISARC2020/0158
8. Leygonie, R., Motamedi, A.: quality management processes and automated tools for FM-BIM delivery. In: Proceedings of the 38th International Symposium on Automation and Robotics in Construction (ISARC). Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). International Association for Automation and Robotics in Construction (IAARC) (2021). https://doi.org/10.22260/ISARC2021/0120

9. Masood, T., Egger, J.: Adopting augmented reality in the age of industrial digitalisation. Comput. Ind. 115, 103112 (2020). https://doi.org/10.1016/j.compind.2019.07.002

10. Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: a class of displays on the reality-virtuality continuum. In: SPIE Proceedings, Telemanipulator and Telepresence Technologies, pp. 282–292 (1994)

11. Motamedi, A., Iordanova, I., Forgues, D.: FM-BIM preparation method and quality assessment measures. In: 17th International Conference on Computing in Civil and Building Engineering (ICCCBE), Tampere, Finland (2018)

12. Porcelli, I., Rapaccini, M., Espíndola, D.B., Pereira, C.E.: Technical and organizational issues about the introduction of augmented reality in maintenance and technical assistance services. IFAC Proc. 46(7), 257–262 (2013). https://doi.org/10.3182/20130522-3-BR-4036.00024

13. Ratajczak, J., Schweigkofler, A., Riedl, M., Matt, D.T.: Augmented reality combined with location-based management system to improve the construction process, quality control and information flow. In: Mutis, I., Hartmann, T. (eds.) Advances in Informatics and Computing in Civil and Construction Engineering, pp. 289–296. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-00220-6_35

14. Röltgen, D., Dumitrescu, R.: Classification of industrial augmented reality use cases. Procedia CIRP 91, 93–100 (2020). https://doi.org/10.1016/j.procir.2020.01.137

15. Segovia, D., Mendoza, M., Mendoza, E., González, E.: Augmented reality as a tool for production and quality monitoring. Procedia Comput. Sci. 75, 291–300 (2015). https://doi.org/10.1016/j.procs.2015.12.250

16. Shahzad, M., Shafiq, M.T., Douglas, D., Kassem, M.: Digital twins in built environments: an investigation of the characteristics, applications, and challenges. Buildings 12(2), 120 (2022). https://doi.org/10.3390/buildings12020120

17. Siewert, J.L., Wolf, M., Böhm, B., Thienhaus, S.: Usability study for an augmented reality content management system. In: Auer, M.E., May, D. (eds.) REV 2020. AISC, vol. 1231, pp. 274–287. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-52575-0_22

18. Wang, X., Kim, M.J., Love, P.E., Kang, S.-C.: Augmented reality in built environment: classification and implications for future research. Autom. Constr. 32, 1–13 (2013). https://doi.org/10.1016/j.autcon.2012.11.021

19. Wolf, M., Vogt, O., Huxoll, J., Gerhard, D., Kosse, S., König, M.: Lifecycle oriented digital twin approach for prefabricated concrete modules. In: Semenov, V.U., Scherer, R.J. (Hrsg.): ECPPM 2021 – eWork and eBusiness in Architecture, Engineering and Construction, pp. 305–312. CRC Press, London (2021). https://doi.org/10.1201/9781003191476-42