# Supporting Digital Twin Integration Using Semantic Modeling and High-Level Architecture

Han Li, Jinzhi Lu, Xiaochen Zheng, Guoxin Wang, Dimitris Kiritsis

► To cite this version:

Han Li, Jinzhi Lu, Xiaochen Zheng, Guoxin Wang, Dimitris Kiritsis. Supporting Digital Twin Integration Using Semantic Modeling and High-Level Architecture. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2021, Nantes, France. pp.228-236, 10.1007/978-3-030-85910-7_24. hal-03806486

HAL Id: hal-03806486

https://inria.hal.science/hal-03806486

Submitted on 7 Oct 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Supporting Digital Twin Integration Using Semantic Modeling and High-Level Architecture

Han Li¹, Jinzhi Lu²*, Xiaochen Zheng², Guoxin Wang¹* and Dimitris Kiritsis²

¹ Beijing Institute of Technology, Beijing 100081, China
² EPFL SCI-STI-DK, Station 9, CH-1015 Lausanne, Switzerland
jinzhi.lu@epfl.ch

**Abstract.** Digital twin (DT) provides a solution for supporting the interconnection between the physical world and the virtual world. When implementing DT integration, it is challenging to implement interface definition, information and service integration across DTs. This paper proposes a semantic modeling approach with a High-Level Architecture (HLA) to support the DT integration. The semantic modeling approach based on Graph-Object-Property-Point-Role-Relationship (GOPPRR) meta-meta models is used to realize the integrated formalisms of heterogeneous DTs. HLA is used to support interface definition and service integration between virtual entities of DT. Finally, a case of an unmanned aerial vehicle (UAV) landing on ship is used to verify the flexibility of this approach. From the results, we find the GOPPRR ontology and HLA specification enables to provide a unified formalism of the DTs of UAV and the ship, and to implement data exchange during the distributed simulation execution.

**Keywords:** Digital Twin, Semantic Model, Ontology, Distributed Simulation.

## 1 Introduction

In the context of Industry 4.0, Cyber Physical Systems (CPSs) have become priority issues and core technologies in the manufacturing industry [1,2]. Computation, Communication and Control (3C) technologies are three main compositions in CPS aiming to realize deep integration and organic collaboration across domains. They are also expected to create the interconnection and interoperability between the virtual world and the physical world. Currently, Digital Twin (DT) is considered as one way to integrate the physical space and virtual space. DT essentially include simulation models related to physical things in the real world. It provides the mappings of the physical systems to the digital models in cyberspace from the perspective of geometry, physics, behavior, etc., ensuring the coordination of the virtual world and physical world. It can also realize real-time monitoring, data collection, simulation, analysis and reasoning of physical objects based on digital models [3].

When building a DT for a CPS, it is necessary to integrate virtual entities in different domains which is a challenging task due to the high heterogeneity. This challenge includes interface definition, data and information integration and service integration.
When integrating the heterogeneous models across domains, the interface specifica-
tions should be defined in a standardized approach to enable the development of APIs
for data exchange between DTs. Data and information integration is the basis to the
integration of DT. For example, Singh et al. summarized the five challenges about DT,
of which data integration is the most important aspect [4]. The end users expect to ar-
range different DTs according to their own requirements regardless the DT is developed
for such business or not. This brings new challenges to the use of existing DTs for the
additional purposes and business.

This paper proposes a semantic modeling approach with High-Level Architecture (HLA) standard to support the integration of DTs. A semantic modeling approach based on Graph-Object-Property-Point-Role-Relationship (GOPPRR) is used to construct DT ontology model, which can realize the integrated descriptions of heterogeneous DT, such as DT creators and descriptions. HLA is a standard for the modeling and simulation of distributed, heterogeneous processes. HLA provides standard interfaces to facilitate the integration of service and interface. The main contribution of this paper is to adapt a semantic modeling approach for DT integration based on HLA.

The rest of this paper is organized as follows. We discuss the related work in Section 2. In Section 3, we specifically introduce the semantic modeling approach using HLA, including ontology design and semantic modeling of DT and DT integration implemented based on HLA. Then Section 4 describes the case study to clarify and validate the proposed approach. Finally, we offer the conclusions and future work in Section 5.

## 2 Related Work

The concept of DT can be traced back to Grieves's speech on product lifecycle man-
agement (PLM) in 2003. Then NASA defines DT as "an integrated multi-physics,
multi-scale, probabilistic simulation of a vehicle or system that uses the best available
physical models, sensor updates, fleet history, etc., to mirror the life of its flying twin"
[5]. A basic Digital Twins consists of three aspects: physical space, virtual space, and
the connection between them for exchanging data and information [6]. In CPS, the DT
can be understood as a digital model inside the CPS, which is a virtual representation
that contains all physical information and knowledge [7, 8].

DT involve different systems during the development process, which makes the in-
tegration of heterogeneous data difficult [4, 9]. However, semantic modeling technolo-
gies provide solutions for data integration of DT [10]. At present, existing studies have
attempted to combine semantic models with DT through ontologies. The combination
of UML and ontology is proposed to find a common system-level semantic model. The
authors of [11] proposed a method of communicating through human-readable text and
computer-readable models, which is very important for engineering experts, software
developers, and decision makers. The authors of [12] proposed an ontology that repre-
sents a DT in the context of CPS and embedded systems, which solves the problem of
fusion of physical space and virtual space. In addition, knowledge graphs have been
applied to acquire and integrate information into an ontology and utilize a reasoner to
derive new knowledge [13]. For example, a query language based on the knowledge
graph is used to enhance the manufacturing process management with reasoning ability [14].

Semantic modeling is important for the construction of DT, but it still lacks a unified architecture, and cannot solve the integration problem of model information in different domains. For this reason, more research work is needed.

# 3 Digital Twin Integration Using Semantic Model and HLA

In this section, the semantic modeling approach based on HLA is first introduced. Then ontology definition to represent DT information is introduced. Finally, distributed simulation based on HLA for DT interaction and service integration is proposed. Ontologies can be used to represent domain knowledge for an agreed understanding between different applications, and it contributes to semantics descriptions and models. Thus ontology is used as the supporter of semantic modeling in this paper.

## 3.1 Semantic modeling method using HLA

In order to solve the challenges of interface integration, data integration, and service integration in the process of DT integration, this paper proposes an approach to support DT integration using HLA and semantic modeling, as shown in Fig. 1.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Semantic modeling method using HLA</figcaption></figure>

A semantic modeling approach is used to describe the topologies between virtual entities for information integration. The GOPPRR meta-modeling approach provides standardized and semantic expressions for different data structures, which is used as the basis of ontology definition and semantic modeling using Ontology Web Language (OWL). DT Information can be described according to Class, Object Property, Data Property and Instance by OWL.

In order to define standardized interfaces of heterogeneous DT, the distributed simulation is used based on HLA. HLA provides specifications, which effectively define
interfaces across DT. In HLA, federate is the basic component for the distributed simulation. When integrating DT and HLA, Federate and virtual entities construct Digital Twin Federate (DTF). Thus, each DT has its related DTF referring to a standardized interface. Semantic models can express the interactions between DTFs and are used to generate the compiled codes for executing co-simulations. Thus, DT exchange information through Run-time Infrastructure (RTI), to implement simulation processes for service integration.

## 3.2 Semantic Modeling for Digital Twin

The ontology is modeled based on the GOPPRR approach which is a modeling framework based on a M0-M3 framework. The M0 layer represents the physical entities of DT in the real world; the M1 layer represents the domain model or virtual entities of DT. The M2 layer represents meta-models, which usually represents the model compositions as a domain specific modeling language. The M3 layer represents the meta-meta model, including six elements of Graph, Object, Relationship, Property, Role, and Point. Table 1 lists a specific definition of GOPPRR.

<table><caption>**Table 1.** The description of meta-meta model</caption><thead><tr><td>Meta-meta model</td><td>Description</td></tr></thead><tbody><tr><td>Graph</td><td>A collection of objects, relationships, roles, points and their relationships which refers to a model diagram.</td></tr><tr><td>Object</td><td>An object type, it is the basic element in the diagram.</td></tr><tr><td>Point</td><td>It describes the port connected to the relationship in the object.</td></tr><tr><td>Property</td><td>It is used to describe their characteristics and is different from system properties which cannot exist alone.</td></tr><tr><td>Relationship</td><td>It is used to indicate how objects are connected, and each "relationship" is connected to objects through roles.</td></tr><tr><td>Role</td><td>It is used to represent the end of the relationship connecting objects and relationships.</td></tr></tbody></table>

OWL can completely express the information in the semantic modeling process and support the exchange of information across virtual entities [15]. This paper uses OWL to describe the GOPPRR information. The ontology using OWL is shown in Fig. 2. Class is used to define GOPPRR meta-meta model, and its subclasses are used to describe meta-model. The Object-property is used to represent the interaction relationship among the six meta-meta models, such as inclusion, binding, etc. The Individual represents models referring to the information of specific DT virtual entities.
<figure><img src="image_4.png" /><figcaption>Fig. 2. Ontology supports GOPPRR method</figcaption></figure>

Through the GOPPRR modeling approach, meta models are used for constructing models for representing DT. In order to transform GOPPRR models to HLA models, the meta-models are developed based on HLA specifications. The details of HLA are introduced in the next section.

## 3.3 HLA Supports Digital Twin Integration

HLA is a universal standard protocol for distributed simulation. It consists of three parts: Interface Specification, Object Model Template (OMT) and HLA Rules. Interface Specification defines the services that support interoperability among federates during the simulation run. OMT defines a set of components that describe the HLA object model. HLA rules are the basic guidelines that must be followed in simulation design. HLA provides a set of rules based on the following main concepts to achieve interoperability and reusability when constructing the standardized interfaces for DT:

* **Federation**: A simulation application composed of a set of simulation components.

* **Federate**: One simulation component that represents the basic elements of HLA.

* **Run-time Infrastructure (RTI)**: Simulation-oriented middleware for managing alliance interaction.

<figure><img src="image_5.png" /><figcaption>Fig. 3. A Digital Twin Federation based on HLA</figcaption></figure>

Fig. 3 shows a Digital Twin Federation based on HLA. The federation contains a number of DTFs based on HLA Interface Specification and OMT. DTF is composed of
simulation model referring to virtual entities of DT, federate code and local RTI component (LRC). The simulation model consists of federate objects that meet the HLA standard aiming to construct the data exchange interface of each virtual entity into the request and response to RTI using the HLA Interaction Specification. Federate code is an application program that performs local DT executions and defines interactions to RTI. The LRC is the API component for the DTF to communicate with the RTI.

The semantic models mentioned in Section 3.2 are used to transform the HLA model. The concept of GOPPRR is used to describe the interactive information between different virtual entities based on HLA. Therefore, it is necessary to define meta-models using the HLA concept. As shown in Table 2, different meta-models are developed to support HLA concept formalisms. In details, DTFs are mainly composed of HLA OMT, in which Publish and Subscribe mechanism is used to describe the interaction between federates: Publish object class of a federate means that this member can generate an instance of the object class and modify its attributes, and Subscribe interaction class means that this member can receive information with parameters.

<table><caption>**Table 2.** The mappings between HLA concepts and GOPPRR meta-models</caption><thead><tr><td>Meta-metamodel</td><td>Meta-model</td></tr></thead><tbody><tr><td>Graph</td><td>Federation Object Model(FOM)<br/>Federation, Federate, PathSpace, ObjectClass, Ob-<br/>jectClassAttribute, InteractionClass, InteractionClassPa-<br/>rameter</td></tr><tr><td>Object</td><td></td></tr><tr><td>Point</td><td></td></tr><tr><td>Property</td><td>DeliveryMechanism, DeliveryOrder, UpdateType, Time-<br/>Constrained, TimeRegulating, ect.</td></tr><tr><td>Relationship</td><td>Publish, Subscribe, Join, Have<br/>PublishFrom, PublishTo, SubscribeFrom, SubscribeTo,<br/>JoinFrom, JoinTo, HaveFrom, HaveTo</td></tr><tr><td>Role</td><td></td></tr></tbody></table>

The HLA Interface Specification provides a unified standard for DT integration as interface definitions, enabling different simulation components to interact effectively. The co-simulation between different DT can be realized through RTI, and the integration of simulation services can be realized for different business and purposes.

## 4 Case Study

Here we use the system simulation scenario of unmanned aerial vehicle (UAV) landing to verify our DT semantic modeling method using HLA. As shown in Fig. 4-A, a system consisting of one UAV, one warship and one radar is demonstrated. The simulation scenario is specifically described as follows: the UAV has executed the mission and is planning to land on the warship. Both the UAV and warship have their own initial location (x, y, z) and the warship has a fixed acceleration (dx2, dy2, dz2). When the UAV is landing on the warship, the radar obtains the UAV and warship location informations in real time, and calculates the best trajectory of the UAV and sends it, so the UAV can
obtain its own acceleration (dx1, dy1, dz1). This paper simplified the UAV and the warship, assuming that they are mass points without volume, and does not consider the specific navigation algorithm used.

In this case, we focus on whether our method supports DT integration, rather than simulation results. The UAV, the warship and the radar are considered as the DTFs and RTI is the medium of communication. First, the ontology model is constructed using the semantic modeling method in Section 3, the meta-models in Table 2 are used to generate the individuals, which represent the information of DT. Then each federate simulation code is developed, and the initial locations and acceleration of warship are provided to initialize the simulation.

Fig. 4-B shows the ontology model of this case. The ontology model includes instances of Graph, Object, Property, Point, Role, and Relationship according to the UAV landing scene. The ontology model records the attributes of DT virtual entities and all the interactions between them. The ontology contains three Federate object instances, two ObjectClass object instances, one InteractionClass object instance, two ObjectClassAttribute object instances, and one InteractionClassParameter object instance. Through owl: ObjectProperty, the relationship between different instances can be established to represent the interactive information between federates.

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** UAV automatic landing simulation A: The simulation scenario; B: Ontology model of UAV landing Federation</figcaption></figure>

Fig. 5-A shows the simulation execution process and result of the DT federation in RTI, where the RTI software is CERTI, an open source RTI software. The co-simulation includes three federate simulation applications and a RunTime Infrastructure Gateway (RTIG). RTIG is the process of coordinating HLA federates with CERTI, and a federation must have at least one RTIG process. Each federate simulation application is developed based on the ontology model shown in Fig. 4-B, and uses CERTI's related APIs. The simulation results is shown in Fig. 5-B.
<figure><img src="image_7.png" /><figcaption>Fig. 5. Simulation applications work together by CERTI and simulation results</figcaption></figure>

The application execution shows that the ontology model can completely, accurately and clearly express the DT models in different domains, so as to achieve information integration. And HLA can effectively solve the problems that different hardware, memory or interface specifications of heterogeneous systems make it impossible to realize the DT model in a stand-alone or integrated manner. The service of DT is the purpose of using DT, and DT should support the integration of simulation services of each system. HLA realizes the co-simulation between independent models through RTI, which not only meets the simulation requirements of a single virtual entity, but also realizes the overall simulation service integration.

## 5 Conclusion

This paper proposes a semantic modeling approach to support the integration of DT using HLA. First, a unified semantic modeling approach is established through GOPPRR, which integrates DT information of heterogeneous data structures into a unified ontology. Then through transformation from GOPPRR ontology model to HLA model, the distributed simulation is executed across DT based on HLA RTI. Finally, a case is used to verify the feasibility of the proposed approach.

The ontology models using HLA support DT integration. However, the related specifications of HLA are more complicated, and the simulation federation development process often requires a significant expertise and a considerable effort. This has led to greatly reduced development efficiency and increased development costs when building complex DT. Since the ontology model has the characteristics of being recognized by the computer, it can be considered to automatically generate simulation application programs through the ontology model to reduce the complexity of simulation application development. In the future, a complete tool kit for ontology modeling, distributed simulation and visualization, will be developed to support the automatic distributed simulation executions for DT integration.
References

1. Lee, E.: Cyber Physical Systems: Design Challenges. In: 2008 11th IEEE Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing, Orlando (2008).

2. Chen, Z., Guo, S., Wang, J., Li, Y., Lu, Z.: Toward FPGA Security in IoT: A New Detection Technique for Hardware Trojans. Ieee Internet of Things Journal 6(4), 7061-7068(2019).

3. Foivos, P.: A generic methodology and a digital twin for zero defect manufacturing (ZDM) performance mapping towards design for ZDM. J. Manuf. Syst. (2021).

4. Qi, Q., Tao, F., Hu, T., et al.: Enabling technologies and tools for digital twin. J. Manuf. Syst. (2019).

5. Glaessgen, E., Stargel, D.: The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles. Aiaa/asme/asce/ahs/asc Structures, Structural Dynamics & Materials Conference Aiaa/asme/ahs Adaptive Structures Conference Aiaa (2012).

6. Grieves, M.: Digital Twin: Manufacturing Excellence through Virtual Factory Replication. A Whitepaper by Dr. Michael Grieves. White Paper, 1–7(2014).

7. Rosen, R., von, Wichert, G., Lo, G., Bettenhausen KD.: About The Importance of Autonomy and Digital Twins for the Future of Manufacturing. Ifac Papersonline 48(3), 567-72 (2015).

8. Tao, F., Zhang, M., Cheng, J., Qi, Q.: Digital twin workshop: a new paradigm for future workshop. Jisuanji Jicheng Zhizao Xitong/Comput. Integr. Manuf. Syst. CIMS (2017).

9. Guo, L., Liu, Q., Shi, K., Gao, Y., Luo, J., Chen, J.: A Blockchain-Driven Electronic Contract Management System for Commodity Procurement in Electronic Power Industry. Ieee Access 9, 9473-9480 (2021).

10. Cho, S., May, G., Kiritsis, D.: A semantic-driven approach for Industry 4.0. In: 2019 15th International Conference on Distributed Computing in Sensor Systems, pp. 347-54(2019).

11. Rzevski, G., Lakhin, O., Skobelev, P., Mayorov, I., et al.: Ontology-Driven Multi-Agent Engine for Real Time Adaptive Scheduling. In: 2018 International Conference on Control, Artificial Intelligence, Robotics & Optimization. (2018).

12. Steinmetz, C., Rettberg, A., Ribeiro, FGC.,et al.: Internet of Things Ontology for Digital Twin in Cyber Physical Systems. In: 2018 Viii Brazilian Symposium on Computing Systems Engineering. Brazilian Symposium on Computing System Engineering, pp. 154-9(2018).

13. Lu, J., Zheng, X., Gharaei, A., Kalaboukas, K., Kiritsis, D.: Cognitive Twins for Supporting Decision-Makings of Internet of Things Systems. In: Proceedings of 5th International Conference on the Industry 40 Model for Advanced Manufacturing. Lecture Notes in Mechanical Engineering, pp. 105-15(2020).

14. Banerjee, A., Dalal, R., Mittal, S., Joshi, KP.: Generating Digital Twin models using Knowledge Graphs for Industrial Production Lines. In: Proceedings of the 2017 Acm Web Science Conference (2017).

15. Wang. H., Wang. G., Lu. J., Ma. C.: Ontology Supporting Model-Based Systems Engineering Based on a GOPPRR Approach. In: Rocha Á., Adeli H., Reis L., Costanzo S. (eds) New Knowledge in Information Systems and Technologies. WorldCIST'19 2019. Advances in Intelligent Systems and Computing, vol 930. Springer, Cham (2019).