<figure><img src="image_1.png" /></figure>

HAL
open science

# The Potential of Game Development Platforms for Digital Twins and Virtual Labs

Ali Abdallah, Matthias Primas, Ioan Turcin, Udo Traussnigg

<figure><img src="image_2.png" /></figure>

## To cite this version:

Ali Abdallah, Matthias Primas, Ioan Turcin, Udo Traussnigg. The Potential of Game Development Platforms for Digital Twins and Virtual Labs. IFIP International Conference on Advances in Production Management Systems (APMS), Aug 2020, Novi Sad, Serbia. pp.117-121, 10.1007/978-3-030-57997-5_14. hal-03635694

HAL Id: hal-03635694

https://inria.hal.science/hal-03635694

Submitted on 8 Apr 2022

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# The Potential of Game Development Platforms for Digital Twins and Virtual Labs

## Case Study of an Energy Analytics and Solution Lab

Ali Abdallah¹[0000-0002-5470-1481], Matthias Primas¹, Ioan Turcin¹ and Udo Traussnigg¹

¹ CAMPUS 02, University of Applied Sciences, Graz A-8010, Austria
ali.abdallah@campus02.at

In this paper, we present the first steps towards realizing a digital twin with integrated virtual laboratory possibilities, for a newly established Energy Analytics and Solution lab, using the Unity3D game development platform. The presented example is a case study that shows the possibilities of such development environment for creating a fully connectable digital twin of an energy analytics lab and other more complex industrial environments.

**Keywords:** Virtual Reality, Virtual Lab, Digital Twin, Unity 3D, Energy Analytics lab.

## 1 Introduction

The digitalization of a physical object or system can be done at various levels; ranging from a simple digital model that has no exchange of data between the model and the real entity, to a digital shadow where there is a one way data transfer and last but not least a digital twin (DT) [1]. A widely accepted definition of a digital twin was introduced by [2], and subsequently by [3]: a DT is defined as an integration of multi-physics and multi-scale simulation of a complex product. These simulations use available physical models and outside sensor data (or other measurements) to mirror a corresponding physical twin.

Big advancements in DTs are enabled by developments in the areas of simulation, communication protocols and very important visualization [1]. Virtual reality (VR), and by extension augmented reality (AR), offers itself as very strong and immersive tool for the visualization of DTs and game development engines such as Unity 3D offer a free access tool for developing VR applications with high levels of integration possibilities. The uses of VR for DT applications in various areas are discussed in works such as [4], [5] and [6].

In this paper we introduce the first step towards creating a DT for a newly established energy analytics and solution lab (EAS-Lab), where Unity 3D has been used to digitalize the lab and connect various relevant measurements and other elements from the actual lab to the virtual world. The developed virtual lab is also intended for teaching purposes. The use of virtual labs for various teaching purposes is reviewed in [7].
## 2 Energy analytics and solution lab

The aim of the project Energy Analytics and Solution lab (EAS-Lab) is to be able to analytically describe and develop core areas in the existing and future energy value chain in lab conditions. For this purpose, a digital energy research laboratory, the EAS-Lab, was built both at the University of Applied Sciences JOANNEUM in Kapfenberg and at CAMPUS 02 in Graz. The project, which was concluded in the beginning of 2020, includes the installation of renewable energy generation systems, hybrid energy systems with energy storage hubs, charging stations for electric mobility, and measurement and automation components on both sites. With these components, it will be possible to analyse, develop and improve existing and future questions relating to energy and business models. In addition, the labs will be suitable for investigating innovative charging technologies for electric mobility and their technical implications.

The EAS-Labs consists of a network of highly connected renewable energy generators (mostly photovoltaic cells), energy management, storage and testing loads covering a wide range of typical household and industrial loads including e-mobility. A smart home infrastructure ensures the high level of connectivity and control needed to create an effective testing lab environment. Several software tools for simulation, visualization and energy management, automation and control supplement the hardware components of the EAS-labs. One of the most important tools in use is the XAM-Control software, which is an industrial tool used for visualization, programming and control of automated processes.

All measured values are transferred to a server on which the XAM Control software is installed. All components (measuring devices, energy storage devices, charging stations...) must therefore have a suitable interface for data transmission (e.g. Modbus TCP, Ethernet). The XAM Control takes over the control of all energy flows in the laboratory (energy and load management).

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Diagram depicting the core elements of the Energy Analytics and Solution Lab (EAS-Lab). This diagram shows how all the main elements of the lab connect together on both energy and data exchange levels.</figcaption></figure>
## 3 EAS-lab case study: digital twin and VR-lab

In this section we shortly introduce the first steps taken towards the realization of a DT and virtual lab of the above introduced EAS-Lab.

The motivation of the presented work is to create an environment that replicates the EAS-Lab in both presentation and functionality, and going along the definition of a DT, offers communication in both ways, between the virtual and the physical lab. This also allows for the implementation of a connected virtual laboratory, where various energy systems concepts can be presented to students in a safe environment.

Unity 3D, a game development engine, is chosen for the implementation of the EAS-Lab DT. Unity supports C# scripts allowing for great flexibility in the implementation of the communication protocols needed to connect the EAS-Lab to its DT and vice versa. The implementation of serial and Modbus TCP communication was the first step, and was realized in the scope of a master thesis [8]. These communication protocols were specifically chosen because they are often used in energy management and home/facility automation systems.

In the above-mentioned master thesis, the successful implementation of both communication protocols was demonstrated through the integration of a measurement station in the EAS-Lab DT. The temperature and humidity measurements were made in the lab using a DHT11 sensor and a microcontroller (Arduino) setup. Data communication to the physical EAS-Lab was demonstrated through the control of various LEDs. Figure 2 below shows the digital setup and the corresponding physical prototype.

This kind of examples also demonstrate the data visualization possibilities presented by a VR environment. The various communication possibilities between the DT and the physical setup are shown in the diagram of Figure 3.

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Figure showing the lab measurement station setup introduced in [8] in both its digital form (left) and as a physical prototype (Right).</figcaption></figure>
Using software development kits (SDK) available for Unity 3D (ZF-Browser), we also implemented a web based interface, allowing the display and control of web interfaces through the VR-environment. This can be especially useful since many home automation systems use web-based control interfaces. Using a game development engine makes it also easier to integrate intractable elements in the DT/Virtual lab, making the DT also into a powerful marketing tool for the scientific topics of the EAS-Lab, significantly adding to the visibility, and increasing the interest in various energy systems and energy management topics.

<figure><img src="image_6.png" /><figcaption>**Fig. 3.** Diagram depicting the various communication and control possibilities between the implement experimental setup in the DT and both the physical lab and the developed Arduino board.</figcaption></figure>

# 4 Conclusion

In this work, we have introduced a DT implementation of an energy analytics lab using a game development engine in an interactive VR environment, in addition to introducing the newly established Energy Analytics Solution (EAS)- Labs.

We have shortly shown the integration of a virtual lab module and the implementation of various communication protocols in the virtual presentation of the lab, thus creating the basic building blocks for a scalable DT of the EAS-Lab. This was established by communicating from a Unity 3D environment with a microcontroller in the physical
lab through both a serial and a TCP protocol. Successful communication in both direc-
tion, i.e. from and to the microcontroller, is a successful first step towards a full DT of
the EAS-lab. In addition, the creation of the VR-environment is important for various
visualization applications in the newly established labs.

The work shown here holds great potential for the EAS-Lab, opening the door for
future scientific work in the areas of VR-based energy systems simulations and visual-
ization and digital twins of renewable energy systems, all while acting as a marketing
tool with great reach.

References

1. W. Kritzinger et al., Digital Twin in manufacturing: A categorical literature review and clas-sification, IFAC-PapersOnLine, Volume 51, Issue 11, 2018, Pages 1016-1022, ISSN 2405-8963, https://doi.org/10.1016/j.ifacol.2018.08.474.

2. Glaessgen, E. H.; Stargel, D. S. (2012). The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles. In 53rd Struct. Dyn. Mater. Conf. Special Session: Digital Twin, Honolulu, HI, US.

3. Tao, Fei; Cheng, Jiangfeng; Qi, Qinglin; Zhang, Meng; Zhang, He; Sui, Fangyuan (2017). Digital twin-driven product design, manufacturing and service with big data. In Int J Adv Manuf Technol, 10 (4), p. 2233.

4. G. Schroeder et al., "Visualising the digital twin using web services and augmented reality," 2016 IEEE 14th International Conference on Industrial Informatics (INDIN), Poitiers, 2016, pp. 522-527.

5. H. Laaki, Y. Miche and K. Tammi, "Prototyping a Digital Twin for Real Time Remote Con-trol Over Mobile Networks: Application of Remote Surgery," in IEEE Access, vol. 7, pp. 20325-20336, 2019.

6. Andaluz V.H. et al. (2016) Unity3D-MatLab Simulator in Real Time for Robotics Applica-tions. In: De Paolis L., Mongelli A. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2016. Lecture Notes in Computer Science, vol 9768. Springer, Cham

7. V. Potkonjak et al., Virtual laboratories for education in science, technology, and engineer-ing: A review, Computers & Education, Volume 95, 2016, Pages 309-327, ISSN 0360-1315, https://doi.org/10.1016/j.compedu.2016.02.002.

8. Barbara Semler, Visualisierung und virtuelle Steuerung eines Messtisches in einem realen Labor.