<figure><img src="image_1.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Digital twin-based services and data
visualization of material handling equipment in
smart production logistics environment

Yongkuk Jeong¹[0000-0003-1878-773X], Erik Flores-García¹[0000-0003-0798-0753], Dong Hoon Kwak², Jong Hun Woo², Magnus Wiktorsson¹[000-0001-7935-8811], Sichao Liu³[0000-0002-1909-0507], Xi Vincent Wang³[0000-0001-9694-0483], and Lihui Wang³[0000-0001-8679-8049]

¹ KTH Royal Institute of Technology, Södertälje 151 36, Sweden
{yongkuk,efs01,magwik}@kth.se

² Seoul National University, Seoul 08826, Republic of Korea
{s2arta2s,j.woo}@snu.ac.kr

³ KTH Royal Institute of Technology, Stockholm 114 28, Sweden
{sicliu,wangxi,lihuiw}@kth.se

**Abstract.** Smart production logistics has introduced in manufactur-
ing industries with emerging technologies such as digital twin, industrial
internet of things, and cyber-physical system. This technological innova-
tion initiates the new way of working, working environment, and decision-
making process. Especially the decision-making process has changed from
experience and intuition to knowledge and data driven. In this paper,
digital twin-based services, and data visualization of material handling
equipment in smart production logistics environment are presented. There
are several applications of digital twin in manufacturing industries al-
ready, however feedback from the virtual environment to physical envi-
ronment and interactions between them which are the essential features
of digital twin are very weak in many applications. Therefore, we have
developed digital twin-based services in the laboratory scale including
feedback and interaction. In addition, data visualization application of
material handling equipment in automotive industry is presented to pro-
vide insights to the users. Both applications have developed based on the
same framework including database and middleware, so it has possibili-
ties to develop further in the future.

**Keywords:** Digital twin · Digital service · Data visualization · Smart production logistics

# 1 Introduction

New technological solutions such as Cyber-Physical Systems (CPS), Industry 4.0 and Internet of Things (IoT) are creating opportunities in various industries including the production logistics [1]. These solutions enabled researchers to develop new paradigm of production logistics called, “Smart Production Logistics
(SPL)”, which addresses to fully autonomous systems to increase the perfor-
mance of the production system [2]. The SPL can be achieved with the ability
of logistic objects to process information, to render and to execute decisions on
their own [3]. Various technologies are contributing to this goal including CPS,
IoT, interfaces, decentralized applications, automatic identification, virtual en-
vironments like Digital twins, and the branches of data science like machine
learning, data mining, and big data analytics [4]. The demand of digitization
and digitalization in production logistics to maximize the performance and the
efficiency of the system adopting the technologies has increased the most [5].

The virtual environments are used in production logistics to simulate various scenarios that couldn't be conducted in reality due to limitations like cost and time. Digital twins, simulations, augmented reality, and virtual reality could be classified as the virtual environments [6]. Digital twin (DT) is defined as "a virtual representation of a physical system that is updated through the exchange of information between the physical and virtual system" [7]. Thus, DT should include physical system, virtual environment, and connection between the physical system and virtual environment generating data and information flow. The characteristics of the DT enables the system to ensure the improvement of data driven approach to the monitoring, management, and knowledge [8]. Due to its capability of high-fidelity reflection of physical system, the DT has been highly contributed in smart manufacturing system design phase where it demands to minimize the gap between the design domain and operation domain [9].

Therefore, this study aims to provide a prototype service of DT technology in SPL improving the efficiency of decision making process. The study is conducted in production logistics design process involving multiple material handling equipment, so called, Autonomous Mobile Robot (AMR). To assist the decision making process in the service, data visualization derived from production logistics environment is also addressed in this study.

The rest of the paper is organized as follows. A background of the recent
studies of DT application and production logistics data visualization are pro-
vided in section 2. The applications of DT and data visualization in production
logistics environment are presented in section 3. The overall objective of this
study and future challenges are described in section 4.

## 2 Related works

2.1 Digital twin applications in manufacturing

With rapid growth of interest toward the smart manufacturing system, there
has been significant efforts on DT implementation for integrating information
technologies and operation technologies in various industries [10]. DT application
not only includes connection between the cyber and physical world, but also
emphasizes the synchronization between the two by offering monitoring and
optimization services and capability to control from the cyber world to enhance
decision making process [11].
These aspects of DT has made significant improvements in different manu-
facturing services from entire business processes to operational procedures [12].
For instance, [13] proposed DT based production process design framework in
automobile assembly line to minimize design errors that can lead to signifi-
cant delays in the design process. The reflection of physical system in virtual
environment enabled designers to test different designs of luminaire in differ-
ent application scenarios with DT framework [14]. [15] conducted DT-driven
methodology for designing automated flow-shop manufacturing system in pre-
production phase considering static configuration and dynamic execution at the
same time. There are also implementations of DT monitoring and optimizing
the operation procedures [11]. [16] proposed a system architecture for the virtu-
alization of manufacturing machines as a middleware to monitor and control the
physical machines. [17] proposed a DT-based process knowledge model includ-
ing real-time equipment status and accumulation of process knowledge for reuse
in future planning. [18] presented a case study evaluating energy efficiency re-
duction using simulations of the DT. In this study, buffering-based solution was
introduced for improving the energy efficiency in surface mount technology PCB
assembly line. To provide safe and productive interaction between human and
mobile robot, [19] proposed augmented reality solution and supporting system
architecture for smart warehouse.

DT can be utilized more than a static environment in cases where the situ-
ation frequently changes dynamically, such as in a production logistics environ-
ment. However, although many DT application cases have already been studied,
it is very difficult to find a case that fully applies the interaction between the
physical and virtual environments.

## 2.2 Production logistics data visualization

Data analytics is necessary for the implementation of SPL by enabling to analyze
the acquired data in more advanced level [20]. Thus, big data technology from
data collection to data analysis and visualization should follow to create value
improving the performance level of the SPL [21]. Especially, the complexity and
variety of data has reached to the peak that emphasized the importance of data
visualization empowering managers to derive valuable managerial insights for
process innovation and efficiency [22]. Therefore, data visualization has been
introduced in various industries studying data analytics in recent years.

[23] introduced Google Web Toolkit and HTML5 as dynamic user interfaces
developed to display real-time values. [24] proposed a framework of sensitivity
analysis and visualization tool for multidimensional nuclear simulation data for
multipurpose probabilistic risk assessment and uncertainty quantification frame-
work so called, Reactor Analysis and Virtual Control Environment (RAVEN).
[25] introduced LiveGantt for planning production schedule that helps users ex-
plore highly-concurrent large schedules from various perspectives. The study is
conducted to multiple engineering researchers and showed significant improve-
ment in scheduling due to its exploration capability in multiple-perspectives. [26]
introduced an interactive visual analytics system for monitoring and controlling manufacturing industries for operators to enhance performance level.

As big data analysis technology developed and many interactive visualization toolkits were released, various studies on big data visualization were performed. However, many studies have not yet been found in the field of production logistics.

## 3 Digital twin-based services and data visualization

This section presents digital services based on DT to material handling equipment and data visualization in SPL system. The DT-based services are developed and tested in a laboratory level environment that represents a simplified version of internal production logistics environment with AMR. The physical and virtual environments are connected through the middleware to interchange data between the two environments.

The data visualization application is developed with the data from an automotive manufacturing industry. The data is related to production logistics and we developed interactive heat map, frequency analysis map, and time-based trajectory map using open-source libraries.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** DT-based AMR monitoring and controlling application</figcaption></figure>
## 3.1 Digital twin-based monitoring and controlling application

The service involves interactions between AMRs between physical and virtual environment. Fig. 1 (a) is visualizing the safety radius around AMRs from both physical and virtual environment in DT. When the two AMRs interfere their safety distance and radius each other, the path will be adjusted to avoid collision between them. This informs that the AMR in the physical environment is capable of sensing and interacting with the virtual AMR in the virtual environment and re-calculate its direction.

This application can be used for the fleet management service in the design phase of the production logistics system. This service could be adopted as an optimization tool in decision making process of finding an optimal number of fleets in the production logistics system. Fig. 1 (b) provides results of scenarios with different number of fleets assigned to execute the same schedule. There are 6 tasks in the schedule, and each AMR in each scenario has assigned the tasks to complete the schedule. The routes are the optimized route regarding the interference between all AMRs. Thus, the real process is promised to follow the simulation result unless there's unexpected disturbance such as malfunction of the AMR, variability of process time of each task, human disturbance, etc. Then, the result is could be considered in decision making process of how many fleet to be purchased regarding the strategic goal of the system.

## 3.2 Data visualization in production logistics with open-source library

The DT-based service is focused in assisting decision making process in terms of providing different scenarios in time and cost effectively. The data visualization has conducted in this paper to get more insight from the historical location data of the forklift in the automotive manufacturing company. The forklifts deliver products from one station to another but they don't have structured schedule since the logistics tasks are supporting activities in the production system. In this paper open-source data visualization library is used to develop interactive dashboard. This library is connected to the database which is storing real-time data. Therefore it is easy to update dashboard with the real-time data.

In this case we can acquire different kinds of data from forklifts; name of forklift, location (longitude, latitude, coordinate, and predefined zone), and timestamp. Fig. 2 (a) shows a heat map of the frequency of forklifts' location in the factory using the location data (longitude, latitude) in the map. The red color represents high frequency of the location, whereas the green color represents low frequency of the location. Fig. 2 (b) shows the time-serial zone frequency graph that presents number of forklifts in each zone throughout timestamp using predefined zone and timestamp data. From this graph, managers can identify the real-time forklift distribution in the factory, in addition to the historical traffic congestion records in each zone. Lastly, Fig. 2 (c) is trajectory of each forklift through time using name of forklift, location data (longitude and latitude), and
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Data visualization in production logistics with open-source library</figcaption></figure>
timestamp. The color of trajectory is differed through time. Recent trajectory is
marked with red dots, while old records are marked with blue dots.

Data visualization can be utilized by various stakeholders related to internal
logistics operations. First, logistics operators such as forklift drivers can make
decisions based on quantitative results rather than their own experiences or in-
tuition, which can be used to improve inefficient driving habits. Also, from the
perspective of a logistics manager who establishes logistics plans, data visualiza-
tion can be used to compare whether planning and operation strategies have been
applied accurately. From a more macroscopic point of view, factory managers
can be usefully used for short-term, mid-to-long-term performance analysis.

4 Conclusions

Digital twins are already being developed in many places, but most are focused on product development, and it is difficult to find many cases that derive ex- amples of interaction between virtual and physical environments even for digital twins targeting production systems. In this study, several cases that can support decision-making through interaction between virtual and physical environments in a production logistics environment are presented. In addition, we developed an interactive dashboard that can provide insight to users based on location data of production logistics facilities stored for digital twin development. This can be used as useful input when analyzing the operating patterns of production logistics facilities and establishing future schedules.

Acknowledgement

The authors would like to acknowledge the support of Swedish Innovation Agency (VINNOVA). This study is part of the Cyber Physical Assembly and Logistics Systems in Global Supply Chains (C-PALs) project. This project is funded under SMART EUREKA CLUSTER on Advanced Manufacturing program.

References

1. Douaioui, K., Fri, M., Mabroukki, C. & Semma, E. The interaction between industry 4.0 and smart logistics: Concepts and perspectives. *2018 International Colloquium On Logistics And Supply Chain Management, LOGISTIQUA 2018.* 21266798, 128-132 (2018)

2. Klumpp, M., Hesenius, M., Meyer, O., Ruiner, C. & Gruhn, V. Production logistics and human-computer interaction—state-of-the-art, challenges and requirements for the future. *International Journal Of Advanced Manufacturing Technology.* **105**, 3691-3709 (2019)

3. Windt, K., Böse, F. & Philipp, T. Autonomy in production logistics: Identification, characterisation and application. *Robotics And Computer-Integrated Manufacturing.* **24**, 572-578 (2008)
4. Kaiblinger, A. & Woschank, M. State of the Art and Future Directions of Digital Twins for Production Logistics: A Systematic Literature Review. *Applied Sciences (Switzerland)*. **12** (2022)

5. Woschank, M., Rauch, E. & Zsifkovits, H. A review of further directions for artificial intelligence, machine learning, and deep learning in smart logistics. *Sustainability (Switzerland)*. **12** (2020)

6. Woschank, M., Kaiblinger, A. & Miklautsch, P. Digitalization in industrial logistics: Contemporary evidence and future directions. *Proceedings Of The International Conference On Industrial Engineering And Operations Management*. pp. 1322-1333 (2021)

7. VanDerHorn, E. & Mahadevan, S. Digital Twin: Generalization, characterization and implementation. *Decision Support Systems*. **145** (2021)

8. Jones, D., Snider, C., Nassehi, A., Yon, J. & Hicks, B. Characterising the Digital Twin: A systematic literature review. *CIRP Journal Of Manufacturing Science And Technology*. **29** pp. 36-52 (2020), https://doi.org/10.1016/j.cirpj.2020.02.002

9. Liu, M., Fang, S., Dong, H. & Xu, C. Review of digital twin about concepts, technologies, and industrial applications. *Journal Of Manufacturing Systems*. **58**, 346-361 (2021), https://doi.org/10.1016/j.jmsy.2020.06.017

10. Lu, Y., Liu, C., Wang, K., Huang, H. & Xu, X. Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues. *Robotics And Computer-Integrated Manufacturing*. **61**, 101837 (2020), https://doi.org/10.1016/j.rcim.2019.101837

11. Cimino, C., Negri, E. & Fumagalli, L. Review of digital twin applications in manufacturing. *Computers In Industry*. **113** pp. 103130 (2019), https://doi.org/10.1016/j.compind.2019.103130

12. Qi, Q., Tao, F., Zuo, Y. & Zhao, D. Digital Twin Service towards Smart Manufacturing. *Procedia CIRP*. **72** pp. 237-242 (2018), https://doi.org/10.1016/j.procir.2018.03.103

13. Caputo, F., Greco, A., Fera, M. & Macchiaroli, R. Digital twins to enhance the integration of ergonomics in the workplace design. *International Journal Of Industrial Ergonomics*. **71**, 20-31 (2019), https://doi.org/10.1016/j.ergon.2019.02.001

14. Martin, G., Marty, C., Bornoff, R., Poppe, A., Onushkin, G., Rencz, M. & Yu, J. Luminaire digital design flow with multi-domain digital twins of LEDs. *Energies*. **12** (2019)

15. Liu, Q., Zhang, H., Leng, J. & Chen, X. Digital twin-driven rapid individualised designing of automated flow-shop manufacturing system. *International Journal Of Production Research*. **57**, 3903-3919 (2019), https://doi.org/10.1080/00207543.2018.1471243

16. Angrish, A., Starly, B., Lee, Y. & Cohen, P. A flexible data schema and system architecture for the virtualization of manufacturing machines (VMM). *Journal Of Manufacturing Systems*. **45** pp. 236-247 (2017), http://dx.doi.org/10.1016/j.jmsy.2017.10.003

17. Liu, J., Zhou, H., Tian, G., Liu, X. & Jing, X. Digital twin-based process reuse and evaluation approach for smart process planning. *International Journal Of Advanced Manufacturing Technology*. **100**, 1619-1634 (2019)

18. Karanjkar, N., Joglekar, A., Mohanty, S., Prabhu, V., Raghunath, D. & Sundaresan, R. Digital twin for energy optimization in an SMT-PCB assembly line. *Proceedings - 2018 IEEE International Conference On Internet Of Things And Intelligence System, IOTAIS 2018*. pp. 85-89 (2019)
19. Papcun, P., Cabadaj, J., Kajati, E., Romero, D., Landryova, L., Vascak, J. & Zolotova, I. Augmented reality for humans-robots interaction in dynamic slotting "chaotic storage" smart warehouses. IFIP International Conference On Advances In Production Management Systems. pp. 633-641 (2019)

20. Witkowski, K. Internet of Things, Big Data, Industry 4.0 - Innovative Solutions in Logistics and Supply Chains Management. Procedia Engineering. 182 pp. 763-769 (2017), http://dx.doi.org/10.1016/j.proeng.2017.03.197

21. Zafarzadeh, M., Wiktorsson, M. & Baalsrud Hauge, J. A Systematic Review on Technologies for Data-Driven Production Logistics: Their Role from a Holistic and Value Creation Perspective. Logistics. 5 pp. 24 (2021)

22. Zhou, F., Lin, X., Liu, C., Zhao, Y., Xu, P., Ren, L., Xue, T. & Ren, L. A survey of visualization for smart manufacturing. Journal Of Visualization. 22, 419-435 (2019), https://doi.org/10.1007/s12650-018-0530-2

23. Vrba, P., Kadera, P., Jirkovský, V., Obitko, M. & Mařík, V. New trends of visualization in smart production control systems. Lecture Notes In Computer Science (including Subseries Lecture Notes In Artificial Intelligence And Lecture Notes In Bioinformatics). 6867 LNAI, 72-83 (2011)

24. Maljovec, D., Wang, B., Rosen, P., Alfonsi, A., Pastore, G., Rabiti, C. & Pascucci, V. Rethinking sensitivity analysis of nuclear simulations with topology. IEEE Pacific Visualization Symposium. 2016-May pp. 64-71 (2016)

25. Jo, J., Huh, J., Park, J., Kim, B. & Seo, J. LiveGantt: Interactively visualizing a large manufacturing schedule. IEEE Transactions On Visualization And Computer Graphics. 20, 2329-2338 (2014)

26. Wu, W., Zheng, Y., Chen, K., Wang, X. & Cao, N. A Visual Analytics Approach for Equipment Condition Monitoring in Smart Factories of Process Industry. IEEE Pacific Visualization Symposium.