# Extended Intelligence for Rapid Cognitive Reconfiguration

Anjela Mayer, Simon Otto, Jean-Rémy Chardonnet, Alexander Puchta, Jürgen Fleischer, Jivka Ovtcharova

► To cite this version:

Anjela Mayer, Simon Otto, Jean-Rémy Chardonnet, Alexander Puchta, Jürgen Fleischer, et al. Extended Intelligence for Rapid Cognitive Reconfiguration. Zeitschrift für wirtschaftlichen Fabrikbetrieb, 2025, 120 (s1), pp.177-180. 10.1515/zwf-2024-0159. hal-05022915

HAL Id: hal-05022915

https://hal.science/hal-05022915v1

Submitted on 7 Apr 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Extended Intelligence for Rapid Cognitive Reconfiguration

Anjela Mayer*,
Simon Otto,
Jean-Rémy Chardonnet,
Alexander Puchta,
Jürgen Fleischer and
Jivka Ovtcharova

With today's trend toward mass customization, fast and efficient reconfiguration of manufacturing systems is essential. New artificial intelligence and robotics developments play a key role in optimizing these processes. AI-driven reconfigurations are becoming increasingly effective because of real-time data from digital twins and intuitive interaction through extended reality. This article presents an approach that integrates digital twins and extended reality in a reconfigurable manufacturing use case, laying the foundation for AI-driven optimization. It highlights potential industrial benefits, such as increased flexibility and reduced downtime, and provides an outlook on future developments.

## Introduction and Background

In today's manufacturing landscape, the need for rapid reconfiguration is growing due to factors such as shorter product lifecycles and increased customization. Reconfigurable manufacturing systems offer a solution by allowing the structure and control of manufacturing systems to adapt to new requirements quickly. However, challenges such as time-consuming processes, errors, and lack of methodological support hinder the efficiency of these reconfigurations [1].

Recent advances in Artificial Intelligence (AI), Digital Twins (DTs), and eXtended Reality (XR) provide practical

tools to address these challenges [2, 3]. For example, DTs enable real-time simulation of physical systems, enabling predictive maintenance and reducing reconfiguration times by up to 58 percent [4]. XR technologies, when integrated with DTs, provide immersive interfaces for operator training, real-time feedback, and system control, improving the effectiveness of human-machine interaction.

## Challenges in Reconfigurable Manufacturing Systems

Reconfigurable manufacturing systems are designed to support the flexibility and

adaptability required in modern manufacturing. However, current reconfiguration practices face long downtimes, complex decision-making processes, and operator reliance on intuition, leading to inefficiencies. As manufacturing systems become more automated, they become more complicated, making traditional reconfiguration methods less effective. This complexity stems from managing large amounts of data, coordinating complex machine behaviors, and ensuring minimal downtime during reconfiguration. AI, DTs, and XR address these challenges by providing powerful tools to streamline processes: AI supports decision-making by analyzing data patterns and optimizing configurations; DTs simulate real-time changes, enabling predictive analysis and error prevention; and XR provides intuitive interfaces for operators to interact more efficiently with complex systems. Together, these technologies reduce human operators' cognitive and operational burden while significantly improving the reconfiguration process.

Despite the growing interest in each technology individually, few industrial projects fully integrate them. Most applications focus on individual optimizations, leaving a gap in realizing their combined potential.

<figure><img src="image_3.png" /></figure>

<footnote>* Corresponding Author</footnote>

<footnote>Anjela Mayer, M. Sc.; Institute for Information Management in Engineering (IMI) at the Karlsruhe Institute of Technology (KIT); Zirkel 2, 76131 Karlsruhe; Phone: +49 (0) 721 608 46619, E-Mail: anjela.mayer@kit.edu</footnote>

<footnote>Co-authors</footnote>

<footnote>Simon Otto, M. Sc.; wbk Institute of Production Science at the Karlsruhe Institute of Technology (KIT)
Prof. Jean-Rémy Chardonnet; LISPEN at Arts et Métiers (ENSAM) Institute of Technology in Chalon-sur-Saône (France)</footnote>

<footnote>Alexander Puchta, M. Sc.; wbk at the Karlsruhe Institute of Technology (KIT)</footnote>

<footnote>Prof. Dr.-Ing. Jürgen Fleischer; wbk at the Karlsruhe Institute of Technology (KIT)</footnote>

<footnote>Prof. Dr. Dr.-Ing. Dr. h. c. Jivka Ovtcharova; KIT & FZI in Karlsruhe</footnote>

<footnote>Note</footnote>

<footnote>This article is peer reviewed by the members of the ZWF Special Issue Advisory Board.</footnote>
## Extended Intelligence for Rapid Cognitive Reconfiguration

eXtended Intelligence for Rapid Cognitive Reconfiguration (XIRCON) aims to address gaps in current reconfiguration processes by integrating AI, DTs, and XR into a cohesive system, as shown in Figure 1. The architecture is divided into components that are part of the virtual world, components that are part of the physical world, and components that connect both worlds. This results in a cyber-physical system where changes in the physical world affect the virtual world and vice versa. First is the group of reconfigurable manufacturing systems, actuators and sensors, and DT.

This group mirrors the reconfigurable manufacturing system in the virtual world as DT, as shown in Figure 2. Changes made to the manufacturing system update the DT through the sensors. Changes made to the DT update the manufacturing system through the actuators. The second group comprises the human operator, the head-mounted display, and the XR. The human operator wears the head-mounted device while interacting with the manufacturing system. The head-mounted device shows the operator the physical world as a virtual world representation. The advantage of this is that information, such as a recommendation

for the next reconfiguration steps, can be visualized in the operator's field of view. The XR component provides helpful information and recommendations to the operator. It can also store and replay recordings of the operator's motion and voice through the head-mounted device. DT and XT exchange information such as possible trajectories. The trajectories can be displayed to the operator or used to interact with the manufacturing system. In addition to trajectories, all data generated by DT and XR can be fed into the AI component. The AI learns from the data and

uses it as the basis for recommendations and information that is rendered to the operator. It can also interact directly with the manufacturing system via the DT.

The XIRCON architecture results in a cyber-physical system where navigation in the physical and virtual worlds becomes one. This allows virtual components such as the AI to interact directly with the physical world. Physical tasks such as reconfiguration can also be virtually recorded and replayed. Applied to the reconfiguration use case, an expert operator can record while reconfiguring the manufacturing system. A novice operator can replay the recordings and get assistance from the AI to learn the reconfiguration task. This is shown in Figure 3. The left side shows virtual hands performing a reconfiguration task as visual feedback for the operator. The right side shows the recorded task with the operator visualized as an avatar.

<figure><img src="image_4.png" /><figcaption>Figure 1. System Architecture - Integrating AI, DT, and XR to a holistic system</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 2. Implementation of the Value Stream Kinematics vision (left) and its DT (right)</figcaption></figure>

## Implementation of the System

A robotic manufacturing system based on Value Stream Kinematics is implemented at the Karlsruhe Research Factory, shown on the left in Figure 2. It shows four industrial robots and a coupling unit stored in the tool magazine. Such a complex system is error-prone and time-consuming to maintain and reconfigure. Combining DT, shown on the right of Figure 2, with XR and AI could reduce costs and improve system understanding.

## Robotic Manufacturing: Value Stream Kinematics

One example of a reconfigurable manufacturing system is a manufacturing system based on the Value Stream Kinematic vision. Value Stream Kinematics represents a different approach to the challenges of mass customization. Universal kinematics, such as jointed-arm robots combined with intelligent software, implement the entire value stream of a product. All manufacturing steps become feasible using
<figure><img src="image_6.png" /><figcaption>Figure 3. View from the XR headset: Human operator explaining the pneumatic quality assurance step (left) and virtual replica of the human operator demonstrating the same step (right)</figcaption></figure>

universal kinematics with a standard set of tools. The intelligent software plans and executes all processes autonomously, from the customer order to the necessary manufacturing steps. The result is highly flexible yet productive manufacturing [5, 6].

Three layers show the dimension of the Value Stream Kinematics: the machine, production planning, and human interaction layers. The machine layer challenges how to perform all manufacturing steps using only joint-arm robots as universal kinematic representants. Machine tools perform most manufacturing steps to achieve high precision. Joint-arm robots are inherently less rigid due to their open-chain kinematics. And the loss of stiffness results in lower precision. Stiffness must be increased to perform typical machine tool tasks such as milling. This is achieved by coupling the robots, which results in closed-loop kinematics. For pick-and-place tasks, each robot can operate independently or must cooperate with another. This variety of possible configurations creates challenges for the next layer, production planning. Production planning analyzes product specifications from customer orders. It makes a task plan, decides which robots will interact and which will be coupled, selects the right tools, executes the manufacturing steps, and sends feedback to the customer. The final layer is the human interaction with the Value Stream Kinematics. Even though the system works autonomously, human work is still part of it. In addition to the customer sending his or-

der, the machines and software must be maintained. Such a complex machine is error-prone and difficult to learn [5, 6].

## Key Results and Approaches

The XIRCON project has made significant progress in developing an architecture that integrates AI, DT, and XR to optimize reconfiguration processes in manufacturing. Key results achieved so far include:

* Integrated system architecture: The project has established a basic software architecture that integrates these components and enables real-time feedback between the XR interface, DT, and AI, as shown in Figure 1. This architecture lays the groundwork for future enhancements, such as fully automated reconfiguration processes and further AI-driven operator assistance and reconfiguration optimization.

* XR component for human operations: The XR interface shown in Figure 3. has been developed. It captures and replicates human operations within the manufacturing cell. This provides a detailed record of human activities during reconfiguration that can be analyzed and used for future improvements. By capturing these operations, the system also helps preserve tacit operator knowledge, which is critical for training and ensuring consistency in reconfiguration tasks.

* DT component for the robotic manufacturing system located at Karlsruhe Research Factory: The project has devel-

oped a DT of the robotic manufacturing cell. Figure 2 shows a kinematic model of the robots that can partially read and interpret robot programs. In particular, the DT can extract trajectory information from the robot code, allowing real-time simulation and validation of reconfiguration steps. This helps reduce the time and errors associated with manual robot programming.

These advances demonstrate how the integration of XR, DT, and AI technologies can streamline reconfiguration processes, reduce errors, and improve the overall adaptability of manufacturing systems.

## Benefits for the Industry

Integrating AI, DT, and XR brings significant benefits to industrial applications. These technologies can reduce reconfiguration time by more than 50 percent, reduce errors during the reconfiguration process, and improve operator interaction with increasingly complex systems [4]. They also enable predictive maintenance, ensuring systems operate at peak efficiency with minimal downtime [1].

In practice, manufacturers can expect:

* Shorter reconfiguration cycles, enabling faster response to market changes.

* Improved human-machine collaboration, where operators are supported by real-time feedback from DTs and intuitive XR interfaces.

* More adaptive systems that can be reconfigured without significant manual intervention.

In particular, AI plays a critical role in achieving these benefits by:

* Optimizing decision making: AI analyzes data from DTs to recommend the most efficient reconfiguration strategies, improving the accuracy of decisions and reducing downtime during system changes.

* Learning from operator input: AI models in XIRCON are designed to elicit operator knowledge during reconfiguration and use this data to train machine learning models that suggest optimal reconfiguration actions for future tasks.

* Enable predictive maintenance: AI algorithms continuously monitor system performance and predict when mainte-
nance or reconfiguration is needed, en-
suring that systems operate at peak ef-
ficiency with minimal downtime.

Reduce human error: By automating reconfiguration tasks and learning from previous configurations, AI minimizes the likelihood of human error and ensures a more reliable and resilient system.

Synergies
between AI, XR, and DT

The combined use of AI, DT, and XR al-
lows for more synergistic and adaptive re-
configuration systems. AI and DT already
enable predictive maintenance and re-
al-time monitoring, while XR provides an
immersive interface that facilitates natu-
ral human-machine interaction. The next
step in this evolution is fully integrating
these technologies to create systems that
can autonomously optimize reconfigura-
tion while maintaining human oversight.

Future Work and Outlook

As AI, DT, and XR technologies continue
to evolve, future work will focus on im-
proving their interoperability and scal-
ability. Emerging trends include the de-
velopment of bidirectional DTs that not
only mirror but also control physical sys-
tems and automated reconfiguration
driven by AI algorithms.

In summary, integrating AI, DT, and
XR will redefine how manufacturers ap-
proach system reconfiguration. These
technologies will play a critical role in the
next generation of reconfigurable manu-
facturing systems by reducing downtime,
increasing flexibility, and improving de-
cision-making.

Literature

1. Marks, P.; Hoang, X. L.; Weyrich, M.; Fay, A.: A Systematic Approach for Supporting the Adaptation Process of Discrete Manufacturing Machines. Research in Engineering Design 29 (2018), pp. 621–641
DOI:10.1007/s00163-018-0296-5

2. Leng, J.; Shen, W.; Wang, D. et al.: Digital Twins-based Smart Manufacturing System Design in Industry 4.0: a Review. Journal of Manufacturing Systems 60 (2021) 9, pp. 119–137
DOI:10.1016/j.jmsy.2021.05.011

3. Napoleone, A.; Negri, E.; Macchi, M.; Pozzetti, A.: How the Technologies Underlying Cyber-physical Systems Support the Reconfigurability Capability in Manufacturing: a Literature Review. International Journal of Production Research 61 (2023), pp. 3122–3144
DOI:10.1080/00207543.2022.2074323

4. Ashtari Talkhestani, B.; Weyrich, M.: Digital Twin of Manufacturing Systems: a Case Study on Increasing the Efficiency of Reconfiguration. at - Automatisierungs-technik 68 (2020) 6, pp. 435-444
DOI:10.1515/auto-2020-0003

5. Mühlbeier, E.; Oexle, F.; Gönnheimer, P.; Fleischer, J.: Wertstromkinematik – Produktionssysteme neu gedacht: Interdisziplinäres Forscherteam arbeitet an der Produktionstechnik der Zukunft (Teil 1). Zeitschrift für wirtschaftlichen Fabrikbetrieb ZWF 116 (2021) 11, pp. 847–851
DOI:10.1515/zwf-2021-0179

6. Kimmig, A.; Schöck, M.; Mühlbeier, E. et al.: Wertstromkinematik – Produktionssysteme neu gedacht: Interdisziplinäres Forscherteam arbeitet an der Produktionstechnik der Zukunft (Teil 2). Zeitschrift für wirtschaftlichen Fabrikbetrieb ZWF 116 (2021) 12, pp. 935–939
DOI:10.1515/zwf-2021-0207

Authors

Anjela Mayer, M.Sc., born in 1989, is a research associate at the Institute for Information Management in Engineering (IMI) at the Karlsruhe Institute of Technology (KIT) and is pursuing her Ph. D. in the topic of collaboration in engineering using immersive technologies.

Simon Otto, M. Sc., born in 1989, is a research associate at the wkb Institute of Production Science at the Karlsruhe Institute of Technology (KIT) since 2024.

Prof. Jean-Rémy Chardonnet, born in 1983, is the head of the Arts et Métiers Institute (ENSAM) in Chalon-sur-Saône (France) and the deputy director of the Laboratory for Physical and Digital System Engineering (LISPEN, ENSAM), specializing in immersive technologies.

Alexander Puchta, M. Sc., born in 1996, is a research associate at the wkb Institute of Production Science at the Karlsruhe Institute of Technology (KIT) since 2020 and chief engineer of the Intelligent Machines and Components group since 2023.

Prof. Dr.-Ing. Jürgen Fleischer, born in 1961, is the director of the Machines, Equipment and Process Automation chair at the Institute of Production Science (wbk) at the Karlsruhe Institute of Technology (KIT).

Prof. Dr.-Ing. Jürgen Fleischer, born in 1961, is
the director of the Machines, Equipment and Pro-
cess Automation chair at the wkb Institute of Pro-

duction Science at the Karlsruhe Institute of
Technology (KIT).

Prof. Dr. Dr.-Ing. Dr. h.c. Jivka Ovtcharova, born in 1957, is the emeritus head of the Institute for Information Management in Engineering (IMI) at the Karlsruhe Institute of Technology (KIT) and director in the field of Intelligent Systems and Production Engineering at the Research Center for Information Technology (FZI) in Karlsruhe.

Abstract

Erweiterte Intelligenz für schnelle kognitive
Umrüstung. Durch die heutige Entwicklung
zur Losgröße 1 gewinnt die Rüstzeit in der
Produktion ein immer höheres ökonomisches
Gewicht. Neue Entwicklungen der Künstlichen
Intelligenz und Robotik spielen eine Schlüssel-
rolle bei der Optimierung dieser Gegebenheit.
Unterstützt durch Echtzeitdaten von Digitalen
Zwillingen und intuitiver Interaktion mittels
eXtended Reality werden Kl-gesteuerte Umrüs-
tungen immer effizienter. Dieser Artikel stellt
einen Ansatz vor, der Digitale Zwillinge und
eXtended Reality in einem wandlungsfähigen
Fertigungsszenario integriert, eine Grundlage
für Kl-gestützte Optimierung schafft und die
potenziellen Vorteile für die Industrie sowie
einen Ausblick auf zukünftige Entwicklungen
aufzeigt.

Acknowledgements

XIRCON is an international research project between the Karlsruhe Institute of Technology (KIT) and the Arts et Métiers Institute (ENSAM). The authors would like to thank the Ministry of Science, Research and Arts of the Federal State of Baden-Württemberg for the financial support of the projects within the Innovation-Campus Future Mobility (ICM).

Keywords

Artificial Intelligence, eXtended Reality,
Digital Twins, Reconfigurable Manufacturing
Systems, Human-in-the-Loop, Flexible
Manufacturing

Schlüsselwörter

Künstliche Intelligenz, eXtended Reality, Digitale
Zwillinge, Rekonfigurierbare Fertigungssysteme,
Human-in-the-Loop, Flexible Fertigung

Bibliography

DOI:10.1515/zwf-2024-0159
ZWF 120 (2025) Special Issue; page 177 - 180
Open Access. © 2025 bei den Autoren,
publiert von De Gruyter.
Dieses Werk ist lizensiert unter der Creative
Commons Namensnennung 4.0 International
Lizenz.
ISSN 0947-0085 · e-ISSN 2511-0896