# Cognition in digital twins for cyber-physical systems and humans: Where and why?

Yannick Naudet, Jana Al Haj Ali, Ben Gaffinet, Hervé Panetto

► To cite this version:

Yannick Naudet, Jana Al Haj Ali, Ben Gaffinet, Hervé Panetto. Cognition in digital twins for cyber-physical systems and humans: Where and why?. 18th IFAC/IFIP International Workshop on Enterprise Integration, Interoperability and Networking, EI2N 2024, Nov 2024, Porto, Spain. pp.399-412, 10.1007/978-3-031-80775-6_27 . hal-04802334

HAL Id: hal-04802334

https://hal.science/hal-04802334v1

Submitted on 25 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Cognition in Digital Twins for Cyber-Physical Systems and Humans: where and why?

Yannick Naudet<fnref n="1" />[0000-0001-8570-9200], Jana Al Haj Ali<fnref n="2" />[0009-0006-8220-602X], Ben Gaffinet<fnref n="1" />,<fnref n="2" />[0009-0005-4641-5978], and Hervé Panetto<fnref n="2" />[0000-0002-5537-2261]

**Abstract.** In modern Cyber-Physical Enterprises, the place of cognition to build more flexible, autonomous, self-adaptive and ultimately more intelligent systems remains unclear. However it could be an essential element to ensure interoperability, as highlighted in recent works on cognitive interoperability. Assuming an environment where cyber-physical systems and humans are coupled with their individual digital twins, we discuss where cognition can exist in the couple <Entity-Digital Twin> and how Cognitive Digital Twin and Cyber-Physical Systems can emerge. This is supported by a proposition for a high-level architecture for Entity- Digital Twin coupling. The place of Cognitive Architectures as a special model of cognition is discussed. Exploring the integration with the ISO23247 Digital Twin framework for manufacturing, we also highlight its lacks to include cognition and humans.

**Keywords:** Cognitive Cyber-Physical System · Cognitive Digital Twin · Cognitive Architecture · Cyber-Physical Enterprise · Digital Twin Framework.

## 1 Introduction

Cognitive systems are becoming a new trend in Industry 4.0 research. With the human-centricity pushed by Industry 5.0 and advances in Artificial Intelligence (AI), it makes more and more sense to implement industrial systems that can better support human workers by better understanding them, making human-machine collaboration possible. Today, together with AI and Internet of Things (IOT), Cyber-Physical Systems (CPS) and Digital Twins (DT) are two pillars of new industrial systems, which when they reach enough degrees of flexibility and self-reconfiguration can contribute to the so-called Cyber-Physical Enterprises (CPE) [21].

Acting components in a CPE can be considered as entities, technological or human, operating in the physical space of a shop floor. These are systems with a capability to operate or produce. Technological systems such as, e.g., machines, robots, sensors, actuators or computing systems can all be considered as CPS. Assuming each entity (CPS or Human) in a CPE is coupled with its own DT, cognition can be found at different places, naturally present or artificially

<footnote n="1">¹ Luxembourg Institute of Science and Technology (LIST), Luxembourg</footnote>

<footnote n="2">² University of Lorraine, CNRS, CRAN, France</footnote>
programmed, with different purposes. If human entities were already cognitive, cognitive versions of CPS and DT have appeared only recently: the Cognitive CPS (CCPS) and Cognitive DT (CDT). Cognition in these systems has so far been considered separately in two research tracks, usually to give them learning, adaptation and decision-making capabilities. In particular for the DT, the question of the purpose of introducing cognition and which kind of cognition has not been addressed.

This paper addresses the case where DTs replicate both non-cognitive and cognitive systems and where we seek for a better understanding between humans and technological entities (all considered as CPS). While different approaches can be taken to implement cognition, we make a special focus on cognitive architectures, postulating that this kind of cognitive model will facilitate human-CPS understanding. Our contributions are the following:

- Clarifying cognition in DT and CPS for flexible enterprises such as CPE, giving guidance on where it should be implemented in the DT, and for which purpose. We highlight the importance of distinguishing between cognitive and non-cognitive entities when implementing their DT. Cognition is then a separate function of a DT, distinct from its main emulation function, which can have two objectives: supporting the other DT functions, or bringing cognition to the coupled entity.

- Providing the rules specifying when a DT is cognitive or not, or a HDT (Human Digital Twin), depending on the twinned entity and the existing DT functions, and the rule when a couple <CPS,CDT> becomes a CCPS.

- Clarifying in which case a cognitive architecture, as a computational model of the human mind, should be used for implementing cognition [2].

- Specifying an architecture for Entity-CDT couples, focused on the main functions of a CDT: emulation, simulation and cognition.

- Opening a discussion on possible extensions to the ISO 23247 DT framework for manufacturing, to better include humans and cognition.

The next subsection briefly recalls definitions of CDT and CCPS and how they currently implement cognition. The reminder of the paper is as follows. Section 2 describes the architecture we propose for a Entity-CDT system, discussing the emulation, cognition and simulation functions of a CDT as well as the technologies for implementing them. Section 3 discuss the place and kind of cognition in DT and CPS or Humans, answering the main questions about the existence, need and purpose of a CDT or a CCPS, giving the rules determining when a DT becomes a CDT, when we have a HDT and when a CCPS is created from a CPS thanks to its CDT. Then, section 4 discuss the place of cognitive architectures. Finally, section 5 discusses modifications of the ISO 23247 DT framework for manufacturing to integrate cognition and human aspects, and the last section gives our conclusions and perspectives.

## 1.1 Cognitive Digital Twins and Cyber-Physical Systems

Digital Twins have revolutionised many industrial practices by enabling real-time monitoring, predictive maintenance and in-depth analysis of physical systems
based on collected data [18]. However, as industry moves towards greater collab-
oration between human and machine, some of the limitations of traditional DTs
have become apparent, particularly in scenarios requiring the synthesis of real-
time data from multiple sensors to make immediate, context-sensitive decisions
[6]. Faced with these challenges, the concept of the CDT has emerged as a critical
development. First introduced in 2016, the CDT is defined as “*a digital represen-
tation encompassing intelligent capabilities that span across all lifecycle phases
of its physical counterpart*” [1]. This new generation of DTs is distinguished by
its ability to learn and evolve dynamically, offering a more adapted interaction
between physical systems and their digital replicas. CDT goes beyond simple
replication of physical states to engage complex cognitive processes, enabling
proactive decision-making and more intuitive human interaction [20]. CDTs of-
fer not just progressive improvements over DTs, but a transformative approach
to harmonising automation and human-centricity in the pursuit of sustainability
and resilience [11]. Therefore, a “*CDT is a DT and a cognitive system. As a DT
it emulates a physical system that can be or not itself cognitive. As a cognitive
system it possesses cognitive functions, bringing it in particular the ability to se-
mantically model, process and interpret information autonomously and actively
learning from its interactions*” [3].

On the other side, a CCPS is a relatively new concept that also includes
complex cognitive processes for effective decision-making and control. CCPSs
are entities capable of autonomy, adaptability and environmental awareness.
These systems are designed to operate without human intervention, adapt dy-
namically to changes in their environment using their cognitive capabilities [5]
and have a deep understanding of their operational context [8]. In human-robot
collaboration, robots (as CCPS) focus on building mutual trust and, above all,
on operating without human intervention. They are able to adapt their actions
based on sensory information and uncertainty management [13]. Summarising, a
CCPS can be defined as “*a CPS and a cognitive system. As such it as cognitive
functions, with abilities enabling not only to interact with its environment, but
also to understand and learn from it, to make autonomous decisions and adapt
dynamically to the context*” [3].

Cognition in these systems has been approached from different angles [3]. If we understand it the way it is defined for humans, the foundational definition provided by Neisser should be enough to understand what it should be: “*Cognition is all the processes by which the sensory input is transformed, reduced, elaborated, stored, recovered, and used*” [16]. This refers indeed to the human mental processes operating information processing in our brain, from sensory inputs to action output. Those processes generate a set of cognitive functions from which we can identify if a system such as a CDT or a CCPS is cognitive or not: *perception, attention, memory, problem-solving, reasoning, memory and learning*.

Cognitive processes are implemented in different ways, supported in par-
ticular by Artificial Intelligence algorithms. A first example are Large Lan-
guage Models (LLMs), which can be used to extract knowledge directly from
human language, and eventually understanding, learning and interacting with humans [9]. Another is Deep Learning, which applied to computer vision can be used to interpret visual data, significantly improving a system's ability to process and respond to its environment [4]. A particular approach consists in using cognitive architectures, which are models of the human mind, structuring the cognitive processes and information processing the way it is in the human brain, resulting from an effort to build a unified theory of cognition [17]. These models aim to describe the fixed mechanisms and underlying structures that support human cognition, and are designed to model human performance in situations involving multiple tasks and modalities [12]. They have not yet been exploited for CDT or CCPS, but by their nature they constitute a very relevant candidate technology for implementing cognition [15], which we will discuss.

## 2 Entity - Cognitive Digital Twin Architecture

Let *CPE* be a cyber-physical enterprise. Taking a systemic perspective, we can define a CPE as a set of interacting physical entities: $CPE = \{ENT, INT\}$, with $ENT = \{CELLS, CPS, HUM, RES, SENS\}$ comprising respectively the sets in *CPE* of production cells, CPSs, humans, resources (ex. object to produce) and sensors, and *INT* being the set of interactions among them. In this view, each entity can potentially have a dedicated DT. Entities other than humans are usually not (yet) cognitive but could benefit from being coupled to a CDT that might bring cognition. However, having cognitive capabilities tends to transform entities into cognitive agents. This is contributed by cognitive functions such as learning, problem-solving and decision-making, leading to adaptation capabilities, which would not work without abilities for sensing and acting. This leads us to consider the CPE as a set of interacting agents, forming a complex adaptive system [21].

In this paper, we adopt an agent perspective for entities, where each entity can sense its environment, analyse related observations and act in their environment accordingly. We focus more particularly on the two main agents of a CPE, namely CPS and humans, for which cognition would be needed to reach a seamless collaboration [15]. Entities are each coupled with an individual DT receiving observation data from their environment. The DT of a human is referred as a HDT [14]. The environment starts here in the production cell to which entities belong up to anywhere useful data can be retrieved, and includes the digital environment of the DTs. Data is generated by any data source. In particular sensors can be part of the environment and part of an entity, allowing a direct sensing of the environment.

### 2.1 Architecture overview

The coupling between an entity and its DT and their respective processing structure can be represented by the diagram in Figure 1. Since we focus on bringing
<figure><img src="image_1.png" /><figcaption>**Fig. 1.** Entity-CDT Architecture. Entity here can be a CPS, CCPS or Human.</figcaption></figure>

cognition to the entity-DT system, the digital twin is in fact a CDT, for different reasons we will explain in section 3.

An *Entity* (here as an agent) has embedded sensors (senses) allowing to observe its environment (*obs*), a controller which processes the observations and transforms them in commands (*cmd*) for its actuators, resulting in actions (*act*) that change the environment. Actions of entities are their response to stimuli (*sti*) from the environment. In turn, those actions also can trigger stimuli for other entities.

The DT can directly receive data (*data*<sub>e</sub>) from its twinned entity. It can be observations from its sensors, notifications of commands sent to actuators or information about internal state. For example, a mobile robot equipped with a LIDAR could inform about its own location or surrounding objects, inform it is about to move to a new location, and that its battery is almost empty. The CDT returns feedback (*fbk*) to the entity, which can take different forms depending on the entity. It can be information (reports), request for changes in the form of direct commands, instructions or recommendations, adaptations or functions to integrate, or even direct actuation. There might be an intermediary between the DT output and the feedback received by the entity, represented by the *Tf* function, which will transform this output into data actionable for the entity. For example, a set of possible solutions to some issue identified by the DT might be actionable only if there is a decision made and the entity is told what to do.

The structure of a CDT is represented here by the three functions we think essential. The *Emulation function* implements the main role of a DT, without which it cannot be called a digital twin. Its role is to estimate the entity response *act* to a stimulus *sti*, from data gathered from the environment *data* and the entity (*data*<sub>e</sub>). The *Simulation function* gives a DT the power to do predictions and go further by computing solutions to possible problems or situations before they occur. Last, the *Cognition function* brings cognitive capabilities to the entity through its DT or to the DT itself.

The *Environment* is seen through the sources providing *data* to the CDT: sensors, other DTs and entities than the one of interest, and any other kind of source bringing useful data about the entity of interest (e.g. messages, observa-
tions, documents, human-originated feedback...). Data received by a CDT from its environment allows the CDT to build its models about the entity it twins within its emulation function. The environment also generates the stimuli (sti) following which entities react, which can be anything observed or received from the entities in the environment or the environment itself.

## 2.2 Emulation function

The DT Emulation function computes an estimation of the entity response (action) $\hat{act}$ to some stimulus sti. This input can be completed by $data_e$ provided by the entity, which can in particular comprise the entity's own observations $obs$ and notification of sent commands cmd as explained before. For a CPS, we can assume observations from its sensors can be transmitted automatically to the DT. In the case of a human, the transmission would be indirect through an HMI, subject to the specific interpretation of the human: it cannot be raw data from human senses, except through a dedicated external sensor (e.g. ECG, EEG, Eye tracker, etc...). The DT retrieve then any useful data coming from its environment or the one of the entity: from sensors and other entities in the entity's environment, from other DTs and from any other source of useful information. From this input, the DT emulation function estimates the entity state and resulting action ($\hat{act}$) in response to the stimulus sti, using the entity models built from acquired data, apriori known or learnt.

During learning phases, the DT emulation function builds a model of the entity seeking to minimise the error $||\hat{act} - act||$ in a supervised way (Supervised ML). This model also includes any other apriori available model, in particular descriptive of the entity with a Knowledge Representation, a mathematical / physical model, or even a program. A DT of a CPS can benefit from using the CPS implemented program when it is available.

## 2.3 Cognition function

The DT Cognition function takes as input $\hat{act}$, the estimated response of the entity from the Emulation function, but also the data from the environment ($data$) or the entity ($data_e$), to provide any useful cognitive processing. Any form of cognition can be implemented here for whatever objective useful for the entity. One in particular can be to identify any deviation between the observed reality and the state and action estimated by the DT for the entity, and reasoning on this to decide on which solution to apply. The cognition function can be implemented for two objectives: (1) bringing cognition to an entity that does not have cognitive capabilities (e.g. a normal CPS); (2) adding cognitive capabilities to the DT to support its functions.

In case (1), a bidirectional communication link is established between the entity's controller and the DT Cognition function, making it a cognitive controller. This is a strong coupling where the DT cognition function is operated by the entity as a distant service.
The case (2) corresponds to the most common approach for CDT in the literature. Cognition can support the Simulation function, to drive the simulation with cognitive processes. It can also support any kind of analytics task performed by the DT, in additional utility functions the DT could provide, not represented here, like the Analytic Service of the ISO23247 DT framework [10]. The coupling between the entity and the DT in this case is weak. The DT exploits cognition to get better results in its functions, thus providing a better feedback to the entity, but the latter cannot get cognitive capabilities since it cannot operate the DT Cognition function itself.

## 2.4 Simulation function

The simulation function of a DT require modelling the complete environment of interest (a production cell of a CPE for example) including the multiple interacting entities and influential variables. A snapshot of such environment might be enough for a predictive model, but more might be required to simulate situations and finding ways to tackle it when the entities have not initially be programmed to do so and do not have the necessary capabilities. The approach where each entity is coupled with its own DT allows conveniently to build an entity-based simulation environment for this. As illustrated in Figure 2, the input is a situation to which entities have to react, while reaching their objectives. Agent models like BDI (Belief, Desire, Intention) can be used. In our case we experiment Deep Reinforcement Learning (DRL), with the hypothesis it is more suitable to find solutions exploiting a more complete behaviour of agents driven by the DT emulation model instead of a parallel restricted model like BDI.

<figure><img src="image_2.png" /><figcaption>**Fig. 2.** DT simulation environment, in the case of a CPS and a human H, with their respective digital twins CDT and HDT.</figcaption></figure>
The DT Simulation function is activated when a situation containing elements leading to a non-programmed reaction occurs. This can be triggered during operations or in parallel, to anticipate possible situations that are known but have not yet occurred. A simulation engine operates the simulation environment where all CDT and HDT of entities impacted by the situation are involved. In the example, $CDT_1$ is detailed, showing the common internal structure of each CDT and HDT. The simulation starts with the situation input. The engine creates a stimulus (simulated stimulus) for each of the DTs. DTs operate first their emulation model in execution mode ($CDT_1Emul$), generating the estimated response to the stimulus. The resulting $act$ is then processed by the DT simulation function ($CDT_1Sim$) that contains the adaptation logics of the DT, driven by the simulation engine. The output of this function at each iteration step within the simulation is the action resulting from the adaptation of the emulated action ($act\_CDT_1^*$). The simulation engine implements the global simulation logics, with constraints and objectives to be optimized for DTs to find together a solution to the inputted situation.

## 2.5 Technologies for Cognitive Digital Twin functions

The fact that an entity is cognitive or not influences the choice for the functions implementation. In our research, we identified three main approaches that seem suitable, among others: supervised Machine Learning (ML) potentially combined with Knowledge Graphs for the semantic labelling and reasoning (KRR), for the emulation function; Deep Reinforcement Learning (DRL) plus potentially KG for the simulation function; and an alternative targeting especially cognition modelling: Cognitive Architectures (CA).

## 3 (C)DT of a (C)CPS, the different cases

The Entity-CDT architecture (Fig. 1) brings a model of a DT valid for any kind of active entity (i.e. agents). A simplified model might work also with passive entities, the difference being that these have not the same internal structure nor react to some stimulus. Still the DT can be built from external observations (data gathered) and models. In the following, we bring answers to the questions listed after, focusing on the twinning of two important entities: Cyber-Physical Systems (CPS) and their cognitive version CCPS, and Humans. We discuss the three first ones in this section, while the last one is discussed in section 4:

- When does a classical DT become a CDT?

- When is a CDT needed?

- When does a CDT bring cognition to a non-cognitive entity and does it make it a cognitive entity?

- When does it make sense to use a Cognitive Architecture in one of the functions of a CDT?
Within the three functions of the DT structure in Figure 1, only Emulation is mandatory to build a DT. A DT with its emulation function can emulate a CPS, or a CCPS or a human if we assume its implementation handles the cognitive actions. In this latter case, the complement of a symbolic AI approach might be needed, especially if cognitive processes generating data should be identified. This is a classical DT, and its emulation function can be implemented with supervised ML (black-box model), potentially completed by knowledge-based or physical models (grey-box model). When it emulates a cognitive entity, it models the cognitive capabilities of this entity but does not possess itself the related processes. It is then the DT of a cognitive entity but should probably not be qualified as CDT. However, since the DT learns from the data it receives, it could maybe be already qualified as cognitive, because learning is a recognised cognitive function. This opens up a lot of discussions, and if this is true, then all DT are necessarily CDT.

A DT with a Cognition function is by definition a CDT. As detailed in section 2.3, the cognition capabilities can be used for the DT, or brought to the twinned entity. A DT with a simulation function is also a CDT, because simulation involves a learning process and decision-making, which are both cognitive functions. Both kinds can emulate a CPS or a CCPS, or a human. When the twinned entity is a CPS, the system formed by the couple <CPS,CDT> becomes a CCPS: we can say that the coupling of a CPS with a CDT creates the emergence of a CCPS. With the Cognition function, the CDT has cognitive capabilities which it can bring to the CPS when there is a strong coupling. With a Simulation function, the CDT brings to the CPS a mean to make predictions and adapt to not initially programmed situations.

A CCPS or a Human have by definition cognitive functions and their DT is therefore necessarily cognitive: a CCPS, or a HDT. Their emulation function has to include a cognitive process in order to be able to reproduce the cognitive behaviours of the twinned entity. Cognition happens both in the CCPS and CDT, resp. H and HDT. The DT emulation function replicates the CCPS/H's cognition, while the simulation and cognition functions if they exist act as digital second brains. The first, seeking and storing solutions to possible situations, should result in a speedup of the CCPS/H adaptation when these situations actually occur. The latter provides additional cognitive capabilities that can be used as a complement and in parallel of the entity's existing ones.

To conclude, we summarise the conditions leading to a DT, a CDT or a HDT as follows, giving the answers to our first three questions. A CPS with a DT having an emulation function is a simple DT. A twinned entity that is a CCPS or a Human requires a CDT/HDT, and it is also a CDT/HDT when the cognition function or a simulation functions are present. Then a CPS coupled with a CDT forms a CCPS. This can be formalised by the simplified logical
formula:

$$ \mathit{CPS} \wedge \mathit{DT.Emul} \wedge \neg \mathit{DT.Cog} \wedge \neg \mathit{DT.Sim} \implies \mathit{DT} \quad (1) $$

$$ (\mathit{CCPS} \lor \mathit{CPS} \land (\mathit{DT.Cog} \lor \mathit{DT.Sim})) \land \mathit{DT.Emul} \implies \mathit{CDT} \quad (2) $$

$$ H \wedge \mathit{DT.Emul} \implies HDT \quad (3) $$

$$ (\mathit{CPS} \lor \mathit{CCPS}) \wedge \mathit{DT.Emul} \wedge (\mathit{DT.Cog} \lor \mathit{DT.Sim}) \\ \implies \langle \mathit{CPS}, \mathit{CDT} \rangle \text{ is a } \mathit{CCPS} \quad (4) $$

# 4 The place of Cognitive Architectures in a Digital Twin

When does it make sense to use a cognitive architecture for the DT emulation function? Only when the CCPS has a human-like cognition or if the goal is to build a human-like CCPS from a CPS. The first case involves a cognitive architecture in the CDT emulation function, and the second case in its cognition function. An alternative for this second case is to implement a cognitive architecture covering both cognition and simulation functions. In this case simulation or prediction become the result of a human-like cognitive process, like other cognitive functions provided by the CDT cognition function.

A cognitive architecture being a model of the human mind, it makes sense to use it when wanting to model human-like cognition. This means CCPS for which a cognitive architecture is used in their CDT emulation function should implement a cognition process close to the ones humans have. Of course, this holds when the entity coupled by the DT is a human: the HDT emulation function should logically be implemented as a cognitive architecture or another algorithm emulating the human mind process.

The second option is when wanting to bring human-like cognition to a (C)CPS (a CPS or a CCPS) which does not have it. This can be realised through the couple <(C)CPS, HCDT>, where HCDT means "Human-like CDT", which implements a cognition model mimicking human cognition. Similarly we can call HCCPS a "Human-like CCPS". In this case, the cognitive architecture should be used for the cognition or simulation functions of the DT, since the emulation function only replicates what the (C)CPS is capable of doing.

In Cognitive Computing and in the latest definition of CCPS, a cognitive system exhibits human-like behaviour or has human-like interactions. This is to be distinguished from having human-like cognition, where the cognitive process mimicking the human one, leads to human-like behaviour. A CCPS can have human-like behaviour without being a HCCPS. LLM constitutes a good example, being able to generate outputs that can probably be understood as human-like.

# 5 Updating ISO23247 for cognitive entities

The recent ISO 23247 series proposes a generic framework for DT in manufacturing to standardise the way digital twins are defined, structured, modeled and used in this area. The framework is based on four layers, strangely called
"Entity" for three of them, where the first one comprise items observed in the shop floor for which DT will be built (named the observable manufacturing elements (OME)), and the fourth level comprises the applications using the DT (named user entities). The core of the DT elements is modelled in the middle layers, with one layer for data collection and device control comprising respectively functions of a DT related to data acquisition or preprocessing, and control or actuation of observable elements. The main DT layer comprise three sets of functions (called sub-entities). The Operation and Management Sub-Entity includes functions (called Functional Elements -FE) related to the management and presentation of the models handled by the DT. The Application and Service Sub-Entity includes analytics, reporting and simulation functions, plus application support. Last, the Resource Access and Interchange Sub-Entity manages access control, interoperability and interfacing with other DTs or applications. All of this is completed by three orthogonal layers for data translation, assurance and security. See [10] or e.g. [7] (Figure 1).

The DT framework does not make any difference between the manufacturing elements that can have a DT. They are all only seen through the lens of what can be observed and the control operated. The eight types of Observable Manufacturing Elements (OMEs) [19] mix active elements like equipment or personnel which can act, with passive elements (products and documents) and organisational elements (process). OMEs are sensed by the Data Collection Sub-Entity and controlled or actuated by the Device Control Sub-Entity [19]. This can be discussed since not all manufacturing resources providing observable elements are devices, nor can they be controlled or actuated. This is particularly true for humans. In this case the DT can send commands, which might be or not followed by the person, but cannot control them in the same way machines are. Action of a human by the DT can also be hardly imagined. To address not only devices but any kind of physical twin, we propose to update the Device Control Sub-Entity into a Physical Twin Feedback Sub-Entity. Doing this, we add a new *Feedback FE*, whose role is to send to a twinned OME any message that is not a command or a direct actuation. For humans it can be a recommendation, instruction, or warning issued by the DT.

Operating DTs of cognitive technical entities as CCPS and human beings require some focus on how cognition is modeled within the DT. The Operation and Management Sub-Entity represents the emulation function of our DT architecture. This part of the framework maintains models managed by the DT, including synchronisation with the physical twin. The DT framework does not take into account cognition. When cognition models need to be integrated in a DT, they would currently be models among others in the Digital Modeling FE. The Application and Services Sub-Entity contains those functions which we consider are not the main function provided by a DT (i.e. emulation), because they relate to how the DT is used, not to what it is. The four functions listed comprise Simulation, but cognition is not considered. This new DT utility function we consider could be considered as a part of the Analytic Service FE. However
its purpose is not to do analytics, so we propose to add a separate *Cognition FE* box to the DT framework within the Application and Services Sub-Entity.

The distinction between active and passive manufacturing elements makes full sense in the context of a CPE or any flexible or reconfigurable manufacturing environment where active elements are given some autonomy or some intelligence. The Cognition FE should exist only for active elements, i.e., agents, to which it adds cognitive capabilities. Moreover, we have proposed in our CDT architecture that it is coupled with the control sub-system of the twinned element. From the DT framework point of view, this implies structuring more precisely the OTE-DT coupling to clarify information flows. More generally, this integration choice with a strong coupling remains also to be tested and compared to a light coupling where the Cognition FE relies only on data gathered by the DT and gives the OTE feedback through the Feedback FE.

# 6 Conclusion

This paper is an effort to clarify cognition in digital twins and cyber-physical systems. We have discussed the place and objective of cognition in digital twins coupled with entities that embed cognition or not, including humans. We have also highlighted that cognition can be implemented with different levels of complexity and discussed the place of cognitive architectures. We have clarified when a DT becomes a Cognitive Digital Twin (CDT), how a cognitive CPS can be built with a DT and the special case where the entity is a human, coupled with its Human Digital Twin (HDT).

The discussion is supported by a generic architecture of the Entity-CDT system in their environment, where the CDT is modelled according to its pillar function (*Emulation* as the intrinsic DT function, *Cognition*, and *Simulation* as the main service provided by a DT), and the entity is modelled taking an agent perspective that fits the context of the cyber-physical enterprise (CPE). The architecture shows the processing structure that any Entity-CDT system should implement and gives an example of how the simulation function could be implemented.

The proposed architecture will be further implemented with a cognitive agent layer on top of an ontology-based FIWARE 3 IOT platform, including a DRL algorithm for conducting simulations with the agents. Each agent drives the three functions to implement a CDT. This will be assessed on an assembly case involving humans and collaborative robots.

Last, we have discussed the integration of our work in the ISO 23247 standard, proposing extensions for cognition and updates to make the framework more suitable to handle humans. The study of the standard has revealed a clear orientation towards building supervision interfaces as digital twins, where the objective is to build a system that allows monitoring, data analysis and control by a human operator. This particular perspective explains probably the focus

<footnote n="3">³ www.fiware.org</footnote>
on control and actuation, which does not fit with humans. The agent perspective that would be needed for building a cyber-physical enterprise is different and questions the suitability of the ISO 23247 for such environments where autonomous technical agents cooperate or collaborate with humans. This will be the subject of a further study.

## ACKNOWLEDGEMENTS

This work has been partially supported by the ANR French National Research agency and the FNR Luxemburgish National Research funds project AI4C2PS (INTER/ANR/22/17164924/AI4C2PS), 2023-2026.

## References

1. Adl, A.E.: The cognitive digital twins: Vision, architecture framework and categories (2016)

2. Al Haj Ali, J., Lezoche, M., Panetto, H., Naudet, Y., Gaffinet, B.: Cognitive architecture for cognitive cyber-physical systems. IFAC-PapersOnLine **58**(19), 1180–1185 (2024). https://doi.org/10.1016/j.ifacol.2024.09.099, 18th IFAC Symposium on Information Control Problems in Manufacturing INCOM 2024

3. Ali, J.A.H., Gaffinet, B., Panetto, H., Naudet, Y.: Cognitive systems and interoperability in the enterprise: A systematic literature review. Annual Reviews in Control **57**, 100954 (2024). https://doi.org/10.1016/j.arcontrol.2024.100954

4. Bocklisch, F., Paczkowski, G., Zimmermann, S., Lampke, T.: Integrating human cognition in cyber-physical systems: A multidimensional fuzzy pattern model with application to thermal spraying. Journal of Manufacturing Systems **63**, 162–176 (2022)

5. De Oliveira, C.S., Sanin, C., Szczerbicki, E.: Visual content representation and retrieval for cognitive cyber physical systems. Procedia computer science **159**, 2249–2257 (2019)

6. Eirinakis, P., Kalaboukas, K., Lounis, S., Mourtos, I., Rožanec, J.M., Stojanovic, N., Zois, G.: Enhancing cognition for digital twins. In: 2020 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC). pp. 1–7. IEEE (2020)

7. Ferko, E., Bucaioni, A., Pelliccione, P., Behnam, M.: Standardisation in Digital Twin Architectures in Manufacturing. In: 2023 IEEE 20th International Conference on Software Architecture (ICSA). pp. 70–81 (Mar 2023). https://doi.org/10.1109/ICSA56044.2023.00015, https://ieeexplore.ieee.org.proxy.bnl.lu/document/10092613

8. Fournaris, A.P., Lalos, A., Kapsalas, P., Koulamas, C.: Decentralized, secure and cognitive architecture for automotive cyberphysical system of systems. In: 2020 9th Mediterranean Conference on Embedded Computing (MECO). pp. 1–5 (2020). https://doi.org/10.1109/MECO49872.2020.9134336

9. Hozdić, E., Makovec, I.: Evolution of the human role in manufacturing systems: On the route from digitalization and cybernation to cognitization. Applied System Innovation **6**(2), 49 (2023)
10. International Organization for Standardization: Additive manufacturing — digital product definition data representation (dpddr) — part 2: [specific part title] (2023), https://www.iso.org/obp/ui/iso:std:iso:23247:-2:ed-1:v1:en

11. Ivanov, D.: Intelligent digital twin (idt) for supply chain stress-testing, resilience, and viability. International Journal of Production Economics **263**, 108938 (2023)

12. Kotseruba, I., Tsotsos, J.K.: 40 years of cognitive architectures: core cognitive abilities and practical applications. Artificial Intelligence Review **53**(1), 17–94 (2020)

13. Krueger, V., Chazoule, A., Crosby, M., Lasnier, A., Pedersen, M.R., Rovida, F., Nalpantidis, L., Petrick, R., Toscano, C., Veiga, G.: A vertical and cyber-physical integration of cognitive robots in manufacturing. Proceedings of the IEEE **104**(5), 1114–1127 (2016)

14. Naudet., Y., Baudet., A., Risse., M.: Human digital twin in industry 4.0: Concept and preliminary model. In: Proceedings of the 2nd International Conference on Innovative Intelligent Industrial Production and Logistics - IN4PL. pp. 137–144. INSTICC, SciTePress (2021). https://doi.org/10.5220/001070900003062

15. Naudet, Y., Panetto, H., Yilma, B.A.: Towards cognitive interoperability in cyber-physical enterprises. IFAC-PapersOnLine **56**(2), 695–706 (2023). https://doi.org/10.1016/j.ifacol.2023.10.1648, 22nd IFAC World Congress

16. Neisser, U.: Cognitive psychology. New York, Appleton-Century-Crofts (1967)

17. Newell, A.: Unified theories of cognition. Cambridge, Mass. : Harvard University Press (1990)

18. Semeraro, C., Lezoche, M., Panetto, H., Dassisti, M.: Digital twin paradigm: A systematic literature review. Computers in Industry **130**, 103469 (2021). https://doi.org/10.1016/j.compind.2021.103469; https://www.sciencedirect.com/science/article/pii/S0166361521000762

19. Shao, G., Frechette, S., Srinivasan, V.: An Analysis of the New ISO 23247 Series of Standards on Digital Twin Framework for Manufacturing. In: Proc. of the ASME 2023 18th International Manufacturing Science and Engineering Conference. vol. 2: Manufacturing Equipment and Automation; Manufacturing Processes; Manufacturing Systems; Nano/Micro/Meso Manufacturing; Quality and Reliability. American Society of Mechanical Engineers, New Brunswick, New Jersey, USA (Jun 2023). https://doi.org/10.1115/MSEC2023-101127

20. Sharma, R., Gupta, H.: Leveraging cognitive digital twins in industry 5.0 for achieving sustainable development goal 9: An exploration of inclusive and sustainable industrialization strategies. Journal of Cleaner Production **448**, 141364 (2024)

21. Weichhart, G., Panetto, H., Molina, A.: Interoperability in the cyber-physical manufacturing enterprise. Annual Reviews in Control **51**, 346–356 (2021). https://doi.org/10.1016/j.arcontrol.2021.03.006