# Toward Improving Dynamic Resource Scheduling in the Context of Digital Twin of Emergency Department

Jessica Florencia, Thierry Moyaux, Lorraine Trilling, Guillaume Bouleux, Vincent Cheutet

► To cite this version:

Jessica Florencia, Thierry Moyaux, Lorraine Trilling, Guillaume Bouleux, Vincent Cheutet. Toward Improving Dynamic Resource Scheduling in the Context of Digital Twin of Emergency Department. IEEE Transactions on Automation Science and Engineering, 2025, 22, pp.7255 - 7267. 10.1109/TASE.2024.3463489. [hal-04884086](https://hal.science/hal-04884086v1)

HAL Id: hal-04884086

https://hal.science/hal-04884086v1

Submitted on 14 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Towards Improving Dynamic Resource Scheduling
in the Context of Digital Twin of Emergency
Department

Jessica Florencia, Thierry Moyaux, Lorraine Trilling, Guillaume Bouleux, and Vincent Cheutet

***Abstract***—The Emergency Department (ED) is a principal sector in the hospital that has to make crucial decisions effectively under uncertainties to ensure that the patients receive good quality care. Improving decision-making in ED, especially the resource scheduling decision, becomes one research interest. A promising approach to improve decision-making is by utilising Digital Twin (DT). This context of decision-making in DT is both (i) *an opportunity given by the DT* which provides data close to real-time and (ii) *constrained by the DT* which forces to make decisions dynamically rather than statically. That is, our research contributes to the decision layer of DT. More precisely, we propose a decentralised organisation to enhance the resource scheduling decision-making process in ED. This is a multi-agent system with a local dynamic resource scheduling based on a Mixed Integer Linear Programming (MILP) model. This proposition is compared to two other organisations, which are a centralised organisation that optimises the dynamic scheduling MILP globally, and the FIFO sequencing currently performed in the paediatric ED of the University Hospital Centre in Saint-Étienne, France. The results show that the proposed decentralised organisation provides good results quickly and seems promising for improving resource scheduling decision-making in ED with DT support.

***Note to Practitioners***—This research provides an initial study on the approach to improve the decision-making process in the Emergency Department (ED) of the hospital by using Digital Twin (DT). We focus more on improving resource scheduling decisions, which is a part of the decision layer of DT. We propose a decentralised organisation based on a multi-agent approach for solving the resource scheduling problem. Each resource agent, i.e. nurse, can run a local dynamic resource scheduling based on a Mixed Integer Linear Programming (MILP) model. Through our proposition with a centralised model and current practice in ED based on the triage-based First in First Out (FIFO) rule, we can see that our decentralised organisation is a favourable decision organisation in the context of decision improvement with DT, as it is capable of providing favourable results in a short time. This research will be further extended by including different resources and tasks in ED to make the model closer to reality in ED.

***Index Terms***—Digital Twin, Dynamic Resource Scheduling, Emergency Department

## I. INTRODUCTION

As the world changes rapidly, every company or institution faces numerous challenges and has to make effective decisions under uncertainties [1]. Improving decision-making under uncertainties for better organisational performance has

J. Florencia, T. Moyaux, L. Trilling, G. Bouleux, and V. Cheutet are with INSA Lyon, Université Lumière Lyon 2, Université Claude Bernard Lyon 1, Université Jean Monnet Saint-Étienne, DISP UR4570, Villeurbanne, 69621, France.

become one of the research interests that has emerged recently [2], [3]. The research on supporting decision-making also arises in the healthcare sector, as this sector deals with many unpredicted and complex events [4]. Many research works explore ways to improve the Emergency Department (ED) performances [5]–[9]. ED is a main and critical part of the hospital as it provides first-stage immediate care to patients at the hospital [9], [10]. Many decisions must be made quickly and accurately in ED under various uncertainties and complexities [8], [11].

Many previous works propose models and solutions for supporting the decision-making process in an ED to improve its performance and patient pathway. Simulation modelling, mathematical optimisation, and recently Digital Twin (DT), are the most proposed approaches for ED improvement [6]–[9]. Among these approaches, DT seems promising as it provides a virtual representation of a physical system with real-time data synchronisation. DT is also a great decision support system that enables faster data access, thus supporting the decision-maker to make more effective and accurate decisions [5], [12].

DT architecture consists of several layers – including information, model and decision layers – as shown in Fig. 1. For the purpose of improving decision-making in ED, this article assumes that the lower-level layers of a DT, i.e. information and model layer, exist and work well. This assumption allows us to focus on investigating how to make decisions on the upper layer of a DT (or decision layer), shown with blue colour in Fig. 1. This *context of a DT imposes both a constraint and an opportunity* for decision-making:

* *Opportunity provided by DT:* The decision-making layer of a DT has access to the “freshest” data available, as the lower-level layers of the DT update them whenever possible. As a consequence, the context of DT allows for better decision-making in the sense that they are closer to the current state of the physical system.

* *Constraint provided by DT:* DT provides real-time synchronisation which expects the virtual replica to provide quick predictions or solutions. However, calculations for supporting decisions may not be performed quickly enough, especially when the system (*i.e.*, *Physical Twin*) is very complex and dynamic.

This work tackles this problem by proposing an improvement for one of the crucial decisions in ED, namely, resource scheduling that has to be made quickly and accurately under various dynamics and constraints [9], [13]. Activities in the
<figure><img src="image_2.png" /><figcaption>Fig. 1. Architecture of DT for ED (adapted from [10])</figcaption></figure>

ED involve various resources, including human resources such as caregivers (physicians, nurses...) and stretcher-bearers, and physical resources such as medical equipment and treatment rooms or beds [9], [14]. In this work, we only address the scheduling of human resources, especially the nurses. Hence, the term "resource" in "resource scheduling" refers to human resources in this paper. The current way for resource scheduling in the considered ED – namely, the paediatric ED of CHU (Centre Hospitalier Universitaire, i.e., University Hospital Centre) in Saint-Étienne, France – follows a First-In-First-Out (FIFO) sequencing based on triage level carrying out FIFO on most severe patients first, then on less severe ones, as explained below. This FIFO is highly decentralised as each resource/nurse chooses their next patient locally. However, this kind of myopic decision is not optimal and can be improved in the context of DT. An approach widely studied in the literature is to use a centralised organisation that lets a single decision-maker optimise a model to generate the global resource schedule for all the nurses in the ED [9], [14]–[16]. This organisation is proven to produce an optimum result for scheduling [16]. Despite its theoretical optimality, a centralised organisation is considered less realistic in practice and may not provide a solution quickly enough, especially when the problem gets more complex and with uncertainties.

Hence, our research contribution is to propose a decentralised organisation for dynamic resource scheduling in the decision layer of a DT, next to compare this organisation to a centralised one and the currently-operated FIFO. This decentralised organisation is modelled as a multi-agent system with nurse-agent interactions and self-scheduling. This organisation is closer to the current practice and seems promising to produce good quality results faster. As just said, this proposed organisation is further evaluated and compared with the aforementioned triage level-based FIFO currently performed in the considered paediatric ED located in Saint-Étienne, France, and the centralised organisation as the optimum approach which constitutes the benchmark.

The outline of the paper is as follows. Section II reviews the literature on DT in healthcare and dynamic resource scheduling in ED. Section III provides a problem definition, an overview of the three organisations of decision-making for dynamic resource scheduling in ED, and then details the FIFO and centralised organisations. Section IV introduces our

decentralised organisation in detail. Section V discusses the experimental results and the quantitative comparisons of the three organisations. Section VI concludes.

## II. RELATED RESEARCH

### A. Digital Twin in Healthcare

Many technologies emerged in the Industry 4.0 era to improve systems and enhance digital transformation to keep up with the fast-changing world. One of the Industry 4.0 key technologies and the digital transformation pillars is Digital Twin (DT) [2], [17]. DT is a virtual representation of a physical system that is built on simulation using real-time data and is characterised by the synchronisation between the digital and physical systems [2], [3]. According to this definition, DT has three main elements, which are the physical entities, the virtual representation of the entities and the bi-directional data connection between these two elements [3], [17]. Since it provides a virtual representation of the entities with near real-time synchronization between the physical and virtual systems, DT may improve the performance of physical entities through computational techniques and analyses performed on their virtual replica. Therefore, DT is used to optimise operations, predict and prevent system mistakes and errors, monitor system performance, support the decision-making process, and improve the efficiency of the physical system [3], [18].

DT is currently implemented in various fields including manufacturing, agriculture, smart cities, and recently the healthcare sector [3], [18]. The implementation of DT would improve the healthcare sector because DT provides a high-fidelity representation of the actual dynamic system that enables the decision-maker to monitor and control the system and to predict and plan the modification or adjustment to care units with less costs [19]. Many previous research works have applied DT to improve the healthcare system. DT with cloud computing can provide real-time health monitoring for elderly or senior citizens [5]. Research of Croatti et al. [2] implements and uses DT to monitor and predict the operative steps to be taken in trauma management at the hospital. Similarly, DT can support hospital strategic planning by simulating patient flow and testing different modification scenarios to identify patient care improvement opportunities [4]. Few papers discussed DT implementation in the Emergency Department (ED), including Moyaux et al. [7] who proposes an architecture of DT for an ED in hospitals with agent-based architecture and Aluvalu et al. [18] who proposes a novel idea using DT for faster patients' data access in ED.

Most of the previous research works on DT in healthcare focus mainly on improving patient pathways in the entire hospital. Few papers discuss specifically the DT application to improve ED, and among them, no paper discusses related to improving the resource scheduling decision-making in ED, hence, the novelty of our work. Our research proposes an organisation and evaluates it with different decision organisations to improve the dynamic resource scheduling in ED with the support of DT.
## B. Resource Scheduling in Emergency Department

ED is an important sector in hospitals that has to provide immediate first-stage care to patients [10], [11]. The operations and decision-making in the ED face numerous challenges, including overcrowding due to sudden surges of patient arrival and limited resources [8], [16], [20]. The effectiveness of decisions made in ED, especially resource scheduling, significantly impacts the ED performance [15]. Therefore, much research works on improving resource scheduling in ED, either by static or dynamic scheduling approach [14]–[16]. Static resource scheduling assumes a fixed environment and generates a schedule in advance based on previous historical data and knowledge of process details. Harzi et al. [14] provide an example of static predictive resource scheduling in ED with Mixed Integer Linear Programming (MILP) that reduces the patients’ waiting time. However, ED is a dynamic environment with numerous uncertainties, and static scheduling may not be applicable when there are sudden changes in the ED [15].

Therefore, many research works nowadays focus on proposing a dynamic resource scheduling approach for ED, as it uses real-time data to react and reschedule resources when there are sudden changes in the system [15]. Most research works utilise an optimisation approach for modelling dynamic resource scheduling. Luscombe et al. [9] propose a dynamic scheduling framework with two layers of scheduling with parallel machines and flexible flow shop scheduling principles to provide real-time support for managing the scarce resources of an ED. Duma et al. [16] reduce the care process duration and make an ED less crowded by using real-time resource allocation policy based on greedy algorithm. Similarly, research by Tan et al. [15] designs and evaluates different dynamic resource allocation strategies with queuing and local search algorithms. De Queiroz et al. [21] combine a reoptimization heuristic and variable neighbourhood search to solve dynamic patient-to-doctor scheduling problems. Li et al. [22] use Markov decision process to study resource allocation with prioritization policy. These optimisation approaches for resource scheduling are considered centralised organisation, as a single decision-maker, i.e. a central computer, runs the models and provides the schedule to resources.

On the contrary, only a few papers discuss the decentralised decision approaches to resource scheduling in ED. Othman et al. [23] use agent-based simulation with a cooperation mechanism between central scheduling and resources agents to assign resources to patients dynamically in ED. Liu et al. [24] use discrete event and agent-based simulation to compare different resource scheduling strategies based on the combination of dispatching rules.

Our proposed decentralised organisation combines the multi-agent system and optimisation approach (mixed integer linear programming or MILP) for dynamic resource scheduling in ED. This model is adapted from the model proposed by Moyaux and Marcon [25] that combines a multi-agent system and MILPs to solve multiple travelling salesman problems (MTSP). The proposed model allows dynamic coordination and communication between the same resource agent types to reach the common goal. In addition, it also uses the

<figure><img src="image_3.png" /><figcaption>Fig. 2. Simplified Patient Pathway in ED. The decisions tackled in this research are taken at the beginning of the phase in blue in this pathway.</figcaption></figure>

<table><caption>TABLE I TRIAGE LEVELS OF PATIENTS</caption><thead><tr><td>Level</td><td>Status</td><td>Weight in MILP</td></tr></thead><tbody><tr><td>1</td><td>Resuscitation/acute</td><td>10<sup>8</sup></td></tr><tr><td>2</td><td>Very urgent</td><td>10<sup>6</sup></td></tr><tr><td>3</td><td>Urgent</td><td>10<sup>4</sup></td></tr><tr><td>4</td><td>Less urgent</td><td>10<sup>2</sup></td></tr><tr><td>5</td><td>Not urgent</td><td>1</td></tr></tbody></table>

concept of resource self-scheduling adapted from Traub et al. [26]. As no previous research work focuses on dynamic resource scheduling in ED using a combination of optimisation approach and multi-agent system with dynamic coordination between same agent type, this proposed organisation is the novelty of our research work.

## III. PROBLEM SPECIFICATION AND REFERENCE ORGANISATIONS DEFINITION

### A. Problem specification

ED provides complex and urgent care for each patient based on their conditions and needs. The summary of the general process or the patient pathway in the ED is illustrated in Fig. 2. Briefly, the patient (or, in our case in paediatric ED, a child patient) arrives at the ED and registers at the reception desk. A triage nurse welcomes the patient, does the registration, and inputs the patient's information into the information system. The triage nurse also performs an initial quick assessment of the patient. The triage nurse assigns a triage level to the patient based on the assessment result. The triage level ranges from scale 1 (most urgent) to 5 (no urgency), as shown in Table I.

According to this triage level and the way or approach of the resource scheduling, the nurse takes the patients to the proper treatment room. The nurse and the physician then examine and provide medical care to the patient. The patient may also need additional examination, such as blood tests in the laboratory. After the treatment, the physician decides the next step for the patient based on the patient's condition. If the patient needs special care in the hospital, the physician will send the patient to a hospital ward for hospitalisation. Otherwise, if the patient is fully treated and is getting better in the ED, the patient may be discharged and leave the ED.

Our research focuses on the resource scheduling decision, i.e. determining resource schedule (which resource treats which patient), in a part of the patient pathway in ED, namely medical examination and assessment. Generally, patients can
<figure><img src="image_4.png" /><figcaption>Fig. 3. Brief presentation of the organisations</figcaption></figure>

receive multi-stage treatment from different resources. However, in this research, we consider only the first treatment for each patient performed by a nurse (a nurse giving care to a patient) as the initial study on resource scheduling decision improvement in the decision layer of DT for ED.

As stated in the introduction, to enhance the decision-making process of nurse-to-patient scheduling, a decentralised organisation of resource scheduling is proposed. This proposed organisation is compared with two other organisations: a centralised organisation that runs a dynamic scheduling MILP and the current nurse-to-patient assignment decision performed in the paediatric ED of Saint-Étienne that follows FIFO sequencing based on triage level, which is called the FIFO organisation. The three organisations evaluated and compared in this research are summarized in Figure 3. The characteristics and the differences between the three organisations are summarised in Table II. FIFO organisation and centralised organisation are explained in detail in the subsection below, while the proposed decentralised organisation are explained further in the next section.

## B. Centralised Organisation

The centralised organisation schedules the nurse-to-patient scheduling by solving a dynamic version of a mixed integer linear programming (MILP) of the ED. This organisation operates by first capturing patients' data and considering nurses' availability as input. Then, it solves the scheduling MILP model implemented in the CPLEX solver. This MILP has an objective function which minimises the total weighted waiting time (WWT) of all patients. The waiting time is defined as the period between the patient's arrival at the ED and the patient's treatment start time by the nurse. The weight depends on the triage level of the patients shown in Table I. Patients with a triage level of one, whose conditions are the most critical, have the largest weight in order to treat them first. This MILP is re-run every time a new patient arrives to re-optimise the schedule and to cope with dynamic patient flow in ED. Therefore, the nurse-to-patient schedule needs

<table><caption>TABLE II
THE CHARACTERISTICS AND DIFFERENCES BETWEEN THE THREE ORGANISATIONS</caption><thead><tr><th rowspan="2"></th><th colspan="3">Organisation</th></tr><tr><th>Centralised</th><th>Decentralised</th><th>FIFO</th></tr></thead><tbody><tr><th>Decision-maker</th><td>A single authority (different from the nurses)</td><td>Each nurse-agent</td><td>Each nurse-agent</td></tr><tr><th>Scheduling approach</th><td>Dynamic scheduling MILP</td><td>Decentralised version of dynamic scheduling MILP</td><td>No scheduling, i.e., myopic sequencing</td></tr><tr><th>Interactions</th><td>None</td><td>Decentralised auction with multiple interaction rounds</td><td>None</td></tr><tr><th>Event triggering decision-making</th><td>New patient arrival</td><td>A nurse finishes a patient's treatment and decides the next patient to be treated</td><td>A nurse finishes a patient's treatment and decides the next patient to be treated</td></tr></tbody></table>

to be dynamically adjusted, which requires a constraint that forbids the current treatment processes to be stopped in order to ensure that the patients currently receiving treatment will not be rescheduled later.

The main part of the dynamic allocation algorithm is the MILP model of nurse scheduling adapted from models by Cao et al. [27] and Luscombe & Kozan [9]. The following Equations 1 through 13 are based on the following notations:

### Indices and sets:

* $j$ Patient $j = 0, 1, \dots, J$. $J$ is the number of patients. Patient $j = 0$ is the “dummy” patient, required to start the schedule and express precedence constraints

* $n$ Nurse $n = 1, \dots, N$, $N$ is the number of nurses

* $\mathcal{J}$ Set of patients, $\mathcal{J} = \{0, 1, \dots, J\}$

* $\mathcal{J}^*$ Set of patients whose treatment is finished ($\mathcal{J}^* = \{j \in \mathcal{J} | ct_j < t\}$)

* $\mathcal{N}$ Set of nurses, $\mathcal{N} = \{1, \dots, N\}$
**Decision variables:**

$s_j$ Treatment start time of Patient j

$c_j$ Treatment completion time of Patient j

$s_j$ Treatment start time of Patient j

$c_j$ Treatment completion time of Patient j

**Parameters:**

$a_j$ Arrival time of Patient j

$p_j$ Treatment duration of Patient j

$u_j$ Weight of triage level of Patient j (see Table I)

$zt_{jkn}$ Treatment order decision variable of Patient j in the previous round of scheduling, i.e., 1 if Patient k was treated immediately after Patient j by Nurse n in the previous round

$ct_j$ Completion time of Patient j in the previous round

$t$ Time of the current round of scheduling

**Equations:**

$$ \min w = \sum_{j=1}^{J} u_j(s_j - a_j) \quad (1) $$

subject to

$$ s_j + p_j = c_j \quad \forall j \in \mathcal{J} \quad (2) $$

$$ s_j \ge a_j \quad \forall j \in \mathcal{J} \quad (3) $$

$$ c_j - s_k \le M(1 - z_{jkn}) \quad \forall j \in \mathcal{J}, \forall k \in \mathcal{J}, \forall n \in \mathcal{N} \quad (4) $$

$$ \sum_{j=0, j \neq k}^{J} z_{jkn} = x_{kn} \quad \forall k \in \mathcal{J} \quad (5) $$

$$ \sum_{k=1, j \neq k}^{J} z_{jkn} \le x_{jn} \quad \forall j \in \mathcal{J}, \forall n \in \mathcal{N} \quad (6) $$

$$ \sum_{k=0}^{J} z_{0kn} \ge 1 \quad \forall n \in \mathcal{N} \quad (7) $$

$$ \sum_{n=1}^{N} x_{jn} = 1 \quad \forall j \in \mathcal{J} \quad (8) $$

$$ z_{jkn} = zt_{jkn} \quad \forall j \in J^*, \forall k \in \mathcal{J}, \forall n \in \mathcal{N} \quad (9) $$

$$ s_j \ge 0 \quad \forall j \in \mathcal{J} \quad (10) $$

$$ c_j \ge 0 \quad \forall j \in \mathcal{J} \quad (11) $$

$$ x_{jn} \in \{0, 1\} \quad \forall j \in \mathcal{J}, \forall n \in \mathcal{N} \quad (12) $$

$$ z_{jkn} \in \{0, 1\} \quad \forall j \in \mathcal{J}, \forall k \in \mathcal{J}, \forall n \in \mathcal{N} \quad (13) $$

Eq. (1) is the objective function of the model that minimises the total WWT over all patients. The waiting time is defined as the difference between a patient's arrival time ($a_j$) and treatment start time ($s_j$).

Eq. (2)-(7) are the sequencing constraints to determine the order of patients' treatment. Eq. (2) calculates the completion time of each patient that is acquired from the sum of the patients' treatment start time and treatment duration. Eq. (3) ensures that every patient's treatment starts after their arrival time. Eq. (4) is the constraint of treatment sequence as well as the constraint that ensures that there is no nurse idle time between two patients. Eq. (5) defines that each patient has a direct predecessor. In this case, the first patient treated by each nurse has a "dummy" patient (which starts each nurse's

schedule) as their predecessor. Eq. (6) ensures that each patient has no more than one direct successor in the nurse's schedule. Eq. (7) allows each nurse to have only one dummy patient ($j=0$) such that each other patient has a direct predecessor.

Eq. (8) is the assignment constraint that ensures that each patient is treated by exactly one nurse. Eq. (9) is the dynamic constraint that forbids to reschedule currently-treated patients later by setting their $z_{jkn}$ equal to their value in the previous schedule ($zt_{jkn}$) if their completion time in the previous schedule ($ct_j$) is larger than the current scheduling time $t$. Eq. (10) and Eq. (11) sets the treatment start time ($s_j$) and treatment completion time ($c_j$) values to be non-negative. Eq. (12) and Eq. (13) sets the value of $x_{jn}$ and $z_{jkn}$ to be binary.

This dynamic scheduling MILP model is solved whenever a patient arrives at the triage nurse. After the triage process is performed by the triage nurse, the patient information is inputted into the ED's information system. The model that is built inside this information system receives the arrival time and triage level of the patient, then sets the scheduling time $t$ equal to the arrival time of this last patient. Next, it solves the scheduling problem in Eq. (1)-(13) to generate the nurse-to-patient schedules. Then, the model stores each patient's completion time to variable $ct_j$ and the patient order variable to $zt_{jkn}$ for the next round. This generated schedule is used by the nurse until the arrival of the next patient; such an arrival triggers a new run of this model.

**C. FIFO Organisation**

The FIFO organisation corresponds to the current nurse-to-patient scheduling in the paediatric ED in Saint-Étienne, France. After a patient arrives and is triaged by the triage nurse, the triage nurse inputs this patient's information, such as their arrival time, triage level and current conditions (symptoms), into the ED's information system. This patient's information is then displayed in the dashboard accessible to all nurses. If the patient's condition is bad and categorised as triage level 1 or 2, any available nurse has to immediately find the doctor or physician and bring the patient directly to the treatment room. Otherwise, the nurse may choose the patient to be treated from this dashboard based on priority management, that is by considering the triage level and the order of arrival of the patients. More precisely, all patients with the most urgent triage level are treated in FIFO order; when these patients are all treated, the patients with the next triage level are treated in FIFO order, and so on until triage level 5.

We implemented this triage level-based FIFO in AnyLogic software as a multi-agent system. The agents involved in our organisation are (i) the nurses and triage nurse, as active agents and (ii) the patients as passive agents. The state chart of the nurses can be seen in Fig. 4. The considered nurse is initially in `FIFO0_initialize_state`. This nurse then fires transition `FIFO1_nurse_start` to start working if FIFO is chosen as the running organisation. The nurse waits for a new patient by regularly checking the dashboard in state `FIFO2_aware_forPatients`. When there is a new patient in the dashboard, the nurse who notices them fires transition `FIFO3_nurse try to take patients`
<figure><img src="image_5.png" /><figcaption>Fig. 4. Nurse statechart of triage level-based FIFO organisation</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5. Scheduling and Interactions in every round of our decentralised organisation</figcaption></figure>

and calls the sorting function to sort the patients based on their triage level (first/main criterion) and arrival time (second criterion). Patients with more urgent conditions and arrived first are placed on the first row of the list. Next, the nurse takes the first patient in the sorted list by firing transition *FIFO5a_nurse_take patient* and treats them for a certain treatment duration in state *FIFO6_nurse_treat patient*. After finishing treating this patient, the nurse can take and treat other patients by checking the dashboard (i.e., going back to state *FIFO2_aware_forPatients*).

## IV. DECENTRALISED DYNAMIC RESOURCE SCHEDULING ORGANISATION

Our proposed decentralised organisation relies on a multi-agent system in which each nurse is represented as an agent who has local information, including the list of patients allocated to this nurse stored in her/his *patientList*. In addition, the nurse can also make local decisions to schedule the patient allocated to her/him and exchange patients with the other nurses. Scheduling involves two subproblems: (i) assigning (or allocating) patients to nurses and (ii) sequencing every nurse's patients. Each nurse solves the sequencing subproblem and generates his/her schedule for the patients in his/her *patientList* by running a decentralised version of the MILP in Eq. 1-13.

In order to solve the allocation subproblem, the nurses also interact and communicate with each other in order to exchange patients according to the Contract Net Protocol (CNP) [25], [28]. CNP in this research work is similar to a decentralised auction because a task/job cannot be split to create a network of contracts. This is a decentralised auction because the auctioneer is not the same in every round. (Below, the auctioneer in a given round will be called the host.) Instead, each nurse may play the role of either host/auctioneer or guest. An interaction round is initiated by a nurse whose patient's treatment has just finished and who has to choose the next patient to be treated. This nurse becomes the host who begins a new interaction round, while all other nurses become guests. After the interaction round finishes, another round may be started by another nurse who becomes the new host. Shortly, the role of host or guest in each round may not be held by the same nurse but may be swapped between nurses.

A round of interactions is explained through the interaction diagram shown in Fig. 5. During a round of interaction, the host and guests interact by sending and receiving messages. They also run three decentralised versions of centralised scheduling MILP as follows.

* Nurse Proposal MILP (NProp): the host and each guest run this MILP once to propose one of their patients from *patientList*.

* Single Nurse Scheduling MILP (SinNS): the host runs this MILP 2n+1 times (with n = the total number of nurses involved in a round of interaction) and each guest runs this MILP 3 times to schedule (sequencing) their patients from *patientList* and calculate the WWTs for three scenarios.

* Host Allocation MILP (Alloc): the host runs this MILP once to allocate locally the proposed patients to the nurses involved in the round of interaction.

These three decentralised versions of scheduling MILP are explained further in the next subsections.

A round of interaction as shown in Fig. 5 is started by a nurse whose patient treatment is finished and who needs to take another patient. This nurse becomes host (h) of the round and starts by proposing one of the patients from her/his *patientList* ($j_h$) by running the NProp. The host sends this $j_h$ to all other nurses, that are called as guests ($g_n$). Each guest runs also the NProp to propose his/her patient ($j_{g_n}$). (S)he runs also the SinNS for calculating three WWT values:

* $w_{g_n}^{j_{g_n}}$: Guest $g_n$'s WWT with all patients in *patientList* without $j_h$;

* $w_{g_n}^{j_h}$: Guest $g_n$'s WWT with all patients in *patientList* without $j_{g_n}$ and with $j_h$;

* $w_{g_n}^{j_h, j_{g_n}}$: Guest $g_n$'s WWT with all patients in *patientList* with both $j_{g_n}$ and $j_h$.

Each guest then sends the $j_{g_n}$ and the three WWTs to the host. The host receives it and puts it in the matrix allocation [$w_N^j$]. The allocation matrix contains the value of the three total WWTs calculated by the host nurse and all guest nurses (row) for each scenario possible (column). Next, the host completes the matrix by calculating several WWTs for three scenarios:
* $w_h^{j_h}$: All patients in the host h's patientList with $j_h$ (without any patient $j_{g_n}$ proposed by guest nurses);

* $w_h^{j_{g_n}}$: All patients in the host h's patientList without $j_h$ and with each $j_{g_n}$ proposed;

* $w_h^{j_h, j_{g_n}}$: All patients in the host h's patientList with $j_h$ and each $j_{g_n}$ proposed.

These values are stored in $[w_N^J]$. After that, the host runs Alloc to allocate patient bundles (set of 0, 1, or 2 patients) to each guest. The host sends these patient bundles to each guest (represented by the last three arrows in Fig. 5). All nurses then update their patientList and the host takes the next patient to be treated based on the results of Alloc.

The three decentralised versions of MILP are explained further in subsection IV-A, IV-B, and IV-C. In addition, subsection IV-D describes the implementation of this round of interaction in AnyLogic.

### A. Nurse Proposal MILP (NProp)

The Nurse Proposal MILP (NProp) is the first decentralised version of centralised scheduling MILP that purposed to find the patient to be proposed from the nurse's patientList. The notations and equations of NProp are as follows.

#### Indices and sets:

$j$ Patient $j = 0, 1, \dots, J$. $J$ is the number of patients. Patient $j = 0$ is the “dummy” patient, required to start the schedule and express precedence constraints

$\mathcal{J}$ Set of patients

$\mathcal{J}^*$ Set of patients whose treatment is finished ($\mathcal{J}^* = \{j \in \mathcal{J} | ct_j < t\}$)

#### Decision variables:

$s_j$ Treatment start time of Patient $j$

$c_j$ Treatment completion time of Patient $j$

$z_{jk}$ 1 if Patient $k$ treated immediately after Patient $j$, 0 otherwise

$\text{kept}_j$ 1 if this nurse keeps Patient $j$ on the patientList, 0 if this nurse proposes Patient $j$ to the other nurses

#### Parameters:

$a_j$ Arrival time of Patient $j$

$p_j$ Treatment duration of Patient $j$

$u_j$ Weight of triage level of Patient $j$ (see Table I)

$zt_{jk}$ Treatment order decision variable of Patient $j$ in the previous round of scheduling, i.e., 1 if Patient $k$ was treated immediately after Patient $j$ in the previous round

$ct_j$ Completion time of Patient $j$ in the previous round

$t$ Time of the current round of scheduling

#### Equations:

$$ \min w = \sum_{j=1}^{J} u_j(s_j - a_j) \quad (14) $$

subject to

$$ s_j + p_j = c_j \quad \forall j \in \mathcal{J} \quad (15) $$

$$ s_j \ge a_j \quad \forall j \in \mathcal{J} \quad (16) $$

$$ c_j - s_k \le M(1 - z_{jk}) \quad \forall j \in \mathcal{J}, \forall k \in \mathcal{J} \quad (17) $$

$$ \sum_{j=0, j \neq k}^{J} z_{jk} = \text{kept}_k \quad \forall k \in \mathcal{J} \quad (18) $$

$$ \sum_{k=1, j \neq k}^{J} z_{jk} \le \text{kept}_j \quad \forall j \in \mathcal{J} \quad (19) $$

$$ \sum_{j=0, j \neq k}^{J} \text{kept}_j = J - 2 \quad (20) $$

$$ z_{jk} = zt_{jk} \quad \forall j \in \mathcal{J}^*, \forall k \in \mathcal{J} \quad (21) $$

$$ s_j \ge 0 \quad \forall j \in \mathcal{J} \quad (22) $$

$$ c_j \ge 0 \quad \forall j \in \mathcal{J} \quad (23) $$

$$ \text{kept}_j \in \{0, 1\} \quad \forall j \in \mathcal{J} \quad (24) $$

$$ z_{jk} \in \{0, 1\} \quad \forall j \in \mathcal{J}, \forall k \in \mathcal{J} \quad (25) $$

Eq. (14) is the objective function of the model (same as Eq. (1)) which minimises the total WWT of all patients in a nurse's patientList. Eq. (15)-(20) are the sequencing constraints. Eq. (15)-(17) are the same as Eq. (2)-(4). Eq. (18) sets that each patient kept by the nurse ($\text{kept}_j = 1$) has a direct predecessor, including the first patient in the sequence that has a “dummy” patient as the predecessor. Eq. (19) ensures each patient kept by the nurse ($\text{kept}_j = 1$) has no more than one direct successor in the patient sequence. Eq. (20) makes sure that only one patient in patientList is proposed and not kept by the nurse. Eq. (21) is the dynamic constraint similar to Eq. (9) that forbids to re-schedule patients finished or treated later. Eq. (22)-(23) set the value of treatment start time ($s_j$) and treatment completion time ($c_j$) to be non-negative. Eq. (24)-(25) set the value of $\text{kept}_j$ and $z_{jk}$ to be binary.

The patient whose $\text{kept}_j$ value is 0 after solving this MILP is the patient $j_h$ proposed by the host in the first three arrows (messages) in Fig. 5. In other words, after solving the NProp problem, the host nurse broadcasts her/his patient $j_h$ to the guest nurses. Next, each guest nurse $g_n$ runs also NProp to propose one of their patients $j_{g_n}$ to the host, which is shown by the fourth, fifth and sixth arrows in Fig. 5.

### B. Single Nurse Scheduling MILP (SinNS)

The second decentralised version of scheduling MILP is the Single Nurse Scheduling (SinNS in Fig. 5). Host and guests run this MILP several times to schedule their patients and to calculate the WWT values ($w_h^{j_h}, w_h^{j_{g_n}}, w_h^{j_h, j_{g_n}}$ by host; $w_{g_n}^{j_{g_n}}, w_{g_n}^{j_h, j_{g_n}}$ by guests) that is stored in matrix allocation $[w_N^J]$ and used for the allocate locally the patients. This MILP has the same sets, decision variables, and parameters as the NProp, except that it involves no variable $\text{kept}_j$. The equations of SinNS are as follows.

#### Equations:

$$ \min w = \sum_{j=1}^{J} u_j(s_j - a_j) \quad (26) $$

subject to

$$ s_j + p_j = c_j \quad \forall j \in \mathcal{J} \quad (27) $$

$$ s_j \ge a_j \quad \forall j \in \mathcal{J} \quad (28) $$

$$ c_j - s_k \le M(1 - z_{jk}) \quad \forall j \in \mathcal{J}, \forall k \in \mathcal{J} \quad (29) $$
$$ \sum_{j=0, j \neq k}^{J} z_{jk} = 1 \qquad \forall k \in \mathcal{J} \qquad (30) $$

$$ \sum_{k=1, j \neq k}^{J} z_{jk} \le 1 \qquad \forall j \in \mathcal{J} \qquad (31) $$

$$ z_{jk} = zt_{jk} \qquad \forall j \in \mathcal{J}^*, \forall k \in \mathcal{J} \qquad (32) $$

$$ s_j \ge 0 \qquad \forall j \in \mathcal{J} \qquad (33) $$

$$ c_j \ge 0 \qquad \forall j \in \mathcal{J} \qquad (34) $$

$$ z_{jk} \in \{0, 1\} \qquad \forall j \in \mathcal{J}, \forall k \in \mathcal{J} \qquad (35) $$

Eq. (26) is the objective function of the model (same as Eq. (1)). Eq. (27)-(31) are the similar sequencing constraints as Eq (2)-(6) except that there is no nurse index involved as this model is locally optimised by each nurse. Eq. (32) is the same dynamic constraint as Eq. 9. Eq. (33)-(34) are the non-negative constraints. Eq. (35) sets the value of $z_{jk}$ to be binary.

### C. Host Allocation MILP (Alloc)

The third decentralised version of centralised scheduling MILP is called the Host Allocation MILP (Alloc in Fig. 5). The host runs this MILP once to assign every patient proposed to exactly one nurse. The results of this MILP are the patient bundles, which are set of 0, 1, or 2 patients. This means that each nurse may have one, two, or no patients assigned. This MILP uses the WWT values stored in the allocation matrix $[w_N^J]$. The notations and equations of the Alloc are as follows.

#### Indices and sets:

j Individual patient j proposed by host or guest nurse
$\bar{j}$ patient bundle $\bar{j}$, i.e. individual patient j proposed and two-patient bundle which are the combination of a patient proposed by the host and a patient proposed by each guest ($\bar{j} \in \bar{\mathcal{J}}$).

n Individual host or guest nurse n

$\mathcal{J}$ Set of patients proposed by host or guest

$\bar{\mathcal{J}}$ Set of patient bundles. $\bar{\mathcal{J}} = \{j_h, j_{g_1}, \dots, j_{g_n}, \{j_h, j_{g_1}\}, \dots, \{j_h, j_{g_1}\}\}$

$\mathcal{N}$ Set of nurses

#### Decision variables:

$x_n^{\bar{j}}$ 1 if patient bundle $\bar{j}$ is assigned to nurse n

#### Parameters:

$w_n^J$ Total WWT of patient bundle $\bar{j}$ treated by nurse n. The value of $w_n^J$ is part of the matrix allocation $[w_N^J]$

$[w_N^J]$ Matrix allocation, contains all value of $w_n^J$ with each row represents nurse n and each column represents a patient bundle $\bar{j}$

$b_n^{\bar{j}}$ Binary relation value between patient bundle $\bar{j}$ and patient j. 1 if patient bundle $\bar{j}$ contains patient j, 0 otherwise

#### Equations:

$$ \min \quad w' = \sum_{\bar{l} \in \bar{\mathcal{J}}} x_n^{\bar{j}} w_n^{\bar{j}} \qquad (36) $$

$$ \text{s.t.:} \quad \sum_{n \in \mathcal{N}} \sum_{\bar{j} \in \bar{l}} x_n^{\bar{j}} b_n^{\bar{j}} = 1 \qquad \forall j \in \mathcal{J} \qquad (37) $$

$$ \sum_{\bar{j} \in \bar{\mathcal{J}}} x_n^{\bar{j}} = 1 \qquad j = j_h \qquad (38) $$

$$ \sum_{\bar{j} \in \bar{\mathcal{J}}} x_n^{\bar{j}} \le 1 \qquad \forall j \in \mathcal{J}, j \ne j_h \qquad (39) $$

$$ x_n^{\bar{j}} \in \{0, 1\} \qquad \forall \bar{j} \in \bar{\mathcal{J}}, \forall n \in \mathcal{N} \qquad (40) $$

Eq. (36) is the objective function of Alloc that minimise the total WWT of all patient bundles. This MILP has four constraints. Eq. (37) ensures that each patient is assigned only to one nurse. Eq. (38) forces the host to take one patient bundle because the host has to take and decide the patient to be treated next. Eq. (39) lets the guest nurse either take one patient bundle or not take any bundle. Eq. (40) make sure that the decision variable $x_n^{\bar{j}}$ is binary.

### D. AnyLogic Implementation

We modelled the described round of interactions of nurse agents in Fig. 5 in AnyLogic 8.5.2 in the form of three state charts as shown in Fig. 6. The host_Nurse state chart models the initialisation process and the host role. The initialisation process here represents the way that the nurses take and treat their first patient. The guest_Nurse state chart is built based on the guest's actions and interactions. Meanwhile, the storingPatients state chart models the process of each nurse taking available patients in the waiting room and storing it in their patientList, such that these patients will be considered for future scheduling in the following interaction rounds. The numbers in Fig. 6 show the order of activation of the states and transitions: CNP0 initialises the host, CNP1 makes every guest wait for a proposal, CNP2 makes one host start, and so on.

## V. RESULTS AND DISCUSSIONS

We implemented the three organisations in AnyLogic 8.5.2 and IBM ILOG CPLEX 22.1.1. We compared their application to a dataset adapted from the paediatric emergency department of Saint-Étienne University Hospital. Each row of the dataset represents each patient, and each column shows the information related to patients, including triage level and arrival time. We extracted the data of one shift from 10 a.m. to 10 p.m. This shift includes 45 patients and 4 nurses. Most patients are categorised as triage level 4 (33 patients) in this shift, 8 patients are categorised as triage level 3 and 4 patients as triage level 5. No patient is categorised as triage levels 1 or 2. In addition, we generated 100 replications from this dataset using data shuffling and applied them to the three organisations. Data shuffling is chosen for population generation because we aim to compare the three organisations with the same metric, thus we need the population for replication with the same parameters but different arrival orders and different numbers of patients arriving per hour, and not generating the new population with totally different parameters.

The first result acquired from implementing three organisations is the nurse-to-patient schedule that is visualised in the form of a Gantt chart. The schedules are generated dynamically on every re-scheduling of the three organisations. Fig. 7
<figure><img src="image_7.png" /><figcaption>Fig. 6. Statecharts for decentralised organisation</figcaption></figure>

shows the final schedule produced for the three organisations from the original dataset. The x-axis of the chart shows the scheduling time point (in minutes), while the y-axis shows the nurses who participate in the scheduling. Each block in the Gantt chart represents a patient. The value on each block is the patient's index $j = 1...45$. The block colour represents the triage level of each patient, as shown in the legend. The final schedule of the centralised organisation is generated after the last (45th) patient arrives inside the waiting room. On the other hand, for both decentralised and FIFO organisations, the final schedules are produced after the last waiting patient is taken by a nurse, i.e., after the treatment of patient 36 is finished, and the nurse needs to take the next patient (in this case, patient 44).

We can see from the schedules generated in Fig. 7 that the three organisations schedule the patients properly by respecting the triage level of patients. The patients with triage level 3 are scheduled at similar time points in the three organisations, while all patients with triage level 5 are scheduled at the end of the schedule. However, the sequence of patients with triage level 4 varies in the different organisations because of the different ways of sequencing patients. The FIFO organisation applied first in first-in-first-out sequencing rules without any scheduling model by respecting the triage level. Hence, it schedules all patients with triage level 4 based on the order of arrival to make sure that the first patient that comes to the waiting list is the first to be out of the waiting list and taken by the nurse. Besides, the proposed centralised and decentralised organisations run the scheduling MILP model, which aims to minimise the total WWT of all patients. This MILP model produces results that follow the shortest processing time rules instead of FIFO sequencing. Furthermore, the decentralised organisation involves interaction and local allocation between agents. Therefore, it generates a slightly different schedule than the centralised organisation.

The second result obtained is the objective function value of the total WWT of all patients. The scheduling MILP model from centralised and decentralised organisations produces this value as the main result. However, this value is not directly

<figure><img src="image_8.png" /><figcaption>Fig. 7. Final schedule generated by three organisations</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Fig. 8. Total weighted waiting time of the 3 organisations</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 9. Average waiting time of patient per triage level</figcaption></figure>

<table><caption>TABLE III
PERCENTAGE OF RESOURCES UTILISATION OF THE 3 ORGANISATIONS</caption><thead><tr><th rowspan="2">Nurse</th><th colspan="3">Centralised</th><th colspan="3">Decentralised</th><th colspan="3">FIFO</th></tr><tr><th>μ</th><th>Med</th><th>σ</th><th>μ</th><th>Med</th><th>σ</th><th>μ</th><th>Med</th><th>σ</th></tr></thead><tbody><tr><td>N1</td><td>94.1</td><td>94.3</td><td>4.8</td><td>91.9</td><td>91.8</td><td>5.9</td><td>93.8</td><td>94.4</td><td>5.1</td></tr><tr><td>N2</td><td>93.7</td><td>95.1</td><td>5.3</td><td>92.3</td><td>94.1</td><td>6.8</td><td>92.8</td><td>93.6</td><td>5.1</td></tr><tr><td>N3</td><td>93.0</td><td>93.6</td><td>5.3</td><td>91.3</td><td>92.3</td><td>6.1</td><td>93.8</td><td>95.6</td><td>5.2</td></tr><tr><td>N4</td><td>92.9</td><td>94.2</td><td>4.6</td><td>91.6</td><td>93.2</td><td>6.4</td><td>94.7</td><td>95.3</td><td>5.1</td></tr></tbody></table>

generated in the FIFO organisation because it does not run any scheduling MILP model. Therefore, the WWT is computed additionally so that the three organisations can be compared using the same metric. The total WWT values, their distributions, and the mean value from the experiment for three organisations are shown in Figure 8.

It can be seen from Fig. 8 that the centralised organisation produces the optimum scheduling with the lowest WWT as expected. The centralised has the global information of the patients and nurses and optimises the global system, so it produces the global optimum solution. The FIFO organisation produces a bigger WWT than the centralised organisation, but it has a smaller WWT compared to the decentralised organisation.

We also compare the average waiting time of patients from the three organisations without any weight associated to the priority level. The box-and-whisker plot in Fig. 9 shows the waiting time distribution from different triage levels. For triage level 3, the average waiting time of centralised organisation is the lowest and the best among the others, while the FIFO is second lowest, and decentralised has the highest average waiting time. This happens because the FIFO organisation always prioritises patients with triage level 3 thus lowering their average waiting time. This result also explains the average WWT result orders as the weight for patients with triage level 3 is 10000 (cf. Table I). From the waiting time distribution of the patients with triage level 4, who are the population majority, we can see that the centralised still performed the best since the average waiting time is less dispersed. In addition, the average waiting times are the lowest among other organisations. The second best is the decentralised organisation, which has a lower mean waiting time than the FIFO organisation. The FIFO organisation has dispersed waiting time and the highest waiting time average among all organisations. As for the patients with triage level 5, the decentralised performs slightly better than the centralised since it produces a lower average and median waiting time. The FIFO still produces the highest average and median waiting time. Overall, the centralised organisation outperforms

the two other organisations based on waiting time distributions.

We also obtain the queue length on each hour during the 12-hours shift as shown in Fig. 10. The queue length values present the number of patients waiting for the first treatment by the nurse in the waiting room. This indicator is useful for knowing the crowdness level of ED. From the result, we can see that the centralised organisation performs best as it has the smallest and least dispersed queue length on each hour. The decentralised organisation has a similar queue length average and distribution to the centralised organisation and is better than the FIFO organisation which has the most dispersed and biggest average queue length among others most of the time.

Besides the nurse-to-patient schedules, total WWT, the average waiting time, and the queue length, we also compare the indicator for the resources or nurses, i.e. resource utilisation. We present the mean (μ), median (Med.), and standard deviation (σ) of the percentage of resource utilisation in Table III. From this table, we can see that in general, the three organisations have a good and similar resource utilisation value. This shows that the nurses are properly scheduled and well-balanced in general. The decentralised organisation has on average the lowest resource utilisation, but the difference with the other organisations is just 2-3%.

In addition, as ED evolves dynamically, it needs a decision support system that can provide quickly good quality solutions. Hence, we calculate and evaluate the computation time of the three organisations. The computation time considered is the time required for CPLEX to produce the result. Hence,
<figure><img src="image_11.png" /><figcaption>Fig. 10. Queue length comparison for a nurse shift (12 hours)</figcaption></figure>

for the FIFO organisation, the computation time is 0 since there is no CPLEX call. For the centralised organisation, the computation time of a replication is the sum of all times required for CPLEX to produce the results (scheduling for 45 patients). Since some instances took a very long time to produce a schedule, we set a 2 minutes time limit for each CPLEX run. For decentralised organisation, the computation time is the sum of all time required for CPLEX to produce the results for local scheduling and allocation for all 45 patients, while also considering parallelism with the maximum times of concurrent CPLEX runs as in [25].

Table IV shows the average and maximum computation time for each organisation. We can see that the FIFO organisation has the lowest computation time since it involves neither scheduling nor interactions and lets the nurse take a patient from the sorted list. The decentralised organisation can also solve the scheduling fast with an average of 11 seconds for 45 interaction rounds. Conversely, the centralised organisation has the longest computation time among others. It takes on average 560.76 seconds or around 9 minutes and a maximum of 3068.83 seconds or around 51 minutes for one replication of 45 scheduling decisions. It can be longer if we do not put any time limit on the CPLEX call.

In summary, the FIFO organisation can provide the quick- est solution, followed by the decentralised organisation that produces a fast solution. On the contrary, the centralised organisation requires a longer time to obtain the solution and to make the scheduling decision. It occurs because the cen- tralised organisation considers all patient information for re- optimising the scheduling, while the decentralised organisation considers only local patient information owned by each nurse. Furthermore, these computation time results illustrate briefly the complexity of the scheduling problem tackled by the three organisations. Further complexity analysis may be interesting, but we currently do not consider it as our scope and the key points to be delivered in this paper. We currently also do not use metaheuristics to shorten the computation time. Nevertheless, since the same metaheuristics could be used for both centralised and decentralised organisations, the impact of

<table><caption>TABLE IV
COMPUTATION TIME OF THREE ORGANISATIONS</caption><thead><tr><th></th><th>Centralised</th><th>Decentralised</th><th>FIFO</th></tr></thead><tbody><tr><td>Average Computation time (s)</td><td>560.76</td><td>11.71</td><td>0</td></tr><tr><td>Max Computation time (s)</td><td>3068.83</td><td>21.66</td><td>0</td></tr></tbody></table>

its utilisation will be equivalent and will not change signifi- cantly our conclusion that FIFO organisation has the shortest computation time, followed by decentralised organisation, and centralised organisation has the longest computation time.

To sum up, each organisation has its strengths and draw- backs. Concerning the solution quality, which considers wait- ing time, queue length, and resource utilisation as the indicator and the schedule generated, the centralised organisation per- forms best and produces the best solution. The decentralised results are better than the current FIFO organisation in terms of overall average waiting time and queue length. In terms of speed to obtain the solution, which is observed through the computation time, the FIFO organisation performs best, followed by the decentralised organisation that can provide a fast solution. Centralised produces the optimal solution, but it takes a longer time to generate the result even in the simplified case in ED of only considering each nurse giving each patient only one treatment. It might take a much longer time when the scheduling considers more complex tasks and resources. As we require a good and quick resource scheduling decision to deal with various constraints and dynamics in ED, also in the context of DT, the centralised organisation may not be realistic as it is unable to give quick solutions despite its best solu- tion quality. However, the proposed decentralised organisation seems a promising alternative for dynamic resource scheduling in ED as it produces relatively good quality and fast solutions. It does not generate results as good as the centralised, but the solution difference between the two is relatively small. It is also more realistic as the nurses can interact and communicate with each other to decide faster. It might be able to work well with more complex cases of dynamic resource scheduling in the context of DT in ED that requires both good quality and
fast solutions to improve the crucial decision-making in ED.

## VI. CONCLUSION

This paper is concerned with improving one of the crucial decision-making processes in ED, which is dynamic resource scheduling. We enhance this process by using a DT and working on its decision layer. This context is both (i) allowed by the real-time data provided by DT which allows better decisions, and (ii) constrained by the need for dynamic decision-making. To have (i) better and (ii) faster decision-making, we propose a decentralised organisation for dynamic resource scheduling; it is implemented as a multi-agent system that involves nurse-agent interaction and self-scheduling. This organisation is evaluated and compared with the centralised organisation that is proven by previous research works to provide optimality, and the current way of resource allocation, namely FIFO organisation, that follows FIFO sequencing considering patients' triage level. These three organisations are explored by accommodating the case of resource scheduling in a paediatric ED in Saint-Étienne, France, which considers each nurse giving only one treatment to each patient.

We compare these three organisations on a dataset adapted from real data from this ED. For that purpose, we run simulations and compare the quality of scheduling and speed of computation of these three organisations. The centralised one produces the best quality but takes a longer time to complete. On the other hand, FIFO organisation generates less effective results but with the best computation speed. The proposed decentralised organisation seems to be a favourable organisation for dynamic resource scheduling in the context of DT as it has relatively good quality scheduling and can generate results in a short time. It is more realistic than the optimum centralised organisation as in real situations, nurses usually communicate with each other and make their own decisions on selecting the next patient to be treated.

In future work, we plan to extend the dynamic resource scheduling problem by considering more tasks for each patient and more types of resources (physicians, boxes, etc.) to make the model closer to reality in ED. For instance, this will make it possible to model that a patient first sees a nurse, then a physician, who will prescribe a medical exam (blood test, scanner...) or treatment, and next the nurse again who will administer treatment or carry out the medical exam. This extension will complete the current version of our models, which considers only the first nurse visit. In addition, as this research work considers parameters to be deterministic, while ED has a lot of uncertainties in reality, we will also model the uncertainties for the parameters, such as arrival time, treatment time, etc. in future work. Furthermore, as this study is an initial study of dynamic resource scheduling in a digital twin context, we plan to extend and consider other DT benefits for future work.

## ACKNOWLEDGMENTS

This work contributes to JUNEAU project that is funded by ANR (Agence Nationale de la Recherche - France) "ANR-22-CE46-0010-01". A CC-BY public copyright license has been

applied by the authors to the present document and will be applied to all subsequent versions up to the Author Accepted Manuscript arising from this submission, in accordance with the grant's open access conditions.

## REFERENCES

[1] O. Cardin, D. Trentesaux, A. Thomas, P. Castagna, T. Berger, and H. Bril El-Haouzi, "Coupling predictive scheduling and reactive control in manufacturing hybrid control architectures: state of the art and future challenges," *Journal of Intelligent Manufacturing*, vol. 28, no. 7, pp. 1503-1517, 2017.

[2] A. Croatti, M. Gabellini, S. Montagna, and A. Ricci, "On the integration of agents and digital twins in healthcare," *Journal of Medical Systems*, vol. 44, pp. 1-8, 2020.

[3] C. Semeraro, M. Lezoche, H. Panetto, and M. Dassisti, "Digital twin paradigm: A systematic literature review," *Computers in Industry*, vol. 130, p. 103469, 2021.

[4] A. Karakra, F. Fontanili, E. Lamine, J. Lamothe, and A. Taweel, "Pervasive computing integrated discrete event simulation for a hospital digital twin," in *2018 IEEE/ACS 15th international conference on computer systems and Applications (AICCSA)*, IEEE. Aqaba, Jordan: IEEE, 2018, pp. 1-6.

[5] Y. Liu, L. Zhang, Y. Yang, L. Zhou, L. Ren, F. Wang, R. Liu, Z. Pang, and M. J. Deen, "A novel cloud-based framework for the elderly healthcare services using digital twin," *IEEE access*, vol. 7, pp. 49088-49101, 2019.

[6] W. Chen, W. Hong, H. Zhang, P. Yang, and K. Tang, "Multi-fidelity simulation modeling for discrete event simulation: An optimization perspective," *IEEE Transactions on Automation Science and Engineering*, vol. 20, no. 2, pp. 1156-1169, 2022.

[7] T. Moyaux, Y. Liu, G. Bouleux, and V. Cheutet, "An agent-based architecture of the digital twin for an emergency department," *Sustainability*, vol. 15, no. 4, p. 3412, 2023.

[8] M. Dugast, G. Bouleux, O. Mory, and E. Marcon, "Improving health care management through persistent homology of time-varying variability of emergency department patient flow," *IEEE journal of biomedical and health informatics*, vol. 23, no. 5, pp. 2174-2181, 2018.

[9] R. Luscombe and E. Kozan, "Dynamic resource allocation to improve emergency department efficiency in real time," *European Journal of Operational Research*, vol. 255, no. 2, pp. 593-603, 2016.

[10] G. Bouleux, H. B. El Haouzi, V. Cheutet, G. Demesure, W. Derigent, T. Moyaux, and L. Trilling, "Requirements for a digital twin for an emergency department," in *Proceedings of SOHOMA 2022*. Bucharest, Romania: Springer, 2022, pp. 130-141.

[11] S. Chaabane and F. Kadri, "Toward a proactive and reactive simulation-based emergency department control system to cope with strain situations," *Operations Research and Simulation in Healthcare*, vol. 1, no. 1, pp. 123-152, 2021.

[12] J. G. Chase, C. Zhou, J. L. Knopp, G. M. Shaw, K. Näswall, J. H. Wong, S. Malinen, K. Moeller, B. Benyo, Y. S. Chiew et al., "Digital twins in critical care: what, when, how, where, why?" *IFAC-PapersOnLine*, vol. 54, no. 15, pp. 310-315, 2021.

[13] B. Morel, G. Bouleux, A. Viallon, M. Maignan, L. Provoost, J.-C. Bernadac, S. Devidal, S. Pillet, A. Cantais, O. Mory et al., "Evaluating the increased burden of cardiorespiratory illness visits to adult emergency departments during flu and bronchiolitis outbreaks in the pediatric population: Retrospective multicentric time series analysis," *JMIR Public Health and Surveillance*, vol. 8, no. 3, p. e25532, 2022.

[14] M. Harzi, J.-F. Condotta, I. Nouaouri, and S. Krichen, "Scheduling patients in emergency department by considering material resources," *Procedia computer science*, vol. 112, pp. 713-722, 2017.

[15] K. W. Tan, W. H. Tan, and H. C. Lau, "Improving patient length-of-stay in emergency department through dynamic resource allocation policies," in *2013 IEEE International Conference on Automation Science and Engineering (CASE)*. Madison, WI, USA: IEEE, 2013, pp. 984-989.

[16] D. Duma and R. Aringhieri, "Real-time resource allocation in the emergency department: A case study," *Omega*, vol. 117, p. 102844, 2023.

[17] H. Hassani, X. Huang, and S. MacFeely, "Impactful digital twin in the healthcare revolution," *Big Data and Cognitive Computing*, vol. 6, no. 3, p. 83, 2022.

[18] R. Alvalu, S. Mudrakola, A. Kaladevi, M. Sandhya, C. R. Bhat et al., "The novel emergency hospital services for patients using digital twins," *Microprocessors and Microsystems*, vol. 98, p. 104794, 2023.
[19] A. Haleem, M. Javaid, R. P. Singh, and R. Suman, "Exploring the revolution in healthcare systems through the applications of digital twin technology," *Biomedical Technology*, vol. 4, pp. 28–38, 2023.

[20] G. Bouleux, E. Marcon, and O. Mory, "Early index for detection of pediatric emergency department crowding," *IEEE journal of biomedical and health informatics*, vol. 19, no. 6, pp. 1929–1936, 2014.

[21] T. A. de Queiroz, M. Iori, A. Kramer, and Y.-H. Kuo, "Dynamic scheduling of patients in emergency departments," *European Journal of Operational Research*, vol. 310, no. 1, pp. 100–116, 2023.

[22] W. Li, Z. Sun, and L. J. Hong, "Who is next: Patient prioritization under emergency department blocking," *Operations Research*, vol. 71, no. 3, pp. 821–842, 2023.

[23] S. B. Othman, H. Zgaya, S. Hammadi, A. Quilliot, A. Martinot, and J.-M. Renard, "Agents endowed with uncertainty management behaviors to solve a multiskill healthcare task scheduling," *Journal of biomedical informatics*, vol. 64, pp. 25–43, 2016.

[24] Y. Liu, T. Moyaux, G. Bouleux, and V. Cheutet, "Hybrid simulation modelling of emergency departments for resource scheduling," *Journal of Simulation*, pp. 1–16, 2023.

[25] T. Moyaux and E. Marcon, "Cost of selfishness in the allocation of cities in the multiple travelling salesmen problem," *Engineering Applications of Artificial Intelligence*, vol. 89, p. 103429, 2020.

[26] S. J. Traub, C. F. Stewart, R. Didehban, A. C. Bartley, S. Saghafian, V. D. Smith, S. M. Silvers, R. LeCheminant, and C. A. Lipinski, "Emergency department rotational patient assignment," *Annals of emergency medicine*, vol. 67, no. 2, pp. 206–215, 2016.

[27] D. Cao, M. Chen, and G. Wan, "Parallel machine selection and job scheduling to minimize machine cost and job tardiness," *Computers & operations research*, vol. 32, no. 8, pp. 1995–2012, 2005.

[28] R. G. Smith, "The contract net protocol: High-level communication and control in a distributed problem solver," *IEEE Transactions on computers*, vol. 29, no. 12, pp. 1104–1113, 1980.

## VII. BIOGRAPHY SECTION

<figure><img src="image_12.png" /></figure>

**Jessica Florencia** received her B.Eng degree in Industrial Engineering from Swiss German University (Indonesia) in 2020, and recently graduated with M.Sc. degree in Production, Logistics, and Procurement Management in IMT Atlantique (France) and M.Eng. degree in Industrial Engineering in Universitas Indonesia. She is currently a PhD student in DISP laboratory at INSA Lyon, France and working on the research of agents exploration for decision-making improvement in ED of the hospital.

<figure><img src="image_13.png" /></figure>

**Dr. Thierry Moyaux** received the Engineering and M.Sc. degrees from Aix-Marseille Université in 2000, and the Ph.D. degree from Université Laval, Canada, in 2004. Then, he was a Research Assistant at the University of Liverpool, U.K. He joined INSA in October 2007 on a temporary position. He has been an associate professor since September 2009 at INSA in which he teaches in the department of Industrial Engineering and does research in the DISP laboratory on when and how to hybridize Operations Research with MultiAgent Systems.

<figure><img src="image_14.png" /></figure>

**Dr. Lorraine Trilling** is an associate professor in the Industrial Engineering Department at INSA Lyon, France. She received a Ph.D. from INSA Lyon in 2006 for her work on decision support for staffing and scheduling shared Human Resources in healthcare systems. She has taken part in several research projects on improving emergency department operations. Her current research interests focus on optimization methods for resource scheduling, vehicle routing problem, with application in healthcare and reverse logistics.

<figure><img src="image_15.png" /></figure>

**Dr. Guillaume Bouleux** received his MSc. and PhD degrees in 2005 and 2007 respectively. He has been an associate professor (HDR) at the University of Saint Etienne since 2008. He joined the DISP laboratory, INSA-Lyon in 2014 and is the co-leader of the Operations Optimization Management team. Since 2014, his main focus has been on healthcare systems, with a particular interest in emergencies, but also on humanitarian crisis management. He is or has been the scientific (co)-leader of public national research projects funded by the Ministry of Health (HCSP, DGOS), Auvergne-Rhône Regional Health Agency (ARS) or French research agency (ANR).

<figure><img src="image_16.png" /></figure>

**Prof. Vincent Cheutet** received in 2006 a Ph.D. degree in the field of mechanical engineering from both the Grenoble-INP engineering school (France) and the University of Genoa (Italy). From 2007 to 2014, he worked at SUPMECA Paris (engineering school) as assistant professor. He became full professor in 2014 at INSA Lyon and joined the DISP Laboratory. He is currently director of this laboratory. His main research topics are industrial information systems (especially PLM and ERP), knowledge management and Digital Twins of production systems.