# Crowd Evacuation Guidance Based on Combined Action Reinforcement Learning
## 1 Introduction
### Crowd Simulation and Evacuation Guidance  
Crowd movement simulation is an essential basis for analyzing and researching crowd behavior characteristics, self-organization, and other crowd evacuation phenomena [6]. Crowd simulation research can be categorized as macroscopic and microscopic models [7]. Macroscopic models mainly examine the crowd's overall state of movement, generally using grid models, such as cellular automata [8]. For example, the data-driven crowd simulation method calculates the velocity field using the fluid dynamics method and acts on the continuum model [9], and the lattice-Boltzmann-based method detects the abnormal movement of crowds [10]. The microscopic models use dynamics methods to simulate the movement characteristics of each individual. Typical methods include social force models that introduce subjective human factors [11].  
Researchers hope to improve the evacuation efficiency in simulation research, and model crowd movements that are closer to reality. Therefore, the problem of path planning in crowd evacuation has attracted attention from researchers. Consequently, researchers have used the bee colony algorithm of swarm intelligence to improve path searching [12]. Some researchers combined multiple sensor information, and introduced a path selection method for perceiving disaster locations [13]. The path planning methods in simulation environments integrate environmental information, and then, calculate an escape path that maximizes global evacuation efficiency. In actual scenes, the survivors can only grasp the information around them with their limited vision and experience. Even if the building monitoring system can master an optimized escape route, the system requires a particular means of informing the survivors.  
To indicate the escape route, emergency escape signs are generally installed in large buildings. These emergency signs can be divided into static and dynamic guidance signs [14]. In experiments based on real scenes [15] and simulation experiments based on social force models [16], static guidance signs played an important positive role in evacuation efficiency. Unlike static guidance signs that indicate a preset evacuation route, dynamic guidance signs can display additional guidance according to real-time conditions, such as the distribution of people in a disaster scene. Studies show that, when a particular exit is unavailable, dynamic guidance signs can effectively induce people to evacuate from other exits [17]; moreover, when danger occurs, dynamic signs can guide people to avoid unsafe routes [18].  
Combining the above-mentioned crowd simulation environment, path planning algorithm, and dynamic guidance signs, researchers have developed several crowd evacuation guidance systems. These systems are based on the building environment model, realize closed-loop feedback from perceiving scene information and evacuation route planning to crowd movement guidance, and have practical value [1]. The dynamic guidance method, which is based on network flow path planning on the topological graph model [2], and the dynamic evacuation system, which uses simulated cameras to collect crowd density information and apply a real-time shortest path algorithm [3], are examples. Some studies have included real buildings and established a parallel emergency evacuation system framework, achieving great practical significance [4].  
This type of system's necessary processes include several steps such as inputting a scene layout, manually constructing a topology map or grid model, inputting model parameters according to factors such as path capacity, applying path planning algorithms, and setting dynamic guidance signs. The steps of constructing models and inputting parameters introduce high manual participation and a significant workload, possibly causing errors due to human agency and magnifying them in the subsequent steps to negatively affect the system's evacuation efficiency.