# ESCAPE: Evacuation Strategy through Clustering and Autonomous Operation in Public Safety Systems
## 6 Numerical Results
### Pure Operation of the Framework  
Figure 2(a) presents the evolution of probabilities \(pr_{m,s_{m}},s_{m}\in S_{m}=\{0,1\}\), i.e., the probabilities to evacuate and not evacuate as they are determined by the minority game (DEMG algorithm) of two indicative evacuees who want to go to the same evacuation route \(e\) as a function of the minority game's iterations required to converge to the PNE point. It is clear that there initially is a common starting point for all probabilities (i.e., assumed to be equal to 0.5), but afterward the minority game very quickly converges in terms of iterations (lower horizontal axis), as well as in terms of real execution time (upper horizontal axis), this way guiding the first evacuee (User1) to evacuate and the second evacuee (User2) not to evacuate. The corresponding probabilities for the aforementioned decisions for both evacuees are equal to 1, while the complementary probabilities are equal to 0, as the non-co-operative minority game converges. Figure 2(b) illustrates the required number of iterations of the minority-game algorithmic component (DEMG algorithm) in order to converge as a function of exponential learning step \(\gamma\). It is clearly observed that, for larger values of \(\gamma\), the time it takes the non-co-operative game to converge decreases, while for smaller values it increases. The aforementioned behavior is observed due to the fact that, for large values of \(\gamma\), each evacuee \(m\) deterministically selects best strategy \(s_{m}\) (evacuate or not evacuate) with the highest accumulated score, while for small values each evacuee more explores the available alternatives. Therefore, in the rest of our analysis, we consider a relatively small value of exponential learning parameter \(\gamma\), i.e., \(\gamma=0.8\), toward allowing evacuees to better investigate the available choices. In Figure 2(c) we consider two distinct evacuees (User3 and User10) from the set of evacuees and evaluate their action probability vectors (updated according to Equations (2a) and (2b)), as a function of the iterations of the reinforcement-learning component. Specifically, we observed that the action probability for only one evacuation route per each evacuee increases, eventually reaching 1.0, while the other probabilities for all remaining evacuation routes decrease until they reach a value equal to 0.0. Such observations reconfirm that the reinforcement-learning component successfully converges to a feasible solution. In the examined scenario, we notice that User3 wants to go to Evacuation Route 1, while User10 wants to go to Evacuation Route 4. Figure 2(d) illustrates the necessary reinforcement-learning component iterations in order to converge (blue line) and the corresponding average reward function with respect to the number of users (red line) as a function of step-size learning parameter \(b\) of the reinforcement-learning component. It is clear that, for large values of \(b\), convergence time as well as the average reward function decreases. This occurs because small \(b\) values enable evacuees to better explore the disaster area's conditions and eventually make better choices regarding which evacuation route they want to follow, like, for instance, exploring evacuation routes that are not so congested as better options. Thus, in the latter case, more time is needed for the reinforcement-learning component to converge, but evacuees experience a superior reward. Considering the tradeoff between evacuees' perceived reward and the corresponding time that the ESCAPE algorithm needs to converge, in the following, unless otherwise stated, we consider the stochastic-learning component's step-size parameter \(b=0.7\), so evacuees enjoy a relatively superior reward and, at the same time, make fast decisions. Figure 3: Pure operation of the proposed framework. Figure 3e presents the cluster size of humans per evacuation route as a function of iterations (lower horizontal axis) and time (upper horizontal axis) that the reinforcement-learning component needs to converge.