## 4 Resilience-Oriented Emergency Responses Driven by Machine Learning
### The Reinforcement Learning-Driven Emergency Response and Recovery  
Reinforcement learning (RL), a subfield of machine learning, centres around how independent agents can make decisions in dynamic environments to maximize cumulative rewards [107]. Unlike other machine learning approaches, RL closely mirrors the learning behaviour observed in humans and other animals. Many recent RL algorithms draw inspiration from biological processes [85]. In RL, an agent interacts with an environment, typically modelled as a Markov decision process (MDP). To address this interaction, dynamic programming techniques are commonly employed [107; 108]. Due to its versatility, RL has found successful applications across various domains, including autonomous vehicles [109] and board games; notably, it played a pivotal role in the development of AlphaGo and AlphaZero [87; 110]. Post-hazard recovery, including emergency responses in urban communities, is an area where RL can be particularly effective. Its unique ability to balance long-term rewards against short-term gains makes it well-suited for such critical tasks [107]. When real-world hazards strike, the overarching objective becomes minimizing human and socio-economic losses throughout the entire event. To fulfil that objective, decision-makers must meticulously weigh the criticality and recoverability of the damaged components within those hazard-impacted CISs. Factors like accessibility, based on the real-time topology of the system, and the ease of repair play a pivotal role in their decision-making. This balancing act persists sequentially and adaptively throughout the whole emergency response campaign. RL, in theory, holds the promise of discovering optimal policies for these campaigns. These policies translate observations on the status of CIS components into specific recovery actions--such as determining which damaged components should be prioritized for restoration. However, practical implementation faces challenges. The MDPs governing post-shock emergency responses are inherently high-dimensional, rendering traditional RL algorithms inadequate for their resolution. In response to this limitation, a new subfield termed deep reinforcement learning (DRL) has emerged. Inspired by the increasing generalization capacity of deep neural networks, DRL leverages these networks as nonlinear function approximators [85] for policies (\(\pi\) (a i s) \(\{\backslash}\)displaystyle \(\backslash\)pi (a i s)\(\}\)) or other learned functions). Despite their successful application in various domains, limited research has been conducted to date on harnessing advancements in RL (including DRL) for post-hazard emergency response and urban community recovery. [PERSON] et al. [111] introduced a discrete optimization approach to the sequential decision-making on the recovery management of urban communities affected by hazards. This approach integrates approximate dynamic programming with heuristics specifically tailored for post-shock recovery planning, effectively addressing the challenge posed by the curse of dimensionality. The approach was applied to inform decision-making regarding the post-earthquake recovery of the EPSS within a testbed community coarsely modelled after Gilroy, California, United States. By considering recovery policies from both public and private institutions, the simulation demonstrated a significant enhancement in post-earthquake recovery performance. Most importantly, the approach facilitates the identification of near-optimal recovery strategies aligned with multiple objectives in an efficient manner. [PERSON] and [PERSON] [112] developed a Markovian framework for determining optimum post-earthquake restoration strategies for highway networks, with a focus on life-cycle costs (LCCs). Within this framework, seismic hazards are considered over an infinite time horizon, and the optimal restoration strategy for each individual bridge in the network is determined using a continuous-time MDP.