## 2 Materials and Methods  
CBR is defined as the process of reusing experiences to deal with current situations that are similar to ones solved and stored previously [10], and the foundation of the CBR system is the representation and definition of a case. We consider marine oil spill emergency response tasks in which a decision maker addresses marine oil spill accidents and makes decisions based on comparisons with historical data by using similarity measurements to identify a relevant past case. At each time step, the decision maker selects an emergency response action \(a\) from the set of legal marine oil spill emergency response actions Set \(A\) and receives feedback as a reward \(r_{t}\), which represents the result of the emergency response action at step \(t\). Note that the emergency response result depends on the entire prior sequence of actions; feedback about an action can only be received after many time steps have elapsed. Therefore, we consider sequences of actions and observations, \(s_{t}=x_{1},a_{1},x_{2},\ldots,a_{t-1},x_{t}\), and learn the actions that depend on these sequences, which represent the internal state of the marine oil spill observed by the decision maker. This state is a vector of values \(x\) representing the current status of the oil spill. All the sequences in the emulator are assumed to terminate after a finite number of time steps. This condition gives rise to a large but finite Markov decision process (MDP) [15; 16] in which each sequence is a distinct state.  
The framework of our approach to scenario-based hybrid CBR/DRL is shown in detail in Figure 1. Scenario analysis provides an approach for addressing unknown but related problems based on marine oil spill historical cases. The CBR [17; 18] method provides retention, retrieval, reuse and revision of scenario analysis results, which is formalized as a four-step process [19]. Three of these steps are implemented with the DQN algorithm.  
* Retention. Scenario analysis is employed to address marine oil spill accident uncertainties, such as spill magnitude uncertainties and the uncertainties related to spill accident evolution. Each individual historical case can be represented as a detailed \"chain of consequences\", which is named the scenario chain in this paper. Through the cluster algorithm, similar scenario instances can be merged as a typical scenario, which consequently expands the scenario and forms a branch to construct scenario trees. Through scenario analysis, marine oil spill cases are stored as scenario instances and scenario trees in the scenario library.
* Retrieval. When applying cases to train the proposed hybrid CBR/DRL model, the scenario library is considered as an environment for the agent to explore, and each marine oil spill scenario instance is regarded as a state of the environment. Thus, each instance is a vector composed of features representing the marine oil spill scenario.
* Reuse. The agent chooses the action with the highest expected value using the \(\epsilon\)-greedy strategy. With the probability of the \(\epsilon\) strategy, the algorithm chooses an action based on the available knowledge, and with the probability of \(1-\epsilon\), a random action is selected [20].  
Figure 1: Four stages of the proposed framework. DQN = deep Q-network.  
* Revision. The revision phase uses the DQN to update to the utilities Q for actions \(a\) chosen by the agent. Eligibilities represent the cumulative contributions of individual state and action combinations in previous time steps.