## 5 Build an Optimal Security Strategy
### Framework of RODE  
Algorithm 1 gives the framework of the mixed strategy-based differential evolution algorithm (RODE). ```
0: population N, maximum number of iterations G
0: optimal protection strategy
1: Initialize the population randomly, actions, states, Q table and other parameters;
2: Opposition-based learning: Find the reverse solution for the initial population
3: Calculate the initial population objective function
4:for\(gen<G-1\)do
5:for e = 1:Np do
6: choose action, calculate F;
7: Mutation: DE/best/1
8:\(V_{i}^{t}=X_{best}^{t}+F(X_{r2}^{t}-X_{r3}^{t})\)
9: Crossover:
10: r = randi([1,D],1,1)
11:for\(n=1\to D\)do
12:\(cr=rand\)
13: CR(e) = 0.9 - 0.5 * (gen/G);
14:if (cr \(<=\) CR(e)\(||\)(n == r)) then
15: u(e,n) = v(e,n)
16:else
17: u(e,n) = x(e,n);
18:endif
19:endfor
20: Selection:
21: Compute objective function of NP new candidates \(u_{e}\)
22: Calculate Reward
23:if\(f(u_{e})\leq f(x_{e})\)then State = 2
24: Opposition-based learning:
25:if\(cr<\{0.1,0.2,0.3,0.4\}\)then u(e) = a + b - u(e)
26: x(e) = u(e)
27:else
28: x(e) = u(e)
29:endif
30:else
31: State = 1
32:endif
33: update Q-table
34:endfor
35:endfor

## 6 Experiment and Discussion

### Experimental Scene

The experimental scene in this paper is a water distribution system, in which there are three PLCs, which correspond to the water pump, pipeline and water tanker in the virtual system, which simulates the real industrial control system, as shown in Figure 6.

### Generating a Bayesian Attack Graph

We use Nessussuss [36] to scan the vulnerabilities of the water distribution system, and according to the scores in Table 1, Formula (2) is used to calculate the successful probability of exploiting each vulnerability. Then, the network configuration and network connectivity information of the vulnerability are input into the input.P file of the MulVAL tool, and finally, the attack graph is obtained. The visualization of the attack graph is as shown in Figure 7.

Figure 6: Experimental scene. Figure 7: Attack Graph. ### Attack Benefits and Protection Costs

The prior probability of attacking all nodes in the graph and the cost of the security strategy can be shown in Table 4 and Appendix A Table 11.

### Result Analysis

In this paper, we set the fitness function \(\alpha\) to 0.5, which means that the attack benefit and protection cost are equally weighted. Cost B is set to have no upper limit. The strategy set selected by the evolutionary algorithm is used as the input of the fitness function, and the fitness value is obtained. In order to prove the effectiveness of the proposed algorithm, we compare four other evolutionary algorithms of the same type. They are differential evolution algorithm (DE), particle swarm algorithm (PSO) [37], particle swarm algorithm based on reinforcement learning (RLPSO) [38], differential evolution algorithm based on reinforcement learning (RLDE) improved in this paper, and differential evolution algorithm based on mixed strategy proposed (RODE) in this paper. The relevant parameters of its setting are shown in Table 5. We uniformly set the initial population number of these five algorithms to 200 or 20, and we set the number of iterations to 20 uniformly. Figure 8 shows the experimental results when the initial population is 200. It can be seen that when the initial population is relatively sufficient, most of the algorithms have achieved good results, among which DE, RLDE and RODE converge to the optimal solution of 459.4463. RLPSO and PSO converge to 532.6584 and 478.0463, respectively. When the initial population is 200, although DE, RODE, and RLDE converge to the optimal solution, the convergence speed of RODE and RLDE is obviously better than that of DE.