# A Review: Machine Learning for Combinatorial Optimization Problems in Energy Areas
## 4 Applications in Energy Field
### Electric Power System  
The scheduling problem in electric power systems is another industrial field that draws attention. Similar to the petroleum supply chain, the electric system is also a multivariate non-linear system with high uncertainty and complexity. Nevertheless, unlike liquid or gas-formed petroleum products, this power cannot be stored due to the special physical feature of electricity. Therefore, solving such a system requires more consideration and is thus more complicated [268]. Table 10 gives a brief summary of the applications surveyed. ML-based approaches were first surveyed to solve CO scheduling problems in the electric system. [PERSON] et al. [112] proposed an improved Ptr-Net combining the deep RL algorithm to obtain end-to-end self-learning calculation for the topology control strategy of distribution network fault recovery. [PERSON] et al. [113] constructed an optimization dispatch model based on the RL framework using the Markov decision process. Then the model was trained by the AC algorithm and deep deterministic policy gradient algorithm. Then the system was divided into multiple agents concerning data interaction, and the optimization model was transformed into an MARL model. Compared to the SARL algorithm, the training process can converge more stably. Aiming at transient voltage stability in energy internet (EI), [PERSON] et al. [114] proposed a deep RL algorithm based on CNN to improve decision-making optimization to balance the power supply-demand. This algorithm has a more accurate prediction compared to conventional ML algorithms, and it also satisfies the expectation of system stability. [PERSON] et al. [115] focused on hybrid energy coordinated control optimization problems for hybrid energy storage systems. The authors designed a deep RL framework embedding a neural network model to solve the formulated decision-making problem. [PERSON] et al. [116] focused on the scheduling problem of charging stations to predict the power supply capacity required and conduct a neural network mapping model based on DL. Experiments show the capability of dealing with the sizing problem of station charging capacity. [PERSON] et al. [117] suggested a hybrid approach based on the original differential evolution (DE) algorithm, combining the ant system to tackle the optimal reactive power dispatch (ORPD) problem. Having been tested on a real system, the approach can achieve lower power losses during transmission with better results and performance compared to previous methods. [PERSON] et al. [118] aimed at optimizing the smart scheduling of energy-consuming devices, utilizing artificial neural network/genetic algorithm (ANN-GA). The best performance was found in a Levenverg-Marquardt-based learning algorithm with which the maximized use of renewable sources and reduced energy demand were achieved. With a great interest in generation expansion planning, [PERSON] et al. [119] presented an investment planning model embedding a double-layer optimization construction and three types of agents built with the Q-learning algorithm \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Problem** & **Reference** & **Year** & **Approach** \\ \hline \hline \multirow{2}{*}{
\begin{tabular}{c} c c} \multicolumn{1}{c}{c} & \multirow{2}{*}{[108]} & \multirow{2}{*}{2021} & deep RL-based \\ problem & & & algorithm + DQN \\ scheduling of & & & \\ steelmaking and & & & \\ continuous casting & & & \\ batch machine & & & \\ scheduling & & & \\ steel-making & & & \\ operation and & & & \\ scheduling & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Application in steel making. and GA, considering a change in electricity price and generation cost. The model is effective and was practicably proved by two real-world instances. Now, research on models that involve game theory are concentrated. This ML approach considers the perspective of the economic market and treats each decision maker as an individual agent. The objective is usually the total profit.