# Multi-Objective Optimization of Cascade Blade Profile Based on Reinforcement Learning
## 5 DDPG-Based Optimization for Blade Profile
### Application of DDPG and Optimization  
DDPG is used in this paper to approach optimization policies. Since the optimization is to achieve the blade profile with the optimal aerodynamic parameters by modifying the geometry, it is suitable to take the geometric parameters space as the state of DDPG network, and to choose the variations of geometric parameters as the action. The hyperparameters of the DDPG network are as follows: The target smoothing coefficient is 0.005. The discounted factor is 0.99. The batch size equals 64, and the replay buffer size equals 50,000. The learning rate is selected as 0.01, to make sure the network can converge faster. The other hyperparameters are selected by referring to Reference [61,62] with little adjustment, which is common and widely-used in the applications of DDPG. The hyperparameters are selected in order to ensure the actor and critic networks can converge and the DDPG network can achieve high rewards. For multi-objective optimization, the analysis of objective space can help us better understand the relationship between optimization variables, which is directly associated with a more problem-oriented formulation of optimization objective. The aerodynamic parameter database generated to train the network can provide a large number of samples for objective space analysis. Principal Component Analysis (PCA) [63] is an effective tool to extract the main modes from objective space considered to be the essential information that may help establish a more reasonable optimization objective. For blade profiles in the profile library sampled corresponding to given \(n\), the aerodynamic parameters of \(n\) profiles achieved through CFD simulation constitute a database represented in Equation (14), where the eight columns are the total pressure loss, relative laminar flow areas, outlet flow angles of different inlet flow angles, respectively. To compare the parameters according to the principal component, the aerodynamic parameters in \(\mathbf{A}\) are normalized. To obtain the relative distribution matrix \(\mathbf{\tilde{A}}\), Each column of the data matrix \(\mathbf{A}\) is subtracted from the mean value of the column. According to Equation (15), the data from objective space systematically forms the covariance matrix \(\mathbf{P}\). The eigendecomposition is shown in Equation (16), where \(\mathbf{\Lambda}\) represents diagonal matrix of eigenvalues and \(\mathbf{U}\) represents agglomeration of eigenvectors. \[\mathbf{A}= \begin{bmatrix}\overline{\omega}^{(1)}&\widetilde{A}_{SS}^{(1)}& \widetilde{A}_{PS}^{(1)}&\beta_{21}^{(1)}&\beta_{22}^{(1)}&\beta_{23}^{(1)}& \beta_{24}^{(1)}&\beta_{25}^{(1)}&\beta_{26}^{(1)}\\ \vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\ \overline{\omega}^{(n)}&\widetilde{A}_{SS}^{(n)}&\widetilde{A}_{PS}^{(n)}& \beta_{21}^{(n)}&\beta_{22}^{(n)}&\beta_{23}^{(n)}&\beta_{24}^{(n)}&\beta_{25 }^{(n)}&\beta_{26}^{(n)}\end{bmatrix} , \tag{14}\]  
\[\mathbf{P}_{n\times n}=\frac{1}{n}\mathbf{\tilde{A}}^{T}\mathbf{\tilde{A}}, \tag{15}\]  
\[\mathbf{P}_{n\times n}=\mathbf{U}\mathbf{\Lambda}\mathbf{U}^{T}. \tag{16}\]  
Through the analysis of eigenvectors, the relationship between the parameters in the multi-objective optimization of the cascade blade can be known and help to determine the reasonable reward function of reinforcement learning method and appropriate objective function of optimization. The first and second principal component of the aerodynamic data are listed in Table 4. The first principal component represents the internal relation of outlet flow angle variations. The second principal component contains information: the decrease of total pressure loss coefficient has a strong dependence on the increase of laminar flow area. It may be inferred that the increase of laminar flow area is beneficial to the reduction of total pressure loss, and the influence of the laminar flow area of the suction surface is much greater than that of the pressure surface.