# Multi-Objective Optimization of Cascade Blade Profile Based on Reinforcement Learning  
[PERSON]  
1 Department of Aeronautics and Astronautics, Fudan University, Shanghai 200433, China;  
[EMAIL_ADDRESS] (S.Q. ); [EMAIL_ADDRESS] (S.W. ); [EMAIL_ADDRESS] (L.W. );  
[EMAIL_ADDRESS] (C.W. );  
2 AECC Commercial Aircraft Engine Co., Ltd., Shanghai 200241, China; [EMAIL_ADDRESS] 2  
[PERSON]  
1 Department of Aeronautics and Astronautics, Fudan University, Shanghai 200433, China;  
[EMAIL_ADDRESS] (S.Q. ); [EMAIL_ADDRESS] (S.W. ); [EMAIL_ADDRESS] (L.W. );  
[EMAIL_ADDRESS] (C.W. );  
wang  
a more global, efficient, and intelligent multi-objective optimization method based on accumulated design experience is proposed in this paper. The optimization methods used in aerodynamic shape design are conventionally gradient-based methods and stochastic methods. Gradient-based methods rely on the gradient of the objection function, usually converge rapidly, and have low requirements for the computational cost. Adjoint methods are typical gradient-based methods and have been used in many aerodynamic shape optimizations [5; 6; 7], such as turbomachinery optimization [8; 9; 10]. However, gradient-based methods easily get trapped in local optima and perform poorly for strongly non-linear or discontinuous objective functions [11]. With the development of computing power, stochastic methods are more and more widely used in aerodynamic shape optimization, e.g., Particle Swarm Optimization (PSO) [12], Simulated Annealing (SA) [13], Genetic Algorithm (GA) [14], and Evolutionary Algorithm (EA) [15], which are good at finding global optima, in many multi-objective optimizations [16; 17; 18; 19]. Multi-objective stochastic methods have been used in the optimization of turbomachinery (Multi-objective Genetic Algorithm (MOGA) [20; 21; 22], Multi-objective Evolutionary Algorithm (MOEA) [23; 24; 25]). However, stochastic methods need large numbers of stochastic cases to decide evolutionary trends via probability-based selection, whereas the existing data is not capable of being utilized in the above process to determine optimization direction. It results in the drawbacks of stochastic methods, like its slow convergence and high computational cost (especially for high-fidelity computational fluid dynamics (CFD)). These limitations are overwhelmingly disturbing in global multi-objective optimization with a larger number of design parameters. In recent years, many artificial intelligence techniques have caught the attention of researchers. Reinforcement Learning (RL) is one such approach that interacts with the environment according to policies, learns from experience, and improves policies. For example, the RL algorithm, Q-learning [26], computes the \(Q\) value of every state and action pair and stores it in one table. This value-based algorithm completes each iterative calculation by looking up the table, so it is suitable for the conditions wherein the space of state and action are discrete and the dimensions are not high. The experience-based learning feature of RL makes it advantageous for optimization application [27]. In its application on aerodynamic shape design, the optimizer modifies the shape according to the design policy as an intelligent designer, interacts with the environment (obtaining aerodynamic data through CFD simulation or surrogate model), evaluates the policy according to the aerodynamic data, learns design experience, and updates the design policy. Some RL algorithms have been developed and applied in aerospace optimization. The application of RL on the direct optimization of aerodynamic shape was presented by [PERSON] et al. [28] and led to new generic optimization strategies. [PERSON] et al. [29] exploited an RL algorithm to learn the policy for drag reduction of supercritical airfoils, where the mean drag reduction of 200 airfoils had been effectively accomplished. Combined with Deep Learning, RL algorithm can extract features from data in high-dimensional state spaces.