# Advancing Additive Manufacturing Through Machine Learning Techniques: A State-of-the-Art Review
## 5 Applications of Reinforcement Learning in Additive Manufacturing
### Quality Control  
In recent years, researchers have increasingly focused on using RL to optimize process parameters in AM processes. Table 9 summarizes several recent works, with further details provided below. [PERSON] et al. proposed a model-based RL and correction framework to control multi-layer and multi-bead (MLMB) deposition in robotic WAAM [173]. The model-based RL approach in this study was utilized to learn an optimal relationship between process inputs and print outputs. To establish the transition from one state to another after taking action (i.e., changing process inputs), the authors collected a dataset along the print path by discretizing it into waypoints with local states and actions. A Gaussian process regression model was then trained as the kriging dynamics function. Additionally, the quality of the layer's surface was periodically assessed, and the necessary corrections were made if needed. The authors experimentally demonstrated and evaluated this learning-correction framework on a robotic WAAM system, testing it with bronze and stainless steel materials. Results showed that the standard deviation of the surface height of each printed layer was significantly reduced compared to using single-bead parameters during the WAAM processes. [PERSON] et al. proposed a novel analysis approach combining CNN and RL methods to monitor product quality in LPBF AM processes [174]. They employed image-based surface roughness estimation, acquiring high-dynamic-range optical images of the product's top surfaces. These images were processed through CNNs to classify surface roughness and defective areas into five categories, ranging from very low roughness to surface distortion. Transfer learning was utilized to enhance the ML model's performance. Subsequently, a model-based RL method was used to train an agent to learn the optimal policy for selecting the best LPBF process parameters based on a given state during the AM process. This study employed an MDP framework, where the applied laser power, scan velocity, mean surface roughness, and percentage of the defective area represented an MDP state. The action space consisted of process parameters, specifically laser power and scan velocity combinations to be applied in the next layer. A random forest algorithm was also used to approximate system dynamics, serving as the transition probability function in this MDP framework for LPBF processes. The agent received rewards based on the percentage of surface defects at each state, with a higher positive reward given for a smaller defective surface area. The optimal policy was learned to maximize accumulated rewards to ensure product quality. [PERSON] and [PERSON] presented a DRL framework to derive a versatile control policy for minimizing the likelihood of melting defects during LPBF processes [175]. In this study, they employed a model-free and policy-based RL method, PPO, in which the policy was directly optimized. The state space was defined by sequential observations of the temperature field, including nine heat maps of the local temperature distribution around the laser's current position. These heat maps were numerically encoded and fed into a fully connected neural network as the policy network to predict an action. The action space comprised various velocities or powers of the laser, while the reward was defined as the absolute error between the target melt depth and the current depth. The algorithm was trained on a simulator, acting as a virtual environment for the agent to interact in. The simulation involved modeling the heat conduction of a moving heat source, approximating the laser, within a rectangular domain. Assumptions included considering only conduction models of heat transfer, assuming thermal properties to be temperature-independent, and treating the powder bed as a solid continuum. The simulation results indicated that the errors could be reduced by up to 91%.