# Advancing Additive Manufacturing Through Machine Learning Techniques: A State-of-the-Art Review
## 5 Applications of Reinforcement Learning in Additive Manufacturing
### Quality Control
Additionally, the authors discussed the potential for extending this method to experimental applications through the NIST AM metrology testbed [176]. Recently, [PERSON] et al. developed another DRL framework to enhance the uniform temperature distribution during the LDED of nickel-based alloys [177]. They first created a fast and efficient temperature simulation model of the deposition process, reducing computational costs while maintaining AM processing speed. This simulation model assumed a straight-line movement of the laser beam and employed the Rosenthal equation to evaluate temperatures at different sampling points on both sides of the product [178]. In their RL framework, the continuous state space was represented by the substrate's thermal distribution, the sample's thermal distribution, and material properties. The available actions included varying laser power, scanning speed, the deposition axis, and the deposition direction. Additionally, the transition probabilities were determined by the temperature simulation model. They employed the PPO algorithm, a policy-based RL method. Specifically, a state, presented by a three-dimensional tensor, was fed into a CNN for feature extraction before applying critic and actor networks to update the policy. Notably, after the optimal policy was learned through simulations, it was evaluated in an LDED processing environment. The results demonstrated that the derived policy improved temperature uniformity in the products, thereby enhancing their hardness. [PERSON] et al. introduced an RL methodology transformed into an optimization problem for metal AM processes to ensure repeatability, control material microstructure, and minimize product defects [179]. They proposed an off-policy RL framework based on Q-learning, a value-based RL method, for the agent to learn optimal process parameters, which included combinations of laser power and scan velocity to maintain a steady melt pool depth. Specifically, the state was represented by discrete process parameters, and the actions were the parameter changes. A Q-table was maintained and updated during the learning since both state and action pieces were finite. A digital twin, developed based on the Eagar-Tsai formulation, emulated the LDED environment with which the agent interacted [180]. The model calibration was conducted experimentally on an LDED system using various laser powers and scan velocities for single-track and single-layer deposits of SS316L powder on an SS304 substrate. The experimentally derived process map also served as a validation tool to evaluate the optimal process parameters learned from the RL framework. The authors investigated the effects of various hyperparameters on the learning process, including domain discretization, the exploration-exploitation tradeoff parameter, discount factor, learning rate, and number of episodes. This study emphasized RL as an alternative approach for process parameter optimization, particularly when system information or large datasets were unavailable. In another pioneering work, [PERSON] et al. employed a model-free RL method to achieve optimal defect mitigation strategies for quality assurance during FFF [181]. Their approach falls into the category of online learning-based methods, designed to update the model incrementally as new data becomes available rather than training it in batch mode with a pre-provided and fixed dataset. However, the limited number of samples for shape deviation during the AM process posed a challenge for the practical utilization of online RL. To address this challenge, the authors proposed a transfer learning-based solution called continual G-learning aimed at detecting and mitigating new defects during