# Advancing Additive Manufacturing Through Machine Learning Techniques: A State-of-the-Art Review
## 5 Applications of Reinforcement Learning in Additive Manufacturing
### Scheduling  
[PERSON] et al. reviewed the applications of RL in manufacturing scheduling, covering areas such as job shop scheduling, flow shop scheduling, parallel machine scheduling, and single machine scheduling [184]. They highlighted the importance of carefully designing proper action and state spaces and accurately describing the scheduling environment. Designed actions could take the form of heuristics, job sequences, or scheduled operators, while state variables might encompass various types of production information. When framing the scheduling problem as a graph, the state could be defined in terms of the conditions of nodes and edges within the graph. They also discussed challenges across problem, algorithm, and application domains, emphasizing the complexity and intricacies involved in applying RL to manufacturing scheduling.  
Specifically, RL has also been applied to optimize the AM scheduling of multiple machines. [PERSON] et al. proposed an RL-iterated local search (ILS) meta-heuristic to achieve optimal solutions with low computational costs [185]. The scheduling problem considered in this study was based on the SLM process to increase the utilization of machines and decrease the average cost of setup and post-processing operations. The problem definition was similar to that of batch processing machine (BPM) problems. However, in an AM scheduling problem, the processing time of a job depends not only on the total volume of the parts but also on the maximum heights of those parts. The proposed ILS algorithm was based on genetic algorithms with the variable neighborhood search, using a Q-learning approach as the local search to improve solution performance. Such a local search could effectively and efficiently choose the best neighborhood to explore for optimal solutions. The computational experiments were conducted to illustrate that the developed method reached optimal solutions faster than other approaches, especially for large productions.  
[PERSON] and [PERSON] recently investigated two-stage assembly additive manufacturing machine scheduling problems (AMMSPs), where multiple parts were produced through job batches in the production stage before being assembled into the final products in the assembly stage [186]. The authors accounted for different specifications of each part and adopted the mixed-integer linear programming (MILP) model to define the AMMSPs as optimization problems [187]. Subsequently, they proposed an RL meta-heuristic, specifically the iterated epsilon-greedy (IEG) algorithm, to balance exploration and exploitation during the scheduling optimization. The goal of the induced optimal scheduling was to reduce the makespan by placing parts in appropriate batches during the production stage and shortening the waiting time for product assembly. To demonstrate the performance of the proposed framework, they utilized the iterated greedy (IG) algorithm as a baseline on a benchmark problem, with numerical experiments and data described in previous works [188]. The results underscored that the proposed IEG algorithm was effective, efficient, and robust in solving AM scheduling problems.  
[PERSON] et al. formulated dynamic scheduling problems as MDPs, incorporating batch processing, random order, and machine eligibility constraints [189]. They proposed an out-of-order enabled dueling deep Q-Network (O3-DDQN) approach with a reward function closely aligned with minimizing total tardiness. Experimental results demonstrated that this approach outperformed both classical scheduling rules and state-of-the-art learning methods. In another study, [PERSON] et al. applied DRL methods for multi-laser scanning planning [190]. The objective was to maximize fabrication efficiency while maintaining the quality of AM processes. Two case studies showed that the DRL methods achieved better scheduling performance than baseline methods.