# A Review on Reinforcement Learning in Production Scheduling: An Inferential Perspective
## 5 State of the Art of Reinforcement Learning in Production Scheduling  
Deterministic scheduling encompasses organizing and planning tasks under fixed parameters and known constraints. Its goal is to optimize production efficiency and minimize turnaround times. A comprehensive work [49] explores fundamental principles and advanced techniques in deterministic scheduling, providing insights into various algorithms and their applications. [PERSON]'s work [50] thoroughly reviews deterministic scheduling theory, algorithms, and practical systems, highlighting their significance in manufacturing and service operations. Flowshop scheduling involves sequencing operations on multiple machines in a fixed order to minimize makespan or total completion time. [PERSON] [51] discusses various flowshop scheduling problems, including setups, and reviews algorithms and approaches to address them. [PERSON] and [PERSON] [52] provide a seminal survey of classic and contemporary research on flowshop scheduling, covering both exact and heuristic methods. Reinforcement learning (RL) techniques have gained popularity, affirming the interest in agent-based models. Previous studies have primarily focused on using RL to solve job-shop scheduling challenges. The manufacturing sector faces challenges such as customer satisfaction, system degradation, sustainability, inventory, and efficiency, impacting plant sustainability and profitability. Industry 4.0 and smart manufacturing offer solutions for optimized operations and high quality products. [PERSON] et al. [53] integrate RL with lean green manufacturing to create a sustainable production environment, reducing environmental impact through minimized material consumption and lifecycle extension via pull production, predictive maintenance, and circular economy practices. Rigorous experimental analysis validates its effectiveness in enhancing sustainability and material reuse. Recently, significant progress has been made in using RL to tackle several combinatorial optimization problems, including production scheduling, Vehicle Routing Problem, and Traveling Salesman Problem [54; 55; 56; 57; 58; 59]. In RL, a production scheduling task can be viewed as an environment in which an agent operates, developing a policy through offline training by interacting with this environment. This approach offers a novel way to address scheduling challenges, requiring stringent real time constraints, such as dynamic job shop scheduling problems [22; 60; 61; 62; 63; 64]. Figure 3: Classification of RL algorithms. In production scheduling problems, value based RL algorithms are commonly used, including Q-learning, temporal difference TD(\(\lambda\)) algorithm, SARSA, ARL, informed Q-learning, dual Q-learning, approximate Q-learning, gradient descent TD(\(\lambda\)) algorithm, revenue sharing, Q-III learning, relational RL, relaxed SMART, and TD(\(\lambda\))-learning. In Deep Reinforcement Learning (DRL), many value-based approaches are employed, such as DQN (Deep Q-Learning Networks), loosely-coupled DRL, multiclass DQN, and the Q-network algorithm [43; 44; 65; 66; 67; 68; 69; 70; 71; 72; 73]. [PERSON] et al. [25] applied multi-agent approximation Q-learning, demonstrating its effectiveness through numerical experiments in static and dynamic environments and various flow shop scenarios. The goal was to create and implement an efficient manufacturing schedule considering realistic interactions between labor skills and adaptive machinery. [PERSON] [74] uses the DRL method to address dynamic flexible job shop scheduling, focusing on scenarios to reduce tardiness. [PERSON] et al. [75] first employed 'hierarchical multi-agent proximal policy optimization' (HMAPPO) for the constantly changing partial-no-wait multi-objective flexible job shop problem (MOFJSP) with new job insertions and machine breakdowns. Machine deterioration during production, increases operational expenses and causes workflow disruptions, necessitating costly corrective maintenance.