# Deep Reinforcement Learning for Selection of Dispatch Rules for Scheduling of Production Systems
## 1 Introduction  
In the era of Industry 4.0, manufacturing technologies are undergoing a revolutionary transformation, embracing digitalization and intelligent automation for a new era of interconnected and smart production [1]. In the new manufacturing revolution, called Industry 4.0, the digitization of all systems that comprise a manufacturing system is a key objective towards smart manufacturing [2, 3]. Information technology is used in various applications, including data collection, data management, knowledge development and sophisticated tasks such as the scheduling of complex production systems [4]. Scheduling is crucial in connecting customer orders to the production capacity, aiming to realize the efficient production of high-quality products. This involves determining the optimal sequence and timing for the processing of jobs with the available resources, considering complexities such as resource constraints, job characteristics and the dynamic nature of manufacturing environments, utilizing optimization methods and advanced technologies like artificial intelligence (AI) [5]. Due to its importance, production scheduling has been studied forseveral decades by many researchers. In particular, a genetic algorithm-based scheduling approach has been shown to enhance the adaptability in manufacturing job shops through the iterative optimization of task sequencing [6]. In parallel, a cloud-based cyber-physical system for adaptive shop-floor scheduling and condition-based maintenance integrates real-time data, resulting in more responsive and resilient operations [7]. The importance of synchronizing both domains for holistic optimization was addressed by a systematic review of integrated production scheduling and process control [8]. Another work reports a real-time scheduling method tailored to Industry 4.0 conditions, which addresses uncertainties like fluctuating job arrivals and machine breakdowns, ensuring robust performance in dynamic manufacturing environments [9]. Drawing on data from cyber-physical production systems (CPPS) and digital twin (DT) frameworks [10], innovative, data-driven approaches have emerged to effectively tackle scheduling problems. This study focuses on optimizing production scheduling using a deep reinforcement learning (DRL) agent for job shop problems. Trained and tested within a discrete event simulation (DES) model, and specifically designed to simulate a pilot case from the bicycle production industry, the DRL agent dynamically selects dispatching rules to achieve the best possible production makespan. More specifically, the contributions of this work are as follows:  
1. We propose Markov decision process (MDP) modeling for a DRL approach to a scheduling problem;
2. We demonstrate that deep neural networks (DNN) and reinforcement learning (RL) can be effectively used for the selection of the best dispatching policy (one rule per resource) given a scheduling problem instance;
3. We present the design, modeling and implementation of a framework for the training and testing of a DRL scheduling agent;
4. We present the validation of the DRL framework and a performance analysis of the DRL scheduling agent based on an industrial case from the bicycle industry.  
The remainder of this work is organized as follows: Section 2 reviews the development and latest research in scheduling with a focus on the applications enabled by RL, Section 3 presents the proposed approach to designing, modeling and implementing a scheduling framework, Section 4 presents the implementation of the proposed scheduling solution, and Section 5 presents the results of the experiments and the comparison of the proposed DRL scheduling agent with typical dispatch rules using data from the bicycle production industry. Finally, in Section 6, the conclusions and future directions for research are discussed.