# Deep Reinforcement Learning for Selection of Dispatch Rules for Scheduling of Production Systems
## 3 Method
### Scheduling Problem Formulation  
The production scheduling problem under study refers to job allocation to machines (Figure 1), in such a sequence that results in an optimal plan regarding a predefined objective. In this work, the objective of the scheduling algorithm will be the production makespan's minimization. For the scheduling problem model, the parameters of a production system should be defined. Moreover, the scheduling parameters and variables are defined below.  
* Product types: A set of product type IDs is denoted as \(P=\{P_{1},P_{2},\ldots,P_{i}\}\), with \(i=1,2,\ldots,maxP\), where \(maxP\) is the maximum number of product types that the production system can produce.
* Process plan: Each product type has a sequence of processes that define its process plan. Let us denote this set of processes as \(H_{i}=\{P_{T1i},P_{T2i},\ldots,P_{Tki}\}\), with  
Figure 1: Overview of scheduling problem under study.  
\(k=1,2,\ldots,maxH_{Pi}\), where \(maxH_{Pi}\) is the maximum number of processes in a product type \(P_{i}\) process plan. For the various product types, let us denote \(H=\{H_{1},H_{2},\ldots,H_{h}\}\), with \(h=1,2,\ldots maxP\).
* Resources: Let us denote the list of resources in a production system as \(M=\{M_{1},M_{2},\ldots,M_{r}\}\) with \(r=1,2,\ldots,maxM\), where \(maxM\) is the maximum number of resources in the production system.
* Suitability and processing times: Let us denote the processing time of a process for a resource as \(I(M_{r},Pr_{ki})\geq 0\). If a process \(Pr_{ki}\) cannot be executed by a resource \(M_{r}\),j, then \(I(M_{r},Pr_{ki})=0\). In this way, the suitability of tasks for resources is also defined.
* Tasks: Let us denote as a task \(T_{b}\), with \(b=1,2,\ldots,maxT\), where \(maxT\) is the maximum number of tasks \(T_{b}\) in a scheduling problem. For this scheduling problem, a product from a certain product type to be produced is represented as a task \(T_{b}\).
* Jobs: Let us denote as a job \(J_{c}\), with \(c=1,2,\ldots,maxJ\), where \(maxJ\) is the maximum number of jobs \(J_{c}\) in a scheduling problem. For this scheduling problem, a certain quantity of identical products to be produced is represented as a job \(J_{c}\).
* Order: Let us denote as an order \(O_{g}\), with \(g=1,2,\ldots,maxO\), where \(maxO\) is the maximum number of orders \(O_{g}\) in a scheduling problem. For this scheduling problem, a certain quantity of jobs \(J_{c}\), regarding a specific product, is represented as an order \(O_{g}\).
* Scheduling workload: The scheduling workload consists of all the orders \(O_{g}\), tasks \(T_{b}\) and jobs \(J_{c}\) that need to be scheduled. The workload follows a hierarchical structure. The scheduling solution refers to the jobs' allocation to resources.  
The above-mentioned parameters formulate the production system and the scheduling variables that need to be considered when formulating the scheduling input, as well as the production environment model. Apart from the above parameters, any other production constraints should be defined. For this work, the following constraints are defined.  
* Constraint 1 (C1): Each machine can perform at most one process at a time.
* Constraint 2 (C2): A job's tasks should be processed by the same set of machines. This means that the tasks of a particular job cannot be processed by different workstations.
* Constraint 3 (C3): The transportation times and setup times are negligible.