# Deep Reinforcement Learning for Selection of Dispatch Rules for Scheduling of Production Systems
## 3 Method
### Reinforcement Learning Approach
#### 3.2.3 State  
The environment where the agent is trained should be adequately explored. Thus, the modeling of the environment from the agent's perspective is one of the most important aspects in such an ML approach. However, the environmental parameters that can be recognized by the RL agent should not consist of every asset in detail in a production system. The agent, in order to perform well and with stability and generalize, needs to identify the production status and the variables that matter. Hence, the overall status of the production at time \(t\) should be mapped in some predefined parameters that formulate the state \(s\) at time \(t\). This state will be given as input to the DRL agent to calculate the output, which is the next action \(a\) in the environment. State \(s_{t}\) is a vector of features representing the production status at a time \(t\). The state consists of information regarding the following features:  
* Jobs status--job IDs, types, the remaining quantity to be produced, the remaining cumulative processing time, the processing times and the suitability;
* Resource status--the available and unavailable resources (e.g., due to breakdown or performing another process). In a production environment, the state space is huge, especially when uncertainties are included. Thus, it is important to ensure sufficient exploration and training to reduce the state space without missing any important information. The state vector consists of \(maxJ\) sub-vectors, where \(maxJ\) is the maximum number of jobs defined in the scheduling problem requirements. Each sub-vector has the following parameters. * Job ID: In the workload, there are orders \(O_{g}\) with jobs \(J_{c}\) and tasks \(T_{b}\), following a hierarchical structure. A job ID is an integer value representing a job \(J_{c}\). * Job type ID: A job type ID is an integer value representing the product type \(P_{i}\) of the job \(J_{c}\). * Job remaining workload: An integer value representing the number of remaining tasks of a job to be executed. Let us denote it as \(Jrw_{c}\). * Job minimum cumulative processing time: Each resource that can perform the tasks of a job has its own processing time. The minimum cumulative processing time is a float value representing the cumulative processing time of the tasks of a job, if this job is processed by the resource with the minimum processing time. Let us denote this as \(Imin_{c}\). * Job maximum cumulative processing time: Each resource that can perform the tasks of a job has its own processing time. The maximum cumulative processing time is a float value representing the cumulative processing time of the tasks of a job, if this job is processed by the resource with the maximum processing time. Let us denote this as \(Imax_{c}\). * Job processing time and suitability: A product type has one processing time per resource. If the tasks \(T_{b}\) of a job \(J_{c}\) are not suitable for the resource \(M_{r}\) or the resource \(M_{r}\) is unavailable, the value is 0. Thus, it is a list of values with a length equal to \(maxM\).