# A Review on Reinforcement Learning in Production Scheduling: An Inferential Perspective
## 4 Taxonomy of Reinforcement Learning Algorithms  
[PERSON] and [PERSON] [42] emphasize the crucial role of the three essential components in the RL process:  
* A policy determines the actions to be taken in each environmental state.
* A reward signal categorizes these actions as beneficial or detrimental based on the immediate outcome of transitioning between states.  
* A value function assesses the long-term effectiveness of actions by considering not only a state's immediate reward but also the anticipated future rewards.  
The primary objective of the agent is to optimize the cumulative reward. Consequently, the reward signal is the foundation for modifying the policy and the value function. In specific RL systems, an additional component known as a model of the environment exists, which is optional. It replicates the behaviour of the climate or enables more comprehensive assumptions about the behaviour of the environment. Models are utilized for strategic planning, which involves making decisions based on anticipation of future scenarios before their actual occurrence [24]. Model-based methods are used to handle reinforcement learning problems by utilizing models and planning, in contrast to more straightforward model-free methods. They exhibit apparent trial-and-error learning behaviour and are considered the reverse of planning [26; 27]. RL processes can be represented and analysed using a mathematical framework called MDP. The stochastic mathematical model can be formalized using a 5-tuple (s, a, p, r, y), in which the first symbol's' represents the finite collection of all possible environment states, while s\({}_{t}\)--is the state at a specific time 't'. Letter 'a' represents the set of all possible actions, while the action taken at time 't' is marked as a\({}_{t}\). Symbol 'p' refers to the transition probabilities matrix, which defines the conditional probability of transitioning to a new state's', with a reward 'r', given the current state's', and action 'a' (for all states and actions) [44]. Figure 2 shows the RL of the production scheduling cycle.  
The learning methodologies of RL algorithms or agents can be classified into two distinct categories, model-based and model free. Model-based RL is often known as indirect learning. The agent uses a predictive model to learn the control policy from the environment through a limited number of interactions. The agent then applies this model to subsequent episodes to obtain rewards [15]. Model-free RL, or direct learning, refers to learning where an agent learns to make decisions without explicitly building a model of the environment. The agent acquires knowledge of the control policy through experiential learning from the environment, employing trial and error methods to optimize rewards without relying on any pre-existing model. This approach showcases the adaptability of RL algorithms, allowing them to learn and evolve in dynamic environments [45].  
As there are many different RL algorithms, it is sensible to understand the difference among them. For this purpose, classification systems have been established to categorize them by different criteria. For instance, [PERSON] and [PERSON] [46] proposed to categorize RL algorithms based on the environment type. RL algorithms can also be classified from the perspective of policy: on-policy vs. off-policy learning [47; 48]. RL are mostly classified according to their learning approaches [15; 45]. With this regard, updated classification of RL algorithms from the perspective of learning approaches used is provided in Figure 3.  
Figure 2: RL production scheduling cycle.