### Reinforcement Learning  
In the early history of reinforcement learning, there were three threads; the first one focused on learning by trial and error; the second one centred on the problem of optimal control; and the third one surged later on, based on ideas from the first two, concerned temporal-difference methods. All of them came together in the late 1980s to give birth to the modern field of reinforcement learning [15]. Nowadays, reinforcement learning has been consolidated as one of the three main machine learning paradigms, together with supervised and unsupervised learning [16; 17]. Reinforcement learning algorithms are based on an iterative learning process. The learning process is based on trial and error and the interaction between an agent and an environment [18]. This interaction is modelled as a Markov Decision Process (MDP), a concept first introduced by [PERSON] in 1957 [19]. Through this idea, the interaction is reduced to three signals: state \(s\) (the current situation of the environment); action \(a\) (operation or decision taken by the agent based on the state and its experience); and reward \(r\) (numerical feedback that the environment returns to the agent to indicate how good or bad is the action taken by the agent) [20]. Figure 1 illustrates this interaction. The objective of the agent is to maximise the accumulative reward in the long run, which entails learning the task [21]. This is reflected within the policy of the agent, \(\pi\), which determines which action is more suitable to take in each state. This policy is updated and improved as the agent interacts with the environment and gains experience [22]. The acquired knowledge is represented in a value function, which can be defined in two ways:State-value function for policy \(\pi\left(V_{\pi}(s)\right)\), which assigns the expected accumulative reward to each state as the agent will receive if the interaction starts in state \(s\) and follows the policy \(\pi\). * Action-value function for policy \(\pi\left(Q_{\pi}(s,a)\right)\)), which determines the expected accumulative reward for each action-state pair as the agent will receive if it starts the in state \(s\), takes action \(a\) and follows the policy \(\pi\) thereafter. In the learning process, the value function and the policy are updated and improved regarding each other. Under this, the exploration-exploitation problem exists, and a trade-off must be found [23; 24]. On the one hand, exploration of new actions is necessary to learn alternative paths to achieve the goal and learn the task, whereas exploitation leverages the acquired knowledge to maximise the accumulative reward [25]. On the other hand, exploitation consists of applying the knowledge learned and mostly taking the optimum action known [26]. Firstly, RL algorithms go through an exploratory phase to learn the dynamics of the environment. In this sense, there are several exploration techniques. The most common technique is a random exploration which usually gives great results, as reflected through impressive performances in self-driving cars [27], autonomous landing [28], Atari games [29], Mujoco simulator [30], controller tuning [31] and much more. There are also complex techniques, such as reward shaping [32; 33], where the algorithm designer arbitrarily modifies the agent's rewards. However, this technique highly depends on the designer's experience and knowledge of the problem. Errors in reward shaping may lead to infinite repetition of action [34] or no actions at all [35]. An extensive analysis concerning the exploratory techniques and their benefits and drawbacks can be found in [PERSON] et al. 's (2022) survey [36]. Secondly, once the agent explores the environment and learns the consequence of its actions, it passes to exploit that knowledge. However, depending on the problem handled, the agent usually maintains part of its exploratory behaviour just to ensure that the policy of actions followed is still the best.