## 1 Introduction
Reinforcement learning techniques are able to learn mapping functions directly from the environment the agent is placed in by performing actions in the environment and observing their outcomes. An agent placed in an unknown environment is faced with decision making under uncertainty and the sequential nature of states it observes. The uncertainty of moving between states can be modeled using a Markov Chain (MC). A Markov Decision Process (MDP) is a Markov Chain (MC) with external actions (\(A\)) and an agent that can act in the environment [7]. A Markov Decision Process (MDP) is a controlled Markov chain described by the 5-tuple (\(S\), \(A\), \(T\), \(p\), \(r\)) [8]. Where in this case the parameters  
* \(S\) denotes the set of states observable in the environment;
* \(A\) denotes the set of possible actions that can be performed;
* \(T\) is the set of time-steps in which decisions must be made. When a set of goal states exists the process terminates whenever these states are reached;
* \(p\) denotes the probability of transitioning from one state to another, and
* \(r\) denotes the reward function. Reinforcement learning methods are applicable when the agent does not know the transition and reward functions in advance [8]. Reinforcement learning differs from the traditional learning paradigms such as, instead of learning mapping functions by observing large samples of pre-labeled examples, the objective is to learn behavioral policies through interaction with the environment [9; 10]. The learner is not told which actions to take but must learn which actions yield the most reward by selecting an action and observing the reward [11]. In the reinforcement learning setting, an agent is placed in an environment where it can perform a set of actions \(A=[a_{1};a_{2};a_{3};...;a_{k}]\) where \(k\) denotes the number of actions possible in the environment. At each timestep \(t\) the agent observes the current state of the environment \(s_{t}\), selects an action \(a_{t}\) and receives a reward \(r_{t}\) that indicates the desirability of the action taken [12]. The task of the agent is to perform actions, observe the resulting states and rewards and learn a control policy through trial and error, which maximizes the agent's reward over time [12; 13]. Figure 1 illustrates an agent interacting with its environment. The agent observes the state \(s_{t}\) of the environment, performs an action \(a_{t}\), and receives a reward \(r_{t}\). The value of starting at an arbitrary initial state \(s_{t}\) and following a policy \(\pi\) from that state onward is defined as:  
\[V^{\pi}(s_{t})=\sum_{i=0}^{\infty}\gamma^{i}r_{t+i} \tag{1}\]  
A Siamese convolutional neural network (SCNN) is a class of neural network architecture that learns a target function that maps inputs into an output space where a distance metric approximates the semantic distance between the inputs in the embedded feature space [14]. The distance measure may be the Euclidean distance, Manhattan distance, cosine similarity, or Canberra distance [15]. In this study, we make use of the Euclidean distance between the two embedding. Semantic distance is a term often used in natural language processing to refer to the similarity or likeness of two inputs. Given two inputs \(x_{1}\) and \(x_{2}\), we expect that the semantic distance between the two should be small if \(x_{1}\) and \(x_{2}\) are similar and large if otherwise. When using SCNNs, this notion of distance is encoded in the output layer which measures the similarity between the two feature vectors produced from the convolutional layers. Each input \(x\) to the Siamese neural network is fed into a separate but identical convolutional network, making the two networks the same ensures that if \(x_{1}\) and \(x_{2}\) are from the same class they cannot be placed far away from one another in the output space [16]. The Siamese CNN is depicted in Figure 2.  
Figure 1: An agent interacting with its environment. Figure 2: Siamese convolutional neural network.