# Biologically Inspired Visual System Architecture for Object Recognition in Autonomous Systems
## 4 The Proposed Architecture
### Reinforcement Learning Module  
Recent successes of reinforcement learning techniques matured into algorithms that yields state-of-the-art performance that in some cases match and even exceed human capabilities in a wide variety of practical problems[42; 43; 44]. However, much of this increase in performance is due to breakthroughs in deep neural networks that are in the core of deep reinforcement learning models. In addition, deep reinforcement learning networks require enormous amounts of trials [44; 45] that are independent and identically distributed, to avoid catastrophic forgetting [46]. As a result, the training and inference phases are separated, which prevents from deployed agents to learn from ongoing experience and adapting to changes in their environment.  
Adding reinforcement learning abilities to the robotic agent reduces the burden of the information gathering for the training phase, while enabling it to improve its classifier using the visual data encountered during its operation. For example, in the experiments done using the VAP model, the training was done on an open source dataset. This dataset, like others, contains mainly images taken by a photographer that was located on the same surface as the object. However, the experiments of the VAP model were conducted using an elevated camera with a top view of the objects. Therefore, the initial classification performance of the VAP model was lower than expected. Nevertheless, as the reinforcement module accumulated information on the objects in the scene, the VAP model became more accurate.  
An object moving around the scene may be varied visually in a number of ways, due to the new viewing conditions in each frame. For the sake of terminology, each specific realization variation of an object, which may result in a different feature vector, is called an instance. For each object tracked by the Object-file mechanism, the reinforcement learning mechanism is looking for instances where there is a gap between the instantaneous visual feedback received from the CNN in the current frame and the stable Object-file semantic description of the objects. Meaning, after the Object-file has accumulated enough instances to reach high confidence level recognition of the object, the reinforcement learning mechanism compares all new incoming instances from the ITC mechanism to this representation of the object, as can be seen in Figure 6. When a new instance creates a false prediction, it evokes a learning process, which uses the features of this instance to refine the model.  
The proposed model uses an SVM coupled with a CNN as part of a multistage process. A CNN, which is an efficient mechanism for learning an invariant features space from images, is used as a first stage. The deep CNN is first trained to learn good invariant representations and then used on incoming frames as a feature extraction mechanism. As a second stage, the feature vectors are fed into a kernel SVM, which produces good decision surfaces by maximizing margins. It has been shown that this multistage process and its variations usually result in similar performance [47; 48; 49] to the fully connected layers of a CNN.  
Working in a high-dimensional feature space increases the generalization error of support vector machines, as there are not enough data points in the training set to accurately define the boundaries between the classes. Our reinforcement learning model detects and stores samples in video sources that: (a) belong to a specific class but were on the wrong side of the hyper plane and were wrongly classified; or (b) close to the hyper plane where there are naturally fewer samples as it is far away from the mean of the distribution of the class. It then uses these samples to refine the SVM decision function when there is no more movement in the scene.