# Lane following Learning Based on Semantic Segmentation with Chroma Key and Image Superposition
## 3 Materials and Methods
### Solution Design
#### 3.1.3 Operating Procedure  
The two main tasks of this problem are feature extraction from images and the use of these features in the process of learning to drive. A camera placed at the front of the vehicle provides a stream of images as input for the automated driving system. This system must extract the features corresponding to the road lines and then use them to control the steering of the vehicle. The expected output of the driving system is the required steering angle. Both tasks are solved using artificial neural networks trained with supervised learning. Figure 2 shows a diagram of this system. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Chroma** & \(\mathbf{H_{min}}\) & \(\mathbf{H_{max}}\) & \(\mathbf{S_{min}}\) & \(\mathbf{S_{max}}\) & \(\mathbf{V_{min}}\) & \(\mathbf{V_{max}}\) \\ \hline Green & 69 & 89 & 123 & 255 & 85 & 255 \\ Red 1 & 160 & 179 & 81 & 255 & 91 & 255 \\ Red 2 & 0 & 16 & 81 & 255 & 91 & 255 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Boundary values used for the chroma masks. Figure 1: Components of the complete working environment. Depending on the required task, the driving circuit will be placed either in the chroma scenario (**left**) or in the target scenario (**right**). The driving circuit can be placed in either stage when needed. The usual alternatives for dealing with these tasks are using either real scenarios or computer-simulated scenarios. * In a real scenario, accurately segmenting the lines or other elements is complicated, and errors occur that can compromise the results. * In a simulator, a generative method can be applied, creating a scenario in which the situation of the relevant elements (the lines) is precisely known [15]. However, in this case, it will be more difficult to achieve realistic images. Another usual approach is to rely on both resources: training a neural model with data provided by a simulator and testing it on real-world scenarios [15; 16; 17; 38]. Our proposal is to apply ideas from both cases. Instead of using a simulator to generate scenarios, we rely on an artificial scenario that simplifies the segmentation of the road lines, which can be combined with an indoor scenario. The whole training process is performed in four phases (Figure 3 illustrates this process):  
1. In the first phase, the road will be placed in the chroma scenario, and a human operator will drive the model car within it while the front camera takes a sequence of images. The image stream is processed in the external computer by segmenting the road and the lines from each image using the chroma key technique. This allows the generation of a dataset with the extracted road and lines. 2. In the next phase, the model car will be driven in a roadless target scenario to provide a stream of images. The images with segmented road and lines from the previous phase will be superimposed on these new images, creating an augmented scenario. The superposed images paired with the images of the extracted lines will form a new dataset that will serve to train an Autoencoder, which is expected to learn to perform semantic segmentation on the images from the working scenario. 3. In the third phase, the car is placed in the target indoor scenario (as shown in Figure 1) that includes the road, and the human operator must drive the model car again. The encoder part of the trained Autoencoder model will process the image stream from the front camera, giving at the output a set of values from the latent vector that describes the road lines. This time, the aim is generating a dataset that associates the values of the latent vector with the steering angle recorded while the operator is driving. 4. In the last phase, an MLP is trained with the dataset generated previously. The combined model with the trained encoder and MLP is then transferred to the Single Board Computer (SBC) in the model car, allowing it to drive automatically. Figure 3: Processing flow for generating the superposed images. The image (**a**) is captured in the chroma stage.