# Complex Color Space Segmentation to Classify Objects in Urban Environments
## 3 Image Segmentation Approach
### Cnm Architecture
Even though the VGG-16 model for a resized \(96\times 96\) path shows greater accuracy than the results shown in Table 4, \(224\times 224\) images have had a more stable performance during the training and validation phase. Additional experimental tests were performed using the ResNet CNN model, and the results are included in Table 4. In [34], the authors describe the residual blocks used for training deeper layers in the network. Using skip connections, it is possible to activate one layer and relocate its output to feed deeper layers in the network. ResNet CNN \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Architecture**} & \multicolumn{3}{c}{**Image Size (in Pixels)**} \\ \cline{2-4}  & **96 \(\times\) 96** & **128 \(\times\) 128** & **224 \(\times\) 224** \\ \hline SmallVGG & 0.73 & 0.82 & 0.87 \\ \hline VGG16-Modified & 0.92 & 0.90 & 0.91 \\ \hline ResNet150 & 0.26 & 0.81 & 0.94 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: CNN accuracy results for different image sizes. Figure 10: Results from training the CNN using \(224\times 224\) images. (a) Accuracy graph. (b) Loss graph. Figure 9: _Class mask_ obtained from a segmented image to generate patches for the category _building_. A similar process is followed for all the categories to process the training dataset. architectures are built by grouping a set of residual blocks. It is important to point out that the adder in the residual block can only be performed if both layers have the same dimension. For six categories and three different sizes of patches, we obtained an accuracy of 94% for 224 \(\times\) 224 image patches. The network model is validated using a training dataset from segmented images. Ground-truth information is not used with the validation dataset, and therefore, patches generated depend only on the areas obtained from the segmented image and the equivalent input color image. Unlike the training dataset, these patches are chosen randomly by area and do not have a predetermined category. This is represented visually in Figure 11. The patches are cropped from the color image using a fixed size \([lu\times lv]=[224\times 224]\), which is the classifier input size. A different number of patches is cropped for each segmented area depending on the size and the number of regions obtained. The fixed size of bounding boxes allows the classification of undefined categories, such as sky and road, which most object-detect methodologies cannot detect and classify. This is one of our contributions to classifiers in urban environments. Experimental tests to validate this approach use patches generated using the CamVid dataset. The classifier assigns a label and a reliability label to each patch. The output image shows different boundary frames with the brand and the reliability value corresponding to each image patch. Some results are shown in Figure 12. The CNN architecture was trained using the CityScape dataset, which has bigger images, and therefore, the process of generating patches was more straightforward. Figure 11: Image patches generated to validate the classifier. Experimental tests were conducted using a PC with Intel Core i5 9 th generation, 32 GB of RAM, and an NVIDIA GeForce GTX 1650 graphics card. Table 5 shows the time for the segmentation algorithm. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**Execution Time (in Seconds)**} \\ \cline{2-3}  & **Per Image** & **Total No. of Images Per Dataset** \\ \hline CamVid & 0.83187 & 394.0215 \\ \hline Cityscape & 3.07230 & 4662.554 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Execution time for the segmentation algorithm. Figure 12: Classification of objects using CamVid. (**a**) Test image #1. (**b**) Test image #2. (**c**) Test image #3. (**d**) Test image #4. (**e**) Test image #5. (**f**) Test image #6. Bear in mind that the time presented in Table 5 depends on the number of areas on the segmented image and their sizes.