# An Indoor Room Classification System for Social Robots via Integration of CNN and ECOC
## 4 Dataset & Simulation Experiments
### Discussion  
One of the most important features that has been studied by researchers is deepness, which is significant in most of the known CNN architectures. It is reported that the deeper and wider the architecture is designed, the better features will be learned [35]. However, this requires a huge dataset, i.e., millions of samples, in order to avoid the overfitting issue. Unfortunately, this is not always the case in robotics applications, wherein in most applications, the number of classes for a specific robotic problem such as room classification is very limited, which means the number of samples might be only thousands, i.e., a small dataset. Therefore, the very deep CNN architecture will most likely lead to overfitting problem, as happened with ResNet [45] in this work, which is why their results are excluded from these experiments. Although the Inception V3 results were not over-fitted, they were less accurate than the architectures with less deepness, i.e., VGG16 and VGG19. The reason might be related to the scene-centric type of dataset, in which learning hierarchical representations can be difficult with more deepness. For this work, we had two ways to address this problem for robotic application: either designing a new CNN architecture for a small dataset, or adopting an existing CNN with less deepness and improving robot's decision by integrating another method. The latter solution was preferable, so we adopted VGG16, i.e., the least deepness of all three architectures, and integrated it with ECOC in a way it was practicable for real time robotic implementation. One could ask why we adopted CNN from the beginning--as discussed before, CNN extracts and learns features from raw images without requiring a careful engineering design that shown superiority over conventional approaches in computer vision.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Binary** \\ **Classifiers** \\ \end{tabular} } & \multicolumn{1}{c}{**1**} & \multicolumn{1}{c}{**2**} & \multicolumn{1}{c}{**3**} & \multicolumn{1}{c}{**4**} & \multicolumn{1}{c}{**5**} & \multicolumn{1}{c}{**6**} & \multicolumn{1}{c}{**7**} & \multicolumn{1}{c}{**Bed vs.**} & \multicolumn{1}{c}{**9**} & \multicolumn{1}{c}{**10**} & \multicolumn{1}{c}{**11**} & \multicolumn{1}{c}{**Dining 13**} & \multicolumn{1}{c}{**Kitchen**} & \multicolumn{1}{c}{**Living**} & \multicolumn{1}{c}{**Average**} \\  & **All** & & & & & & & & & & & & & & & & \\ \hline Accuracy \%. & **98.50** & 93.62 & 97.38 & 92.01 & 94.84 & 93.95 & 96.06 & 95.73 & 95.59 & 96.10 & 94.94 & 95.89 & 92.76 & 97.95 & 95.31 & **95.37** \\ \hline \hline \end{tabular}
\end{table}
Table 3: 15 binary classifier accuracies.  
Binary classifier results for ECOC explain the challenge of feature similarities in a house's rooms. Let us discuss the obvious results of 'class vs. all' in classifiers number 1, 8, 12, 14, and 15. The most distinguishable rooms are the bathroom and kitchen, as shown in classifiers 1 and 14 respectively. Meanwhile, the other three classes are very similar to each other. There are many reasons related to the dataset or the architecture of VGG16 that the model is less accurate with these similar rooms than the distinguishable ones. The first reason is having some sharable objects in different rooms, such as tables or TVs, or the wide variety styles of those rooms such as open/closed spaces, or even culture-based styles, e.g., no beds for sleeping. The second reason is that the architecture with less deepness will not be able to differentiate between objects similar in shape, e.g., rectangular shapes in dining tables, coffee tables, and beds from different rooms. Therefore, it is a tradeoff between learning deep features and having a small dataset. For our future work, we will consider the associated models between scenes and objects for better robot decisions.