# An Indoor Room Classification System for Social Robots via Integration of CNN and ECOC
## 5 Real-Time Experiment on NAO Humanoid Robot
### Practical Implementation Results  
The proposed methods were tested and compared practically in five different houses using the NAO humanoid robot. The goal was that the robot should be able to predict the room class using the three different models, i.e., CNN, CNN-ECOC & CNN-ECOC-REG, with all models returning the probability for each class. Since rooms in the houses had different sizes, layouts, furniture, etc., NAO  
Figure 5: NAO humanoid robot during experiments.  
was positioned in different spots in the rooms, i.e., center, corners, beside the wall or the door, during different time of the day under different light conditions. Accordingly, 56 images were taken by its top camera as follows: 12 bathrooms, 13 bedrooms, 6 dining rooms, 13 kitchens, and 12 living rooms, while the robot's head faced the x direction and the top camera covered 60.97\({}^{\circ}\) and 47.64\({}^{\circ}\) on the y and z directions, respectively. Figure 6 shows some examples of scenes taken by the NAO humanoid robot. An important point to be noted is that NAO is very short compared to an average human's height, and its camera is mounted about 514 mm from the floor level. This is quite different from the field of view of the images from the adopted dataset. Consequently, the highest (top-1) probabilities were negatively affected, especially in the kitchen class. However, the second highest (top-2) probabilities show the superiority of CNN-ECOC and CNN-ECOC-REG over the regular CNN for prediction of most of the five classes.  
Table 4 shows the testing accuracy of the five classes, whereas Table 5 gives confusion matrix for both top-1 and top-2 predictions. Notice that the confusion matrix of top-1 tables has all 56 images, however the confusion matrix of top-2 tables includes only the number of false predicted images in the top-1 table. The best room prediction was the bathroom, where accuracy was 100% with all three models. Therefore, the bathroom was considered to be the most distinctive room. All three models were similar at predicting bedrooms, in which their top-1 accuracy was 69.2%, increasing to 100% in top-2. The distinction in performance between these models is shown in the prediction of dining rooms, kitchens, and living rooms. The top-1 predictions of dining rooms for three models were similar, however, the top-2 prediction of CNN-ECOC and CNN-ECOC-REG increased to 100%, which was better than CNN. In the living room cases, the CNN and CNN-ECOC-REG gave the best rate of 50% in the top-1, however the CNN-ECOC and CNN-ECOC-REG were able to increase up to 91.7% and 100% in their top-2 predictions, respectively. Although the kitchen top-1 prediction was the worst in all models, it surprisingly showed a huge improvement in top-2 prediction for CNN-ECOC and CNN-ECOC-REG compared to CNN. Overall, the three models performed similarly in the top-1 prediction, however the multi-binary classifier solutions, i.e., CNN-ECOC and CNN-ECOC-REG, gave much better performance in the top-2 results in contrast to the CNN for multi-classification.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Bath**} & \multicolumn{2}{c}{**Bed**} & \multicolumn{2}{c}{**Dining**} & \multicolumn{2}{c}{**Kitchen**} & \multicolumn{2}{c}{**Living**} \\ \cline{2-10}  & **T1** & **T2** & **T1** & **T2** & **T1** & **T2** & **T1** & **T2** & **T1** & **T2** \\ \hline CNN & _100_ & - & _69.2_ & _100_ & _66.7_ & _83.3_ & _15.4_ & _61.5_ & _50_ & _75_ \\ \hline CNN-ECOC & _100_ & - & _69.2_ & _100_ & _66.7_ & _100_ & _15.4_ & _100_ & _41.7_ & _91.7_ \\ \hline CNN-ECOC-REG & _100_ & - & _69.2_ & _100_ & _66.7_ & _100_ & _30.8_ & _100_ & _50_ & _100_ \\ \hline \hline \end{tabular}
\end{table}
Table 4: Testing accuracies of all models with top-1 (T1) and top-2 (T2) for all five classes.  
Figure 6: Scenes examples taken by NAO humanoid robot.