# An Indoor Room Classification System for Social Robots via Integration of CNN and ECOC
## 4 Dataset & Simulation Experiments
### Multi-Class Room Classification Experiments Using Fine-Tuning Pre-Trained Models  
Since our main concern was to recognize only five room classes, we did not require huge amounts of data. Additionally, the learned features from pre-trained models in literature are relevant to the room classification problem, therefore fine-tuning pre-trained models was the best strategy for this work, instead of training a CNN model from scratch. Fine-tuning can be achieved through two main steps. The first step is to proceed room images to non-trainable ConvNet in order to extract features, then use these features to train our new classifier, i.e., softmax layer. The second step is to retrain the whole network, i.e., ConvNet and classifier, with a smaller learning rate, while freezing a few layers of the ConvNet.  
All experiments were completed through the Graham cluster provided by Compute Canada Database [43] using Keras API, which is written in a python deep learning library [44]. Several CNN models were fine-tuned for this project, i.e., VGG16, VGG19, and Inception V3, with different freezing layers to be trained. All these CNN models were followed by a similar fully connected (FC) layer. FC begins with an average pooling layer, then a layer of 1024 neurons with the _relu_ activation function, and ends with a logistic layer to predict one of the five classes. Keras provides a compile method with different optimizers for learning process. In the first stage of the fine-tuning, the _adam_ optimizer was used with a 0.001 learning rate, whereas we applied a SGD (stochastic gradient descent) optimizer in the second stage with learning rate of 0.0001 and momentum of 0.9. All models were trained for 10 epochs in each stage with both the original data as well as the cleaned data, and with different non-trainable layers. It was noticed that training with more epochs did not provide that much improvement in the final accuracy, but it took a very long time in the training process. Table 2 shows the superior results of all models trained with clean data compared to all data. The best result shown from these experiments is VGG19 and VGG16 with 0 freezing layers using clean data, which gives an accuracy of 93.61% and 93.29%, respectively.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Class** & **\# Images out of 11,600** & **\% of Removed Images** \\ \hline Bedrooms & 9323 & 19.6\% \\ Dining Rooms & 7919 & 31.7\% \\ Kitchens & 6611 & 43.0\% \\ Living Rooms & 8571 & 26.1\% \\ Bathrooms & 8959 & 22.8\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: Number of images for each class after cleaning.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**CNN Models**} & \multirow{2}{*}{**Non-Trainable Layers**} & \multicolumn{2}{c}{**All Data**} & \multicolumn{2}{c}{**Clean Data**} \\ \cline{3-6}  & & _Time_ & _Accuracy \%_ & _Time_ & _Accuracy \%_ \\ \hline \multirow{4}{*}{**VGG16**} & _15_ & 11:50:40 & 86.03 & 8:16:16 & 89.69 \\ \cline{2-6}  & _11_ & 11:50:41 & 88.09 & 8:15:38 & 91.49 \\ \cline{2-6}  & \(7\) & 11:53:42 & 88.9 & 8:18:49 & 93.22 \\ \cline{2-6}  & \(0\) & 12:13:55 & 87.78 & 8:34:45 & 93.29 \\ \hline \multirow{4}{*}{VGG19} & _20_ & 13:11:07 & 78.69 & 9:16:00 & 82.50 \\ \cline{2-6}  & _17_ & 13:14:56 & 86.22 & 9:18:38 & 89.65 \\ \cline{2-6}  & \(0\) & 13:43:40 & 90.30 & 9:40:52 & 93.61 \\ \hline \multirow{2}{*}{Inception V3} & _299_ & 10:17:29 & 75.12 & 7:8:33 & 78.83 \\ \cline{2-6}  & _249_ & 10:17:46 & 79.11 & 7:07:50 & 84.05 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of accuracies of fine-tuning different CNN models using all data and clean data (the yellow shaded ones are the best for the real-time experiment).