# An Indoor Room Classification System for Social Robots via Integration of CNN and ECOC
## 2 Related Work
It was proposed by LeCun [5], who introduced the first CNN architecture called LeNet in 1998, after several successful attempts since the 1980s [4; 21]. There are two main advantages of this algorithm over other machine learning algorithms and the conventional fully connected feedforward neural networks. First, CNN extracts and learns features from raw images without requiring a careful engineering design for extracting features in advance [7]. Second, CNN considers the spatial structure of the image by translating inputs to outputs through shared filters [22]. After the huge improvements in data collection and computer hardware between 1990s and 2012, AlexNet was introduced [6] for addressing the object detection problem using the ubiquitous _ImageNet_ to classify 1.2 million images into 1000 different classes. Since 2012, many articulate architectures have been proposed that are essentially built on the early architecture of LeCun, in order to improve the performance on ImageNet database [23]. However, the effective progress that was demonstrated on ImageNet for object classification by these pre-trained models has not shown the same success for the scene classification problem. Consequently, the first significant dataset for scene-centric images, referred to as Places, was proposed in Reference [8]. In general, indoor scene classification is challenging due to features' similarity in different categories. This problem has been studied with different learning methods as well as CNN, which so far has been employed in few studies. In Reference [24], a solution was proposed by designing a model that combined local and global information. The same problem was addressed by applying a probabilistic hierarchical model, which associates low-level features to objects via an object classifier, and objects to scenes via contextual relations [25]. There are also some research studies that have employed CNN for learning robots in indoor environments. [PERSON] et al. [26] addressed the room classification problem for household service robots. The performance of a pre-trained hybrid-CNN model was examined in Reference [8] on segmented images, i.e., learning through parts, of eight classes from the _Indoor67_ dataset. The result generated 85.16% accuracy using a part-based model, which is close to the accuracy of the original hybrid-CNN, 86.45%. However, learning through parts gave much better accuracies on deformed images than the original model. The authors in Reference [27] took advantage of CNN for scene recognition in laboratory environments, with 89.9% accuracy, to enhance the indoor localization performance of a multi-sensor fusion system using smartphones. Furthermore, the objective of Reference [28] was to find the best retraining approach for a dynamically learning robot in indoor office environments. The paper examined and compared different approaches when adding new features from new images into a learned CNN model, considering the accuracy and training time. The new added images to the features database were the failed ones that were selected and corrected by the user. The authors simulated one of the categories to be the new environment. This paper reported that a pre-trained CNN with a KNN classifier was the most appropriate approach for real robots, as it gave a reasonable accuracy with the shortest training time. All their experimentswere executed on the VidRILO dataset [29] using only its RGB frames, i.e., excluding the D frame. The methodology of this presentation, however, is different from previous studies, as we examined several CNN architectures with five categories of indoor scene rooms, i.e., bathrooms, bedrooms, dining rooms, kitchens, and living rooms downloaded from the Places dataset. In addition, these models were examined after cleaning and reducing the number of samples. Furthermore, a combination of CNN and multi-binary classifiers method called ECOC was proposed and evaluated in order to improve the real-time performance on a NAO humanoid robot.