# Technology Enhanced Learning Using Humanoid Robots
## 4 The Proposed Architectural Design
### Object Detection Module  
Choregraphe Component (CC).The CC of this other module is responsible for the interaction with the user. If the user says _recognize_, the CC is activated and informs the user that in five seconds the robot will take a picture (whose resolution is 640 x 800) from its front camera. Once the picture has been taken, it is forwarded to the SSSC to be processed and classified. The SSSC computes the bounding boxes and the related object categories and confidence values. The CC receives the output of the classification process performed by the SSSC and informs the user about the recognized objects asking if they are correct. The user, finally, does or does not confirm and their annotations are stored in a database within the SSSC and added to the existing training set. Periodically the classifier is re-trained with the updated (increased) training set.  
Server-Side-Support Component (SSSC). The SSSC includes an object detection engine able to identify 91 different objects types. Moreover, to provide a more accurate object detection, for two object types (dogs and cats), the module performs one more fine-grained classification. In fact, when the recognized object is either a dog or a cat, the module employs two more classifiers to understand which breed the dog or the cat represents. Overall there are three classifiers we have trained: One for the general objects, one for the dog breeds, and the other for cat breeds.  
We employed the TensorFlow Object Detection APIs ([[https://bit.ly/2](https://bit.ly/2) IPqHJk]([https://bit.ly/2](https://bit.ly/2) IPqHJk) (accessed on 25 January 2021)), which provides an open-source framework built on top of TensorFlow that makes it easy to construct, train, and deploy object detection models. TensorFlow Object Detection APIs can be used with different pre-trained models.  
For the general object detection task, we have chosen a Single Shot MultiBox Detector (SSD) model with Mobilenet (ssd_mobilenet_v1_coco) which has been trained using the Microsoft COCO dataset ([[http://cocodataset.org](http://cocodataset.org)]([http://cocodataset.org](http://cocodataset.org)) (accessed on 25 January 2021)), and that consists of 2.5 M labeled instances in 328,000 images, containing 91 object types. The ssd_mobilenet_v1_coco is reported to have a mean average precision (mAP) of 21 on the COCO dataset. For further details on the SSD and the evaluation carried out on the COCO dataset please check the work of authors in [20].  
For the object detection of the second level, we have employed the Oxford-IIIT-Pet dataset ([[http://www.robots.ox.ac.uk/~vgg/data/pets/](http://www.robots.ox.ac.uk/~vgg/data/pets/)]([http://www.robots.ox.ac.uk/~vgg/data/pets/](http://www.robots.ox.ac.uk/~vgg/data/pets/)) (accessed on 25 January 2021)), which consists of 4110 images distributed along 37 dog breeds and 12 cat breeds.  
The SSSC has been embedded into a server-side application that exposes REST APIs that, given an input image, return the bounding box of each recognized object in the image along with a category and confidence value. We considered valid only the recognized objects having a confidence value equal to or higher than 60%. The SSSC is hosted in the cloud, where it can exploit GPU technology to largely speed up the classification process. Figure 4 shows an example image is already processed and with several bounding boxes related to different recognized items. The two with a label have a confidence value higher than 60%.  
A video example of the interaction between the human and the object detection module of Zora is freely available ([[https://www.youtube.com/watch?v=WLjUPBTtdkE](https://www.youtube.com/watch?v=WLjUPBTtdkE)]([https://www.youtube.com/watch?v=WLjUPBTtdkE](https://www.youtube.com/watch?v=WLjUPBTtdkE)) (accessed on 25 January 2021)).