## 4 Data and Experimental Set-Up  
The proposed methodology for action recognition was tested on two datasets: the University of Central Florida (UCF) Sports dataset [18] and the self-collected HNH human action dataset. The UCF Sports dataset is a moving camera dataset that includes realistic sports videos from broadcast television networks like the BBC and ESPN. It includes around 150 sports videos containing 10 action classes: running, lifting, diving, skateboarding, kicking, swing-golf, walking, swing-side, horse-riding and swing-bench. The frame rate of each sports video is 10 frames per second (fps), whereas the resolution of these videos is 720 \(\times\) 480 pixels, which is comparatively higher than other datasets, thereby making it relatively more challenging. Moreover, the number and duration of clips for each action class is different (e.g., actions like kicking are shorter and less periodic compared with actions such as walking and running). Figure 2 shows snapshots from different videos in the UCF Sports dataset to illustrate the different types of action classes used for testing the proposed action recognition methodology. In addition to the UCF dataset, the proposed methodology was also tested on the HNH moving camera dataset, which is a self-collected dataset comprising 180 video clips recorded at different resolutions in both indoor and outdoor environments. Each video clip in this dataset is labeled as one of six action classes: walking, greeting, shaking hands, dancing, sitting, and standing. The video clips labeled as videos containing the _shaking hands_ action contain two persons, whereas the rest of the video clips contain only one person. Similar to the UCF Sports dataset, the number of clips is not the same for each action class. The dataset is divided into a training and validation subset of 140 video clips and a test set of 40 video clips. The face recognition component of the proposed methodology was tested using the standard Labeled Faces in the Wild (LFW) dataset [19; 46] and the self-collected HMTA dataset for face recognition tasks, comprising 100 identities. Figure 3 shows some sample face images from the HMTA dataset. The LFW is perhaps the most widely used benchmark dataset for unconstrained face recognition in videos and images. It comprises 13,233 images with 5749 people [19]. Figure 3: Sample face images from the self-collected HMTA dataset for face recognition. Figure 2: Snapshots of different videos from the UCF Sports dataset depicting the different types of action classes. Moreover, the emotion recognition component of the proposed framework was tested on the FER2013 dataset [20] for facial emotion recognition, which comprises more than 32,000 images 48 \(\times\) 48 pixels in size each that are split across 7 different types of emotions. The proposed methodology was tested and compared with other techniques by implementing it on a self-developed, intelligent, humanoid robot named _BonBon_, shown in Figure 4. (The word _BonBon_ is used as a common nickname for kids in Vietnam and is used in the meaning of _cute_. The word _Bon_ has its roots in French and means _good_). _BonBon_ has been developed to support the instruction of the English language to non-native speakers in elementary schools. It has the appearance of a boy, as shown in Figure 4, is 1.2 meters tall and weighs 35 kilograms. The robot consists of an upper body with 21 degrees of freedom, a head, two arms, two hands, a ribcage and a mobile platform with three omnidirectional wheels. The robot's body houses two computers that control the entire operation of the robot, including motion planning, voice recognition and synchronization, face, emotion and action recognition, receiving and processing commands from an external control and monitoring station, receiving and processing input from a microphone array, cameras, and other sensors and receiving and sending signals to the mobile module and the upper body controllers.