# Using Common Spatial Patterns to Select Relevant Pixels for Video Activity Recognition
## 4 Experimental Setup  
In this section the details of the experiments that have been carried out are explained. First, the database used is presented. Then, different modalities are introduced and, finally, the optical flow method is explained, which has been used to make a comparison with the CSP approach. In Figure 2 a graphical overview of the presented approach is shown. HMDB51 [[http://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database](http://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database)]([http://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database](http://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database)) [28] is an action recognition database which collects videos from various sources, mainly from movies but also from public databases such as YouTube, Google and Prelinger Archives. It consists of 6849 videos with 51 action categories and a minimum of 101 clips belong to each category. The action categories can be divided into 5 main groups:  
1. General facial actions: smiling, laughing, chewing, talking. 2. Facial actions with object manipulation: smoking, eating, drinking. 3. General body movements: carstwheeling, clapping hands, climbing, going up stairs, diving, falling down, backhand flipping, handstanding, jumping, pull-ups, push-ups, running, sitting down, sitting up, somersaulting, standing up, turning, walking, waving. 4. Body movements with object interaction: brushing hair, catching, drawing a sword, dribbling, playing golf, hitting something, kicking a ball, picking something, pouring, pushing something,  
Figure 1: Proposed approach. Figure 2: An overview of the full process of the presented technique. riding a bike, riding a horse, shooting a ball, shooting a bow, shooting a gun, swinging a baseball bat, drawing sword, throwing. 5. Body movements for human interaction: fencing, hugging, kicking someone, kissing, punching, shaking hands, sword fighting. Apart from the action label, other meta-labels are indicated in each clip. Those labels provide information about some features describing properties of the clip, such as camera motion, lighting conditions, or background. As some videos are taken from movies or YouTube, the variation of features is high and that extra information can be useful. The quality of the videos has also been measured (_good, medium, bad_), and they are rated depending on whether body parts vanish while the action is executed or not. It is worth mentioning that this extra information has not been used in this paper. For our experiments only 6 classes have been selected due to the large amount of images. The selected classes are _brushing hair, cartwheeling, fencing, punching, smoking_ and _walking_. To work with videos, their frames have been extracted in the first place. It has been decided to extract the same number of frames in every video of each class, so the largest video has been selected and the number of frames of the videos of that class is defined by it (in order not to cut any video and maybe lose the action performance). In Table 1 the number of videos and number of frames are indicated for each class. In the case of HMDB51 as the number and length of the videos vary a lot, the number of frames changes in each class. However, the process to get the frames is the same, first the largest video is selected and it is used to determine the number of frames for the class. As some videos need to be extended to get the determined length, some of the frames of these videos are repeated. Due to the difference between the number of videos of some of the classes, it has been decided to use the same amount of videos for both classes when performing the classification. The class with fewer instances indicates the number of videos in each of the experiments. For instance, when performing the classification between _fencing (116)_ and _smoke (109)_ 109 videos are used from each class, with a total of 218 videos.