# Action Classification for Partially Occluded Silhouettes by Means of Shape and Action Descriptors
## 2 Related Works on Action Recognition and the Problem of Object Occlusion
### Action Recognition under Occlusion  
The approaches for action recognition mentioned in Section 2.1 yield high classification accuracy on popular benchmark datasets, however, have not been demonstrated on partially occluded action sequences so far. Occlusion is an important and challenging problem for vision-based systems, and foreground object occlusion can be perceived as a loss of data. In a single-camera setup, self-occlusion or inter-object occlusion may be present [11]. Moreover, the object of interest can remain partially occluded (temporarily or not) by the edge of the video frame--in other words, a part of the silhouette is outside the camera's field of view. Occlusion can coexist with other problems, such as changes in scene illumination, cluttered background, moving shadows or similar foreground to background, all of which can contribute to artefacts in images resulting from the background subtraction process. Artefacts in binary images have a form of extra pixels or a lack of some pixels in a silhouette area, thus it can be referred to as false data or data loss, respectively.  
Occlusions are very problematic in real environments and busy scenes, making it difficult to detect and track objects. The selection of experimental database depends mainly on the recognition problem, application and required action types [11; 32]. Literature sources indicate various ways to evaluate the action recognition method in the presence of occlusion, some of which add the occlusion to existing videos or foreground masks (e.g., [33; 34; 35]). In [33], a method for human action recognition based on sequences of depth maps is proposed. Each depth map consists of a human silhouette on a black background, where the area of a silhouette is coloured according to depth information. In order to avoid joint tracking, temporal dynamics are modelled using expandable graphical model (action graph) and the postures are represented using a bag of 3D points. A test dataset was collected by the authors, and include actions such as high arm wave, hand clap, bend, side kick, jogging or tennis serve, grouped into three subsets. The proposed solution is tested in each subset under simulated occlusion, where a depth map is divided into quadrants and one or two various quadrants of a silhouette are occluded. Compared to the experiments on unoccluded data, the recognition accuracy did not decrease significantly, except the case where the upper body was under heavy occlusion.  
Another action recognition technique for corrupted silhouettes is proposed in [34]. It extracts normalized regions of interest containing as little background as possible (called a silhouette block) and represents each region as a binary vector--an image sequence of frames is transformed into vector sequence. Each silhouette block is partitioned into 12 sub-parts and partial template matching is applied. The results are integrated using voting or the sum of distances. During the experiments, two types of regions are superimposed on image sequences to simulate occlusion. These include horizontal and vertical stripes of different width and height, which increased the test dataset. The proposed approach obtained almost perfect accuracy for the original Weizmann database and is quite effective for horizontal occlusions.  
In [35], several local descriptors are tested in the presence of static occlusion, namely Trajectory, Histogram of Oriented Gradients, Histogram of Orientation Flow and Motion Boundary Histogram. These methods are tested in combination with a standard bag-of-features representation and classified using SVM. The experiments were performed on the KTH benchmark dataset. Silhouette regions of interest were extracted and 10%, 25%, 50% or 75% of an area of each region was occluded. A combination of Trajectory and Motion Boundary Histogram (MBH) yielded average accuracy of 90% for partially occluded silhouettes.