# Extending Appearance Based Gait Recognition with Depth Data
## 2 Related Work  
Gait recognition methods that use RGB video stream as input have been extensively researched. The reader can consult for example the work of [PERSON] et al. [3] for a recent survey. Particularly relevant to this research is the work from [PERSON] and [PERSON] [4], which uses a spatiotemporal gait representation, called Gait Energy Image (GEI), to characterize human walking properties. Binary silhouettes are extracted from RGB images, size normalized, and aligned. Then, the average silhouette over a gait cycle is constructed, where pixels of higher intensity correspond to positions where human body occurs more frequently. This average silhouette is named Gait Energy Image (GEI). GEI is one of the most influential works in the field of gait recognition, from which several other appearance based methods are derived. Appearance based gait recognition methods that use both RGB and depth video streams are more recent. The reviewed methods generally introduce new gait features based on depth or RGB-D streams. These features are then used for side-view gait recognition [2; 5], or for frontal gait recognition [2; 6; 7]. Some methods [8; 9] are also used for gender recognition tasks. In this section, they are briefly described. [PERSON] et al. [6] extended GEI with 3D information and presented Gait Energy Volume (GEV) which uses reconstructed voxel volumes instead of temporally averaging segmented silhouettes. They tested basic GEV implementation on CMU MoBo database, and also experimented on frontal depth images evaluating them on an in-house developed database created with Kinect sensor. [PERSON] et al. [5] also extended the traditional GEI using depth information to, in addition to GEI, calculate also the edges and depth gradients within the person's silhouettes, which are then aggregated in cells, normalized, and finally averaged over a gait cycle to obtain a new representation called Depth Gradient Histogram Energy Image (DCHEI). [PERSON] et al. [8] presented DG Gait, a new gait database acquired with a depth camera. DG Gait was acquired with Kinect sensor in an indoor environment and contains videos from 53 subjects walking in different directions, and each video is labeled according to the subject, gender, and age. In addition, they performed gait-based gender classification experiments with the mentioned database to show the usefulness of depth data in this context. In their experiments, they extracted 2D and 3D gait features based on shape descriptors for gender identification using an advanced version of support vector machine (SVM) called Kernel SVM. According to the results, depth can be very relevant for gait classification problems. [PERSON] et al. [9] investigated the problem of human identity and gender recognition from gait sequences with arbitrary walking directions. They obtained human silhouettes by background subtraction and clustering. For each cluster, a cluster-based averaged gait image (C-AGI) is calculated. Then, they presented a sparse reconstruction based metric learning (SRML) method for discriminative gait feature extraction. Besides that, the authors created a new gait database with a Kinect sensor in an indoor environment called Advanced Digital Sciences Center Arbitrary Walking Directions (ADSC-AWD). [PERSON] et al. [7] proposed a new feature called Pose Depth Volume (PDV) for frontal gait recognition by partial volume reconstruction of the frontal surface of the silhouettes from Kinect RGB-D streams. [PERSON] et al. [10] proposed a novel method where an image of the human body is divided into multiple areas and features are extracted for each area. For each area, a matching weight is estimated based on the similarity between the extracted features and those in the database for standard clothes. The identification of a subject is performed by weighted integration of similarities in all areas. [PERSON] et al. [2] proposed the Backfilled Gait Energy Image (BGEI) appearance based feature. BGEI can be constructed from frontal depth images as well as side-view silhouettes.