Computer vision techniques for gait-based visual surveillance

The importance of automated visual surveillance in public space has been increased in response to the recent rising concerns about safe and security, and computer vision-based person identification techniques play a key role in it. Gait as a biometric cue has received much attention in recent years due to the ability of identifying individuals at a distance, and gait-based person identification technique could contribute much to crime investigation and safety confirmation through wide-area surveillance. Although gait-based person identification has such a promising ability, several critical issues need to be sufficiently considered when applying it to real visual surveillance tasks. Among them, this thesis addresses following three issues, each of which is corresponded to a primal step in gait-based person identification: i) accuracy of foreground segmentation in preprocessing step, ii) robustness to intra-subject variations in identification step, and iii) statistical reliability in performance evaluation step. First, a research for the first issue is described. We propose a method for accurate foreground segmentation in the presence of strong shadow. For the separation of foreground and shadow, the homography constraint in binocular system is used. In addition, while existing homography-based methods often suffer from the occlusion relationship, the proposed method takes such relationship into account explicitly by using a homography correspondence pairbased symmetric labeling scheme. The scheme is formulated in the form of energy minimization problem and optimized by an a-f3 swap algorithm. The experimental results demonstrate that the proposed method realizes more accurate segmentation than the existing methods in the presence of strong shadow and occlusion. Next, we propose a novel person identification framework where the identification performance could be enhanced against intra-subject variations. We pay attention to the fact that people often act in groups such as friends, family, and co-workers in social living and we utilize this as a cue for identifying individuals to improve the identification performance. The individual cues and the group cue are integrated in the form of conditional random field model, and the identities of individuals are optimized via belief propagation algorithm. The comparison experiments with the straightforward identification scheme show the effectiveness of the proposed method. Finally, we construct the world's largest gait database. The database includes 4,007 subjects (2,135 males and 1,872 females) with ages ranging from 1 to 94 years. The database enables the statistically reliable performance comparison among state-of-the-art gait features for person identification. Also, we investigate the dependences of the identification performance on gender and age group, and several novel insights are provided such as the gradual change in identification performance with human growth. Together with the considering of these issues, this thesis could make a large contribution to the development of more accurate and practical gait-based visual surveillance.