Vision-based human affect analysis is an interesting and challenging problem, impacting important applications in many areas. In this paper, beyond facial expressions, we investigate affective body gesture analysis in video sequences, a relatively understudied problem. Spatial-temporal features are exploited for modeling of body gestures. Moreover, we present to fuse facial expression and body gesture at the feature level using Canonical Correlation Analysis (CCA). By establishing the relationship between the two modalities, CCA derives a semantic "affect" space. Experimental results demonstrate the effectiveness of our approaches.
Shan et al. REF use the fusion of facial expressions and body gestures at the feature level and derive an "affect" space by performing Canonical Correlation Analysis.
Beyond facial expressions: Learning human emotion from body gestures
in Proc. British Machine Vision Conf
Computer Science