# A New Framework of Human Interaction Recognition Based on Multiple Stage Probability Fusion
## 1 Introduction  
Human interaction recognition and understanding is an important research topic in the computer vision community [1; 2]. This research direction has received considerable attention in recent years due to the increasing number of potential applications, such as visual surveillance, human-computer interaction, video indexing and retrieval, smart homes/offices, healthcare rooms, physical sciences, health-related issues, natural sciences and industrial academic areas [3; 4; 5; 6], etc. The human interaction recognition requires an understanding of spatio-temporal relationships between different objects, in additional to individual variability, cluttered background, viewpoint changes, and other environment induced conditions [7]. So it remains challenging to recognize human interaction from complex scenes. Recent research on interaction recognition can be characterized by two classes of methods. (1) **Interaction is recognized as a general action. ** This kind of method usually represents the interaction as an integral descriptor including all the people involved in the interaction. Then a traditional classier is utilized to classify interactions. [PERSON] et al. obtained a powerful discriminative codebook by introducing semantic texton forests (STFs) to local space-time volumes. Then the hierarchical k-means algorithm with pyramid match kernel is applied to achieve robust structural matching and interaction recognition [8]. The feature extraction of this method is relatively simple, however, the matching method is relatively complicated. [PERSON] et al. improved the spatio-temporal representation by introducing a spatio-temporal layout of actions and obtainedsuccessful human interaction recognition [9]. [PERSON] et al. utilized multi-scale dense trajectories with a four advanced feature (DT (Dense Trajectory) shape, Histogram of Oriented Gradient (HOG), Histogram of Optical Flow (HOF), Motion Boundary Histograms(MBH)) encoding method to achieve human interaction recognition [10]. [PERSON] et al. proposed a hybrid framework which incorporates both a global feature (Motion Context) and a local feature (spatio-temporal interest point (STIP)) to recognize human interactions. The method achieves promising results by respectively using a Genetic Algorithm (GA)-based random forest and calculating the S-T correlation score as the recognition method [11]. As Kinect was introduced by Microsoft, both RGB images and depth images of the scene can be simultaneously captured. Some researchers have studied the RGBD-based human interaction recognition method. [PERSON] et al. combined color and depth information to develop two feature representations, i.e., spatio-temporal interest points (STIFs) and motion history images (MHIs). The proposed multi-modality fusion method was tested and demonstrated superior performance on a home-monitoring oriented human interaction recognition dataset [12]. [PERSON] et al. designed a variety of related distance features (such as joint distance, joint movement, plane features, normal plane features, velocity features and normal velocity features) to carry out a two-person interaction recognition [13]. This method can achieve real-time human interaction detection, but the performance of the method depends on the accurate extraction of the joint point. This kind of method treats people as an entity and so does not need to segment the individual as a feature in the interaction, which makes the processing method relatively simple. However, this kind of method does not accurately represent the intrinsic properties of the interaction, and therefore, a better performance requires comprehensive motion features and a matching method. (2) **Interaction recognition using motion co-occurrence. ** This kind of method proposes that the interaction is composed of a set of temporal-ordered elementary actions performed by the different persons involved in the interaction [14; 15]. [PERSON] et al. proposed interactive phrases to describe the motion relationships between two people in the interaction.