In this paper, we tackle the problem of 24 hoursmonitoring patient actions in a ward such as "stretching an arm out of the bed", "falling out of the bed", where temporal movements are subtle or significant. In the concerned scenarios, the relations between scene layouts and body kinematics (skeletons) become important cues to recognize actions; however they are hard to be secured at a testing stage. To address this problem, we propose a kinematiclayout-aware random forest which takes into account the kinematic-layout (i.e. layout and skeletons), to maximize the discriminative power of depth image appearance. We integrate the kinematic-layout in the split criteria of random forests to guide the learning process by 1) determining the switch to either the depth appearance or the kinematiclayout information, and 2) implicitly closing the gap between two distributions obtained by the kinematic-layout and the appearance, when the kinematic-layout appears useful. The kinematic-layout information is not required for the test data, thus called "privileged information prior". The proposed method has also been testified in cross-view settings, by the use of view-invariant features and enforcing the consistency among synthetic-view data. Experimental evaluations on our new dataset PATIENT, CAD-60 and UWA3D (multiview) demonstrate that our method outperforms various state-of-the-arts.
A random forest classifier is applied on the DMM in REF , where the geometry between scene layout and the human skeleton is considered for representation in the random forest.
Kinematic-Layout-aware Random Forests for Depth-based Action Recognition
BMVC 2017
arXiv: Computer Vision and Pattern Recognition
Computer Science