# Unsupervised 3D Motion Summarization Using Stacked Auto-Encoders
## 3 The Proposed Dance Summarization Workflow
### Physics-Based Attributes of 3D Motion Primitives  
In the following, let us denote as \(\vec{I}_{k}^{\widetilde{G}}(t)=(x_{k}^{G}(t),y_{k}^{G}(t),z_{k}^{G}(t))\) the \(k\)-th joint out of the \(M\) extracted by the Vicon architecture for each dancer for the \(t\)-th frame of the dance sequence. In our case \(M=40\), that is, 40 joints are extracted per human dancer. Variables \(x_{k}^{G}(t),y_{k}^{G}(t)\) and \(z_{k}^{G}(t)\) indicate the coordinates of the \(k\)-th joint with respect to a reference point setting by the VICON architecture (in our case the center of the square surface) for the \(t\)-th frame. These joints have been obtained after the application of a density-based filtering on all the detected joints to remove noise from the acquisition process (see the third paragraph of Section 6). This noise becomes apparent when multiple dancers are performing in the choreography.  
The main problem in directly processing the extracted joints \(\vec{I}_{k}^{\widetilde{G}}(t)\) is that they refer to the VICON coordination system which do not reflect the dancer's position in 3D space. For this reason, we first compute the center of the mass for each dancer and then the coordinates of \(\vec{I}_{k}^{\widetilde{G}}(t)\) is transformed to a local coordinate system, the origin of which coincides with the center of mass of a dancer, that is \(\vec{I}_{k}^{L}(t)=\vec{I}_{k}^{\widetilde{G}}(t)-\vec{C}_{cm}(t)\), where \(\vec{C}_{cm}(t)\) denotes the center of mass of a dancer. As far as the  
Figure 3: The proposed architecture for video dance summarization using stacked auto-encoders and representative algorithms.  
kinematics attributes is concerned, the velocity and the acceleration are taken into account. In particular, the velocity is given as \(\vec{u_{k}}(t)=d\vec{l}_{k}^{\mathcal{L}}(t)/dt\), while the acceleration as \(\vec{\tau_{k}}(t)=d\vec{u_{k}}(t)/dt\) for each detected human joint. Since velocity and acceleration are given through a derivative formula, their calculation is independent from local/global coordination system and thus they are independent of a global translation. Alternative, we could use global dancers' velocity along with small local velocities of the joints to improve the feature analysis. But in this paper, we prefer to concentrate on simpler features. Gathering all these features together a vector is constructed as \((\vec{l}_{k}^{\mathcal{L}}(t),\vec{u_{k}}(t),\vec{\tau_{k}}(t))\). In the aforementioned notation, we focus only on one dancer and thus we omit indices describing the dancers for clarity purposes.  
Figure 2 show the humans' joints extracted both on RGB content (the first and the third row of Figure 2) and on a plane depicting the movement of the dancers in the space (second and fourth row of Figure 2). Since we have two dancers executing the choreography, it is clear that severe occlusions and merges are encountered, mainly due to the 3D geometry of the dancers. This is the case, for example, of the fifth and sixth frame of Figure 2 where one can notice, by observing the frame content, that only one dancer appears to perform.