## 3 Materials and Methods
### SPI-Net Architecture
#### 3.2.1 Geometric Features Branch
This gives us a one-dimensional distance vector as our branch input of size equal to 3000 for each sequence: \(T*(\begin{subarray}{c}n\text{{h}}_{\text{{h}}_{\text{{h}}_{\text{{h}}_{\text{{ h}}_{\text{{h}}_{\text{{h}}_{\text{{h}}_{\text{{h}}_{\text{{h}}_{\text  
More specifically, we obtain an auto-encoder with a separability constraint term that focuses on two completely different pieces of information in the data. One in an unsupervised manner, the other in a supervised manner:  
* The inherent structure of the data captured in an unsupervised manner thanks to the reconstruction of the auto-encoder and its abstraction ability. Some of the important and discriminating information in the data set would then be retained. * The separability of classes thanks to Linear Discriminant Analysis projection of the instances in the latent space. Formally, we define the problem as follows: \[\min_{\theta_{1},\theta_{2}}\left\|\mathbf{X}-g_{\theta_{2}}\left(f_{\theta_{1 }}(\mathbf{X})\right)\right\|^{2}\] (2) Equation (2) is the usual reconstruction function of an auto-encoder with \(\mathbf{X}\) a data matrix, \(\theta_{1},\theta_{2}\) the parameters of the encoder and decoder blocks and \(f(),g()\) are, respectively, the transition functions such that: \[ \begin{array}{l}f_{\theta 1}:\mathbf{X}\rightarrow\mathcal{F}\\ g_{\theta_{2}}:\mathcal{F}\rightarrow\mathbf{X}\end{array} \] (3) where \(\mathcal{F}\) is the feature space, which can be regarded as a compressed representation of the input matrix \(\mathbf{X}\). Throughout that study, we refer to \(\mathcal{F}\) as the bottleneck or the latent space of the auto-encoder. We then add a statistical supervised constraint specific to the separability of classes in the cost function: with \(\mathbf{S}\) being the projection matrix of the instances in the latent space obtained with a linear discriminant analysis (LDA) and \(\lambda\) a weighting parameter as presented in Equation (4): \[\min_{\theta_{1},\theta_{2},\mathbf{S}}\left\|\mathbf{X}-g_{\theta_{2}}\left(f_ {\theta_{1}}(\mathbf{X})\right)\right\|^{2}+\lambda\left\|f_{\theta_{1}}( \mathbf{X})-\mathbf{S}_{f_{\theta_{1}}}(\mathbf{X})\right\|^{2}\] (4)  
Figure 5: Pipeline of the approach for the Geometric branch: (**1**) we train an auto-encoder to reconstruct a sequence representing an action according to the evolution over time of the distances (represented by the red arrows) of selected keypoints (Torso, Left and Right Shoulders, Left and Right Knees). We also add a constraint specific to the separability of classes in the latent space. (**2**) We then extract the weights of the encoder part up to the bottleneck represented in red and add a classifier, which transforms the encoder part into a pre-trained network on the data for action classification. The training method is a simple iterative algorithm, optimizing an appropriate objective function. This algorithm is based on two updating steps according to the scheme written in Algorithm 1:  
```
Input: data matrix \(\mathbf{X}\), ground truth labels \(y\), weighting parameter \(\lambda\), loss threshold \(\varepsilon\)  Initialization of the encoder and decoder parameters \(\theta_{1}\) and \(\theta_{2}\); while\(\left\|\mathbf{X}-g_{\theta_{2}}\left(f_{\theta_{1}}(\mathbf{X})\right) \right\|^{2}+\lambda\left\|f_{\theta_{1}}(\mathbf{X})-\mathbf{S}_{\theta_{1}} (\mathbf{X})\right\|^{2}>\varepsilon\)do  Update \(\theta_{1}\) and \(\theta_{2}\) using the auto-encoder;  Update S using Linear Discriminant Analysis on \(f_{\theta 1}(\mathbf{X})\) data matrix and \(y\);  end while Result: \(\theta_{1}\), parameters of the encoder block
```  
**Algorithm 1**Auto-encoder with statistical separability constraint training algorithm  
We choose the value of \(\lambda\) for the supervised separability constraint part empirically, by modifying its value for different trainings and evaluate its gain for later stages.