## 4 Results
### Implementation Details  
As our SPI-Net implementation relies on multiple networks being trained independently and then concatenated for fine-tuning, we firstly here present our entire training setup to obtain SPI-Net:  
* Training the Geometric features branch:
* Training the auto-encoder with a separability constraint term: We use a standard feed-forward non-recurrent MLP, the dimensions of which are \((3000)\rightarrow(128)\rightarrow(64)\rightarrow(32)\rightarrow(1) \rightarrow(32)\rightarrow(64)\rightarrow(128)\rightarrow(3000)\). We use a value of fixed \(\lambda=5\) for the LDA constraint term ponderation in the modified reconstruction cost function. To address the vanishing gradient problem, each perceptron in the given auto-encoder network uses the LeakyRelu [77] activation function. For regularization purposes, we use Dropout [78] (\(p=0.5\)), \(L_{2}\) regularization with \(\lambda=1^{-1}\) and batch normalization [79] after each layer. We choose Adam (\(\beta_{1}=0.9,\beta_{2}=0.999\)) [80] as the optimizer, with an annealing learning rate that drops from \(1^{-3}\) to \(1^{-8}\). In order to obtain a good separability in the latent space with the LDA separability constraint, we choose to send all the training examples at once for the auto-encoder training and select a batch size of 240.
* Training the Encoder part for classification: we recover the encoder part of the auto-encoder, then train a classifier with weights initialized via the auto-encoder. We use the same values of the Adam optimizer for training. We, however, divide the training set into 30 batches of size 8. We use _ReduceLROnPlateau_ with a factor of 0.2 and patience of 10.
* Training the Cartesian features branch: The Cartesian features branch is composed of four 2D-convolution blocks composed of 2D-convolution layers (kernel size \(=3\times 3\)). Similarly to the auto-encoder, we use the LeakyRelu activation function, \(L_{2}\) regularization with \(\lambda=1^{-4}\) and a Dropout value of 0.5. Each convolution layer is then followed by a Batch Normalization layer and an Average Pooling layer. The fully connected layers following the spatio-temporal features extraction done by convolutions is then completely similar to any other Dense layer of the Geometric feature branch for hyper-parameters tuning. We choose Adam (\(\beta_{1}=0.9,\beta_{2}=0.999\)) with a learning rate that drops from \(1^{-2}\) to \(1^{-8}\) and _ReduceLROnPlateau_ with a factor of 0.5, patience of 5, cooldown of 5 and a batch size of 8.
* Concatenating the branches: We then remove the classification layer of each branch and concatenate those two networks deprived of their last layer into a single one. It allows us to keep the previously learned weights of each network independently. We then add a classification layer in which the weights are initialized randomly after the concatenated layer of the obtained network. Finally, we fine-tune the entire network, from pre-trained weights to the randomly initialized classification layer. We get our presented SPI-Net: a late fusion and fine-tuned version of the Geometric and Cartesian features branches. As proposed in [81], we increase the batch size over time during the training and therefore fine-tune the approach with two different trainings on the same SPI-network with two different batch sizes. For the first training, we use Adam with a learning rate that drops from \(9^{-3}\) to \(5^{-8}\), _ReduceLROnPlateau_ with a factor of 0.5, patience of 25 and a batch size of 8. For the second one, we use [PERSON] with a learning rate that drops from \(9^{-8}\) to \(5^{-18}\) and _ReduceLROnPlateau_ with a factor of 0.5, patience of 25 and a batch size of 240.