# Gait Recognition via Deep Learning of the Center-of-Pressure Trajectory
## Materials and Methods
### Hyperparameter Tuning and Model Testing  
Table 1 summarizes how the CNN hyperparameters were tuned. Regarding the model architecture, both the number of filters and the number of intermediate blocks (Figure 3) were adjusted. I also evaluated two different approaches for the final layers--the first using a classical combination of dense-dropout-softmax layers, and the second using a global average pooling layer [40] preceding the softmax layer.  
Figure 3: Architecture of the deep convolutional neural network (CNN). Overall drawing displays the general characteristics of the CNNs. The arrows show the residual shortcuts (ResNet). The number of intermediate blocks was adjusted according to the hyperparameter tuning results. The weights of the trainable block were tuned in the transfer-learning analysis. For a detailed drawing of the final models, see the supplementary online files.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Hyperparameters** & **Method** & \multicolumn{2}{c}{**Values**} & \multicolumn{2}{c}{**Best Results**} \\  & & **CNN** & **sepCNN** \\ \hline CNN architecture & & & & \\ Filter size in layers & Choice & A: 15, 13, 11, 11, [11, 11, 11], 3, 2 & C & C \\  & & B: 11, 9, 7, 7, [7, 7, 7], 3, 2 & & \\  & C: 9, 7, 5, 5, [5, 5, 5], 3 & 2 & & \\ Number of filters in layers & Choice & A: 16, 32, 64, [64, 64, 64], 148, (128) & B & D \\  & & B: 32, 32, 64, 128, [128, 128, 128], 128, 256, (256) & & \\  & & C: 64, 64, 128, 256, [256, 256, 256, 256], 56, 512, (512) & & \\  & & D: 128, 128, 256, 512, [512, 512, 512], 512, 1024, (1024) & & \\ Number of intermediate blocks & Choice & & 2 & 1 \\ Top-layer configuration & Choice & A: Global Average Pooling + Dense (Softmax) & A & A \\  & & B: Flatten + Dense + Dropout + Dense (Softmax) & & \\ Weight initialization & Choice & A: Glorot (Xavier) normal initializer & A & A \\  & & B: He normal initializer & & \\ Activation & Choice & ReLU & Swish & Swish \\  & & LeakyReLU & & \\  & & PReLU & & \\  & & Trainable Swish & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Hyperparameter tuning. Bayesian optimization was used to explore the hyperparameter space in 300 trials. The optimal choices are shown in the last two columns.  
Regarding activation, four algorithms were tested: ReLU [41], LeakyReLU [41], PReLU [42], and trainable Swish [43]. Swish is a recent algorithm, similar to the sigmoid-weighted linear unit proposed in [44], but with a trainable parameter. Regarding convolutional layer initialization, two algorithms were tested, the so-called Glorot-normal (Xavier-normal, [45]) and the He-normal [42]. Regarding L2 regularization, the optimal weight decay (\(\lambda\)) was searched for between \(10^{-7}\) and \(10^{-3}\). Finally, the optimal initial learning rate was searched for between 0.1 and 2 times the recommended value of 0.02; that is, between 0.002 and 0.04.  
Bayesian optimization was used to search through the hyperparameter space. More precisely, I applied the tree of Parzen estimator (TPE) algorithm [46], as implemented in the Hyperopt library [47], and its Keras wrapper, Hyperas [48]. The overall accuracy on the dev set was used as the metric. Three-hundred trials were run and the combination that provided the highest accuracy was chosen.  
A final training set was built by concatenating the train and dev sets. The accuracy of the best setting (Table 1) was assessed in the test set. An early-stopping algorithm was applied to reduce the training time. The assessment was repeated 10 times to take model stochasticity into account (Table 2).