## 7 Parameters Selection
### Cnn Parameters Selection  
The first parameter analyzed for the CNN model and the pump dataset was the size of the input image representing a piece of the MFCC-based features of size \(96\times M\). The value of \(M\) varied in the range [32; 160]. According to the plot in Figure 9(a), representing the accuracy as a function of \(M\), the optimal value is 80. However, using this value, the variance in the k-fold cross-validation is high even when the accuracy could be high. Consequently, we selected a higher value of 128 to reduce the variance. The batch size was varied in the range [16; 256], and the accuracy variation is shown in Figure 9(b). Lower values of this parameter provide high accuracy. However, the loss convergence is noisy, and the variance in the k-fold cross-validation could be higher. We set this parameter as 136, which provides a more \"clean\" loss convergence, i.e, reduced local variance of the loss function with lower variance in the k-fold cross-validation. An in-depth discussion concerning the relation between batch size and loss variance is reported in [58].  
The dropout probability was varied in the interval [0.01, 0.6], and the accuracy obtained is shown in Figure 9(c). The curves suggest using a probability value of 0.3 to attain the highest accuracy. Concerning the initial learning rate, their values were varied in the range [0.0005, 0.07], and the variation of accuracy is shown in Figure 9(d). The curve suggests setting the initial learning rate as 0.01, the default value. Once the previous parameters were set, an additional test was performed to choose the optimization solver. Several Stochastic Gradient Descent (SGD) based solvers were tested. In particular, the Stochastic Gradient Descent with Momentum (SGDM) [59] is capable of improving the SGD performance when dealing with surface curves that are steeper in one direction than the rest, particularly for Recursive Neural Networks (such as LSTM). An Alternative variant of the SGD algorithm is the Adagrad algorithm [60] that can adapt the learning rate to the nature of the Neural Network parameter. It performs more extensive updates for infrequent parameters and is  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Equipment** & **Model** & **M** & **Batch Size** & **Dropout** & **Solver** & **Epochs** & **TT (s)** \\ \hline \multirow{3}{*}{Centrifugal Pump} & CNN & 128 & 136 & 0.3 & ADAM & 8 & 720.0 \\  & LSTM & 148 & 256 & NA & SCDM & 50 & 1210.0 \\  & BiLSTM & 148 & 256 & NA & SCDM & 60 & 2094.0 \\ \hline \multirow{3}{*}{Reciprocating Compressor} & CNN & 128 & 136 & 0.3 & ADAM & 8 & 590.4 \\  & LSTM & 136 & 256 & NA & SCDM & 100 & 1900.0 \\  & BiLSTM & 136 & 256 & NA & SCDM & 100 & 2420.0 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Parameters and training time (TT) for each models tested using MFCC features.  
well-suited for sparse data. Its main weakness is the accumulation of square gradients in the denominator that eventually leads to a severe reduction of the learning rate [46].  
Consequently, in this research, we tested other solvers that were able to resolve this limitation. In particular, we tested the Adam solver [61] and the RMSProp solver proposed by [PERSON] [62]. Considering the accuracy and the computation time spent while training, we selected the Adam solver in the case of the CNN model. The SGDM was selected for the LSTM and BiLSTM models.  
A similar procedure was followed for setting the hyper-parameters for each model considered and for each dataset.