## 4 Representative Machine Learning Models for IONM-Related Research
### Hyperparameters for Each ML Model  
Each ML model has a set of hyperparameters to specify the model size, the types of the components, the strength of regularization, and other properties. The performance of the ML model is significantly affected by the hyperparameters [60; 61]. Table 2 lists the major hyperparameters of ML models. Some hyperparameters in Table 2 specify training options, such as the batch size, learning rate, maximum number of training iterations, early stopping criteria, and algorithms for initialization and optimization, which are important for training ML models.  
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline \hline \multicolumn{2}{c}{**ML Models**} & \multicolumn{1}{c}{**Major Hyperparameters**} \\ \hline \multirow{4}{*}{ANN} & The number of layers & \\  & The number of units in each layer & \\  & The type of activation functions (e.g., ReLU, sigmoid, tanh, softmax, ELU, swish, mish) & \\  & Training hyperparameters (e.g., batch size, learning rate, maximum number of iterations, early stopping criteria, and the algorithms for initialization and optimization) & \\ \hline \multirow{4}{*}{CNN} & All of the hyperparameters of artificial neural networks & \\  & The type of each layer (e.g., conv., max-pooling, batch-norm, dropout, group conv.) & \\  & The width, height, and channel of each layer & \\  & Kernel size, stride, padding & \\  & Existence of skip connection & \\ \hline \multirow{4}{*}{RNN (including LISM and GRU)} & All of the hyperparameters of artificial neural networks & \\  & Unidirectional or bidirectional & \\  & The size of cell blocks (LSTM) & \\ \hline \multirow{4}{*}{Transformers} & The number of layers & \\  & The total dimension of hidden features & \\  & The number of heads in the multi-head attention & \\  & The dimension of key and value & \\  & The dimension of MLP layers & \\  & All of the training hyperparameters of artificial neural networks & \\ \hline \multirow{4}{*}{SVM} & The weight of soft margin & \\  & The type of kernel (e.g., polynomial, Gaussian, RBF) and its parameters (e.g., \(\gamma\) of RBF kernel) & \\  & Training hyperparameters (e.g., learning rate, number of iterations) & \\ \hline Regularized logistic regression & Regularization factors & \\  & Training hyperparameters (e.g., learning rate, number of iterations) & \\ \hline \multirow{4}{*}{Random forests} & The number of trees & \\  & Maximum depth of the tree & \\  & Maximum number of leaf nodes & \\  & Quality measure of a split (e.g., Gini, entropy, log\_loss) & \\  & Regularization factor & \\ \hline \multirow{4}{*}{XG-Boost} & The number of trees & \\  & Maximum depth of the tree & \\  & Type of booster and its parameters (e.g., learning rate, gamma, max delta step) & \\ \cline{1-1}  & Regularization factors & \\ \hline \hline \end{tabular}  
* ANN, artificial neural network; ReLU, rectified linear unit; ELU, exponential linear unit; CNN, convolutional neural network; RNN, recurrent neural networks; LSTM, long short-term memory; GRU, gated recurrent unit; MLP, multi-layer perceptron; SVM, support vector machines; RBF, radial basis function; XG, extreme gradient.  
\end{table}
Table 2: Important hyperparameters of machine learning models.