## 3 Proposed Procedure to Develop Predictive Models in Healthcare
### Stage 5: Predictive Model Generation and Evaluation  
In the next stage, a predictive model according to the established objective is designed based on the filtered database obtained in the previous stage. To do this, we must select those algorithms that are considered viable for the specific clinical project, such as decision trees, neural networks, or support vector machines (SVM), among others [63]. If the volume of data is very high, the use of specific support software can facilitate selecting a suitable algorithm by performing iterations before generating the full predictive model. For example, the IBM SPSS modeler classifier node (v. 18) can create several models and then compare them to select the best approach for a particular analysis. In this stage, the performance of the selected algorithms should be tested to choose the most convenient one to implement in the predictive model. To evaluate their stability and effectiveness, different cross-validation approaches can be considered [19; 30; 64; 65; 66]. The simplest method consists of separating the sample into two sub-samples: one to train the model and another one to test it (holdout method). Other more advanced methods include dividing the sample into k sub-samples (k-fold cross validation), stratifying the sample with the same percentage of each class (stratified k-fold cross validation), or even making as many combinations as the number of data instances (leave-one-out cross \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Patient** & **Gender** & **Age** & \begin{tabular}{c} **Educational** \\ **Level** \\ \end{tabular} & \\ \hline \begin{tabular}{c} **Clinical data: assessed** \\ **and reported data** \\ \end{tabular} & Contracture & Traffic accident & \begin{tabular}{c} Spinal column \\ alterations \\ \end{tabular} & WDQ \\ \hline \multirow{4}{*}{\begin{tabular}{c} **Clinical data: data** \\ **measured with sensors** \\ \end{tabular} } & Mean Speed [\({}^{\circ}\)/s] in: & Flex.-Ext. & Rotation & Lateralization \\ \cline{2-5}  & Max Speed [\({}^{\circ}\)/s] in: & \begin{tabular}{c} Flexion \\ Extension \\ \end{tabular} & \begin{tabular}{c} Right Rotation \\ Left Rotation \\ \end{tabular} & \begin{tabular}{c} Right Lateral \\ Left Lateral \\ \end{tabular} \\ \cline{2-5}  & Max Angle [\({}^{\circ}\)] in: & \
\begin{tabular}{c} Flexion \\ Extension \\ \end{tabular} &
\begin{tabular}{c} Right Rotation \\ Left Lateral \\ \end{tabular} \\ \cline{2-5}  & Total Range [\({}^{\circ}\)] in: & Flex.-Ext. & Rotation & Lateralization \\ \cline{2-5}  & Total Length [\({}^{\circ}\)] in: & Flex.-Ext. & Rotation & Lateralization \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Final variables considered in the case study after feature selection. validation). Furthermore, a validation set could be used (besides the \"test set\" and the \"training set\"), which is a set of examples used to tune the parameters of a classifier [67]. This is useful because the performance of the selected algorithms can be improved through parameter tuning, which consists of varying values of parameters of the algorithms in order to find the most suitable configuration for each of them. Once the suitable parameter configuration for each algorithm is selected, the performance of the different algorithms can be compared. It must be stressed that the most appropriate prediction method depends on the data. Besides, overfitting (a phenomenon that occurs when the adjustment of the model to the training data is too strong and, as a consequence, finds difficulties in obtaining suitable conclusions about unobserved data) should be avoided, as this would lead to a model that will only be able to make predictions for the data with which it has been trained [52]. There are several techniques to deal with this, such as regularization (smoothing the models), data augmentation (increasing the training data), early stopping, etc.