# Multimodal Classification of Anxiety Based on Physiological Signals
## 3 Materials and Methods
### Pipeline Description  
All the steps were implemented using a pipeline comprising feature scaling, supervised feature selection with the RFECV method, the application of several data balancing techniques, and the optimization of the hyperparameters of the different models trained.  
When applying the different data balancing techniques to our data while cross-validating, although the supervised feature selection technique selects the feature sets before the data balancing, the best feature set is chosen considering the classifier's performance after the data balancing technique and classifier hyperparameter tunning. Then, for each classifier and data balancing technique, the best combination of features may be different.  
The pseudocode of our pipeline is shown in Algorithm 1.  
```
Input :\(\mathit{ds}\) // dataset with features and labels sampling_techs // sampling techniques to balance the dataset learning_algs // learning algorithms and hyperparameters settings metrics // performance metrics  nr_outer // nr of outer stratified folds for cross-validation nr_inner // nr of inner stratified folds for cross-validation Output :overall_scores // scores of metrics by sampling technique and learner // split \(\mathit{ds}\) into \(\mathit{nr\_outer}\) stratified folds outer_cv = SplitData(\(\mathit{ds}\), \(\mathit{nr\_outer}\)) // for each sampling technique foreach sampl in sampling_techs do // for each learner and hyperparameters setting foreach(learn, learn_params) in learning_algs do // outer loop for performance estimation foreach(train_outer, test_outer) in outer_cv do  outer_scores = [], best_perf = 0 best_parms, best_grts = None // split train_outer into \(\mathit{mr\_inner}\) stratified folds inner_cv = SplitData(train_outer, \(\mathit{nr\_inner}\)) // inner loop for model tuning foreach(train_inner, test_inner) in inner_cv do // train a model on \(\mathit{train\_inner}\) and estimate performance in test_inner \(\mathit{fs}\) = RecursiveFeatureElim(\(\mathit{train\_inner}\)) train_inner, test_inner = ScalingData(\(train\_inner\)[\(\mathit{fs}\)], test_inner[\(\mathit{fs}\)]) model = BuildClassif(\(\mathit{learn}\), \(\mathit{learn\_params}\), \(\mathit{train\_inner}\)) perf = EstimatePerform(model, test_inner, 'F1') ifperf > best_perf then best_perf = perf  best_params = learn_params best_fs = fs // train a model on \(\mathit{train\_outer}\) and estimate performance in test_outer train_outer, test_outer = ScalingData(\(train\_outer\)[\(\mathit{best\_fs}\)], test_outer[\(\mathit{best\_fs}\)]) train_outer = SamplingData(\(\mathit{sampl}\), \(\mathit{train\_outer}\)) model = BuildClassif(\(\mathit{learn}\), \(\mathit{best\_params}\), \(\mathit{train\_outer}\)) scores = EstimatePerform(model, test_outer, metrics) outer_scores = append(outer_scores, scores) // for each learner, compute the average and standard error of metrics' scores scores[\(\mathit{learn}\)] = (avg(\(\mathit{outer\_scores}\)), \(\mathbf{se}\)(\(\mathit{outer\_scores}\))) // for each sampling technique and each learner, store the overall performance overall_scores[\(\mathit{sampl}\), \(\mathit{learn}\)] = scores[\(\mathit{learn}\)] returnoverall_scores
```  
**Algorithm 1**Pipeline for performance estimation of sampling techniques and learning algorithms for a set of metrics, using grid-search for model tuning