# Detecting Plasma Detachment in the Wendelstein 7-X Stellarator Using Machine Learning
## 3 Results
### Best Model  
From this point we used data gathered from profile A on all our models. To find the most accurate model, we performed hyperparameter search using the grid search cross fold validation method. In machine- and deep learning the model specific parameters which are used to control the learning process are called hyperparameters, these are different from the previously mentioned weight parameters, because they cannot be modified while training. Some examples for hyperparameters are learning rate, the method of optimization algorithm, etc.  
During grid search we define a range of hyperparameters, at each iteration we choose a random combination of such parameters, train the models on these using the training set, calculate their \(F1\) score on a random subset of the validation set and in the end, after a given number of iterations, we adopt the best performing combinations for the models. The advantage of applying cross fold validation is that at each iteration we calculate the \(F1\) scores on random subsets of a given validation set, thus testing the model's performance on new data that was not seen before, we can avoid overfitting and see how the model generalizes to new independent data.  
The results can be seen in Table 3. On the training set the logistic regression and the support vector machine models have the best performances, almost identical. However, when calculating the \(F1\) scores on the previously not seen two discharges, the test dataset, the logistic regression by far beats the other two models.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**A**} & \multicolumn{4}{c}{**B**} & \multicolumn{4}{c}{**C**} \\ \hline  & TPR & \(FPR\) & \(F1\) & AUC & TPR & \(FPR\) & \(F1\) & AUC & TPR & \(FPR\) & \(F1\) & AUC \\ \hline Logistic Regression & 0.978 & 0.060 & **0.901** & **0.977** & **0.983** & 0.077 & 0.883 & 0.960 & 0.951 & **0.060** & 0.888 & 0.958 \\ \hline Support Vector Machine & 0.966 & **0.060** & **0.897** & 0.983 & 0.985 & 0.087 & **0.897** & 0.962 & 0.956 & 0.069 & 0.879 & 0.957 \\ \hline Random Forest & 0.990 & **0.060** & **0.907** & **0.990** & 0.985 & 0.066 & 0.871 & 0.982 & **0.995** & 0.070 & 0.897 & 0.983 \\ \hline \hline \end{tabular}
\end{table}
Table 2: TPR, \(FPR\), \(F1\) and AUC scores of the models on the separate profiles. Ee indicated the highest score (or lowest for \(FPR\)) per model with bold and green text, if we had matching best values, we used only bold text.  
The predictions of the three models on one of the test shots, namely 20181010.035 is seen on Figure 6. From this we can identify where the logistic regression exactly beats the other models. The support vector machine and the random forest makes a lot more false positive predictions, particularly before the beginning of the detached interval. All three models perform well inside this interval, but they also have a common mistake.  
We know that the transition to detachment takes place gradually, but we used binary indicators in our datasets, a consequence of such actions is that we cannot determine the exact point in time where the turnover to detachment takes place. Relying on our experiences gained while working with the dataset, we could establish there is a maximum of 40-frame-long \"transition zone\", indicated with gray-shaded areas on the figures. It is important to note that the F1 scores were not calculated in these intervals.  
Before performing the analysis of the models, our precondition was, that the final algorithm must have at least an F1 score of 0.9 on the test dataset. Since the logistic regression could only produce this number, that is the model we used going forward. For further analysis on the models performance we refer to Appendix A.