# A New Measure for Serious Games Evaluation: Gaming Educational Balanced (GEB) Model
## 3 Results
### Statistical and Machine-Learning Validation of the Games Evaluation
Figure 7: Violin plot analysis between the different model variables for each game. to classify each game. Obviously, there are some users' surveys mixed up (lower right corner) when only those two first principal components are considered; however, as PCA is able to group the three games clearly, more complex classifiers will be able to extract more information from these datasets. Fourthly, different machine learning classifiers were tested to evaluate if the survey results contain useful information. The Weka software tool was used for this task under a 10 \(\times\) 10 validation scheme to assure the generalization of the obtained results. Machine learning algorithms are usually tested against a baseline method. The most common one is Naive-Bayes (called ZeroR in Weka), that always predicts the majority class. For this test, the game was considered as a class, or value to be predicted. Therefore, the class had three different possible values (1 for Neverending Nightmares, 2 for Actual Sunlight and 3 for Sym). The considered inputs were the 12 concepts evaluated in the GEB model plus a binary input about the level of expertise of the user in games (0: low experience in gaming, 1: high experience). The tested machine learning techniques were k-nearest neighbours (kNN), Decision Trees, Multilayer Perceptron and Random Forest. KNN is a very simple classifier that uses the closest example of any new data to build the prediction for the new one. Decision Trees are simple, unstable but visual classifiers. Multilayer Perceptions are able to predict complex relationships between inputs and outputs and are often used as a standard in machine learning tasks. Random Forest is an ensemble, a more recent machine learning technique, very insensitive to parameters tuning. Table 3 summarizes the capability of those models to predict the evaluated game, considering the score given to a game by a user and his videogame playing habits. This capability was evaluated in terms of accuracy and the Receiver Operating Characteristic (ROC) Area. The accuracy is defined as the evaluations properly classified divided by the total of evaluations. The ROC area is the area under the Receiver Operating Characteristic curve and it is a more complex indicator, representing the ratio between the true positives and the false positives. Both quality indicators, accuracy and ROC area, are better the higher they are (accuracy close to 100% and ROC area close to 1.00). All the machine learning methods have been tested with their default values in Weka to simplify the tuning process, because a very high model accuracy was not the main objective of this analysis, but the level of hidden information in the dataset. Figure 8: PCA representation for each game (different colours). Table 3 shows firstly that all machine learning techniques are statistically significantly better than the baseline method, the Naive-Bayes. Therefore, it can be assured that the GEBs surveys contain information different and coherent for each game, independent of the user who has filled in the survey. Secondly, this information is structured in a complex way, because more complex classifiers like artificial intelligence networks or ensembles achieve a better performance than more simple classifiers, like decision trees or kNNs. Thirdly, the hidden information cannot be extracted by just considering the answer from the previous user who gave the closest answer in the survey, because in this case kNN would be the most accurate method; this result is expected because the evaluation of a game by humans is a noisy task, and therefore many hidden and personal human factors might affect the evaluation of each user, making the use of the GEB metric a noisy task but also a predictable task. In this way, although users are different in their metrics values, there is coherence between their answers and the three games present a personal fingerprint that the machine learning algorithms can properly identify.