# A Granularity-Based Intelligent Tutoring System for Zooarchaeology
## 3 Results
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Method (Parameters)** & **Accuracy** & \begin{tabular}{c} **Precision** \\ **(Weighted)** \\ \end{tabular} & \
\begin{tabular}{c} **Recall** \\ **(Weighted)** \\ \end{tabular} &
\begin{tabular}{c} **F1-Score** \\ **(Weighted)** \\ \end{tabular} \\ \hline Random forest, SMOTE (classifier class weight: & & & & \\ balanced, classifier max features: auto, classifier n & 0.86 & 0.86 & 0.86 & 0.86 \\ estimators: 500) & & & & \\ \hline SVM, SMOTE (classifier C: 100, classifier gamma: & & & & \\
0.001, classifier kernel: rbf) & 0.74 & 0.81 & 0.74 & 0.77 \\ \hline Naive Bayes, SMOTE & 0.68 & 0.76 & 0.68 & 0.66 \\ \hline Neural Networks, SMOTE (classifier solver: lbfgs, & & & & \\ classifier alpha: \(1\times 10^{-5}\)) & 0.67 & 0.75 & 0.67 & 0.71 \\ \hline KNN, SMOTE (classifier n neighbors: 3) & 0.75 & 0.79 & 0.75 & 0.77 \\ \hline Random Forest, ADASYN (classifier class weight: & & & & \\ balanced, classifier max features: auto, classifier n & 0.86 & 0.85 & 0.86 & 0.86 \\ estimators: 100) & & & & \\ \hline SVM, ADASYN (classifier C: 100, classifier gamma: & & & & \\
0.001, classifier kernel: rbf) & 0.72 & 0.80 & 0.72 & 0.75 \\ \hline Naive Bayes, ADASYN & 0.68 & 0.78 & 0.68 & 0.65 \\ \hline Neural Networks, ADASYN (classifier solver: & & & & \\ adam, classifier alpha: \(1\times 10^{-5}\)) & 0.67 & 0.79 & 0.67 & 0.72 \\ \hline KNN, ADASYN (classifier n neighbors: 3) & 0.74 & 0.79 & 0.74 & 0.76 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance of the different methods in the coarse granularity. Synthetic minority over-sampling technique (SMOTE), support vector machine (SVM), k-nearest neighbors (KNN), adaptive synthetic (ADASYN). Figure 4: Confusion matrix of the coarse granularity. Figure 5: Confusion matrix of the medium granularity. This first version of the ITS included the coarse granularity and was used in August 2019 by 3 students to classify 51 remains that were characterized following the format of the same database but belonged to a different archaeological site called \"Abric del Pastor\". Abric del Pastor belongs also to Middle Palaeolithic and is near \"El Salt\". Two of the three students also answered the questionnaire. The results of the tutor, compared to the predictions of the students frequently agreed even when the remains used for training corresponded to a different site (\"El Salt\"). Students, due to their limited knowledge and the difficulties of analyzing very fragmented bone remains, usually described the remains based only on the size, which usually agree with the families predicted by the tutor. For example, a lot of bone remains predicted as \"small size\" are described by the tutor as Bovidae, which is a family included in the category of small animals because most bovids belong to the wild goat (_Capra pyrenatica_). This information can help students to give more accurate answers or verify their initial guess. On the other side, sometimes the tutor fails and not all the sizes and families are correctly classified. For example, some bones that are correctly described as \"small size\" by students are characterized as Leporidae by the tutor, which is a mistake. Other mistakes are related to the wrong characterization of birds as Leporidae. Figure 6: Confusion matrix of the fine granularity. Analyzing the results of the questionnaire, students appreciated the use of the tutor as a complementary tool in their activities. In the subjective open answer, they highlighted two concepts: (1) The need of improving the interface and (2) the need of increasing the accuracy of the predictions. Figure 7: Tutor interface to introduce the bone fragments.