## 2 Materials and Methods
### Confusion Matrix Calculation  
To compute the confusion matrix of each model, the following criteria was adopted:  
* In the event that the model correctly identifies the diagnosis on the first listed answer, it is classified as a true positive (TP). * Should the correct diagnosis be identified on the second listed answer, it is recorded as one true positive (TP) and one false positive (FP). * If the correct diagnosis appears as the third listed answer, it is categorized as two false positives (FPs) and one true positive (TP). * In instances where no correct diagnosis is provided, the result is recorded as three false positives (FPs) and one false negative (FN). It is important to note that there are no true negatives (TNs) in this evaluation, as all cases under consideration represent actual patients. The fact that no TNs where considered is a first approach to the problem, used here to simplify the analysis because we are not evaluating what would happen if the models were presented with a person that does not suffer from any heart disease. Precision is calculated as \(P=\
icefrac{{TP}}{{TP}+FP}\), sensitivity as \(S=\
icefrac{{TP}}{{TP}+FN}\), and accuracy as \(A=\
icefrac{{TP}}{{\dividebox[1]{\dividebox[  
To determine similar patterns in the models, we apply a clustering algorithm. The Calinski-Harabasz index [27] was used to determine the optimal number of clusters, exploring from 2 to 7. The optimal value was found to be 5. Hence, we used the \(K\)-means algorithm with \(K=5\) using the accuracy, precision, and sensitivity as features. The results of this pattern analysis are shown in Figure 3. \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline  & **OH** & **MM** & **NM** & **TS** & **CS** & **Llama** & **Palm** & **CGPT** & **Claude** \\ \hline
**\# Cases** & 20 & 20 & 20 & 20 & 20 & 20 & 20 & 20 & 20 \\
**FP** & 41 & 33 & 34 & 34 & 18 & 26 & 39 & 22 & 27 \\
**TP** & 10 & 6 & 11 & 12 & 17 & 14 & 9 & 14 & 15 \\
**FN** & 10 & 13 & 9 & 8 & 3 & 7 & 11 & 6 & 6 \\
**TN** & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
**Accuracy** & 50\% & 30\% & 55\% & 60\% & 85\% & 70\% & 45\% & 70\% & 75\% \\
**Precision** & 20\% & 15\% & 24\% & 26\% & 49\% & 35\% & 19\% & 39\% & 36\% \\
**Sensitivity** & 50\% & 32\% & 55\% & 60\% & 85\% & 67\% & 45\% & 70\% & 71\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Confusion matrix of each model, together with the accuracy, precision, and sensitivity. Specificity is zero for all the models as there are no TNs. Figure 2: Best regression line between the number of parameters (in billions) and the values obtained for accuracy (**left**), precision (**centre**) and sensitivity (**right**). The cluster analysis revealed five distinct groups among the nine models based on their accuracy, precision, and sensitivity. In Table 4 the key characteristics of each group are summarized, including the models, their performance metrics (accuracy, precision, and sensitivity), and notable characteristics or biases. The first group consists of model CS (5), which stands out for exceptional performance with the highest accuracy (85%), precision (49%) and sensitivity (85%) compared to all other models. This model stands out for correctly identifying true positive cases, while maintaining low false positive and false negative rates, suggesting a possible bias towards optimism, and potentially underestimating the severity of conditions. The second group consists of models CGPT (8), Llama (6) and Claude (9), which have moderate to high accuracy, precision, and sensitivity values. In particular, CGPT has an accuracy of 70%, a precision of 39% and a sensitivity of 70%, while Llama has an accuracy of 70%, a precision of 35% and a sensitivity of 67%. Claude follows closely with 75% accuracy, 36% precision and 71% sensitivity. These models are biased towards sensitivity, indicating their ability to correctly identify true positives, but also generate a moderate number of false positives.