## 3 Results
### Experimental Results
To demonstrate the out-of-sample utility of our model, we tested it in a dataset of clinical vignettes compiled from the Blackwell Clinical Vignettes for Behavioral Sciences [70]. We asked our model to predict the top-1 and top-3 most likely diagnoses for each vignette. We did not train the model on this dataset, making it a true out-of-sample test. We also asked three professional psychologists to perform the same task and calculated the top-1 and top-3 accuracies based on the majority vote of their answers. Some examples of vignettes and predictions are shown in Table 3. Our model achieved a top-1 accuracy of 0.54 and a top-3 accuracy of 0.86. On the other hand, the professionals attained a top-1 accuracy of 0.6 and a top-3 accuracy of 0.67. Interestingly, although the top-1 accuracy of the model was lower than that of the professionals, its top-3 accuracy was significantly better. As illustrated in Table 3, some vignettes are challenging to categorize, as the limited description of symptoms can fit into many categories. Consider, for example, the second vignette, where the correct label is dementia (Neurocognitive Disorder category) but the model predicted a Psychotic Disorder. The symptoms are very similar, and the accuracy of the model could be considerably increased if it had the capability to ask further questions. To test this, we simulated a question and answer process with the model, where the model asks the user to confirm or deny the presence of symptoms. We then updated the probabilities of the diseases based on the response of the user and repeated the process for \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Category** & **Precision** & **Recall** & **F1-Score** & **Support** \\ \hline Trauma and Stressor-Related Disorders & 0.69 & 0.66 & 0.68 & 803 \\ Substance-Related and Addictive Disorders & 0.88 & 0.84 & 0.86 & 668 \\ Anxiety Disorders & 0.79 & 0.80 & 0.80 & 1840 \\ Neurocognitive Disorders & 1.00 & 0.12 & 0.22 & 74 \\ Neurodecupmental Disorders & 0.83 & 0.61 & 0.70 & 240 \\ Dissociative Disorders & 0.75 & 0.77 & 0.76 & 1400 \\ Feeding and Eating Disorders & 0.85 & 0.85 & 0.85 & 239 \\ Somatic Symptom and Related Disorders & 0.90 & 0.37 & 0.52 & 52 \\ Impulse-Control and Conduct Disorders & 0.86 & 0.70 & 0.77 & 163 \\ Depressive Disorders & 0.78 & 0.77 & 0.77 & 2263 \\ Personality Disorders & 0.79 & 0.84 & 0.81 & 4851 \\ Schizophrenia Spectrum and Other Psychotic Disorders & 0.71 & 0.70 & 0.71 & 1032 \\ Sexual Dysfunctions & 0.80 & 0.78 & 0.79 & 942 \\ Sleep–Wake Disorders & 0.67 & 0.76 & 0.71 & 123 \\ \hline Macro Avg & 0.81 & 0.68 & 0.71 & 15,170 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results of the LoRA-fine-tuned model on the extended dataset. a total of \(t\) iterations. In each step, \(q\) symptoms are asked to the user, who can confirm or deny the presence of the symptom. This simulates a symptom checking process, where the symptom checker asks the user to confirm or deny the presence of symptoms in the format of multiple choice questions. We tested the model with \(q=1\), \(q=4\), and \(q=8\) symptoms asked as multiple choice questions per iteration and for \(t=1,3,5,8\) refinement iterations. The results are summarized in Figure 5.  
Interestingly, a decline in performance is observed after one iteration of the symptom refinement process, something that could be attributed to several factors; however, primarily, it seems that the posterior probabilities of the conditions \(D\) are strongly influenced by both the initial prior probabilities and the conditional probabilities of the symptoms given the conditions. In the absence of additional information, these conditional probabilities, calculated with little evidence yet, exert a stronger influence than the prior probabilities initially determined by the predictive BERT model, which already makes a good prediction based on the textual description of symptoms. However, the situation changes after three iterations of symptom confirmation.