## 3 Results  
### Experimental Setup  
In this section, we present the experimental setup and results of our proposed methodology. We first describe the datasets used for training and evaluation, followed by the experimental results for the Reddit classification task and the extended dataset classification task. Finally, we present the results for the Bayesian network diagnosis refinement task. For our research, we used the \(\text{BERT}_{base}\) model. The \(\text{BERT}_{base}\) comprises 12 layers, 769 hidden units, and 12 heads within its multi-head attention mechanism, aggregating to a total of \(110M\) parameters. We used the HuggingFace implementation for these models [68]. The model was pretrained on the complete _PsyReddit_ dataset that covers submissions from 2007 to 2021. Training was carried out over three epochs, amounting to \(203,661\) steps. In general, training strategies (Warmup, Weight Decay, Adam) and default hyperparameters recommended by the authors of BERT [58] were used unless otherwise stated. For comparison purposes, after pretraining, the models were fine-tuned to predict the class or subreddit of submissions between the years 2017 and 2018, following the methodology in [48] for a valid comparison. Like the previous authors, the dataset was split into 80% training and 20% testing, and trained on the former set for the multiclass classification task for the 11 classes defined by the chosen subreddits in the reference. The models were fine-tuned for three epochs in this dataset. To select the hyperparameters for both BERT and LoRA models, we employed a grid search strategy. For LoRA, \(r=4,8,16\) were tested, with \(r=8\) selected based on its accuracy on the validation set of the fine-tuning dataset. For BERT, we explored maximum sequence lengths of 128 and 256. Pretraining parameters were set with a learning rate of \(2\times 10^{-5}\), weight decay of 0.01, and warmup steps of 10. During fine-tuning, we adjusted the learning rate to \(5\times 10^{-5}\). To obtain a model useful for identifying general conditions in mental health-related posts, we fine-tuned the model using our newly proposed extended dataset, which contains a much more diverse array of diagnoses. We trained the model for three epochs and chose the best model based on validation performance. The final model is capable of identifying among 18 clinically valid diagnostic categories. We selected the best-performing model from the previous step to fine-tune on this dataset. Furthermore, adhering to standard practices in the evaluation of symptom checking systems [4, 69], we created a validation set using clinical vignettes, or clinical cases, within the realm of mental health. Clinical vignettes are detailed representative cases used in the training and evaluation of health professionals. These cases are known for their complexity and resemblance to real-life scenarios that are challenging to diagnose, yet are reflective of actual clinical experiences. We sourced these clinical vignettes from the Blackwell Clinical Vignettes for Behavioral Sciences textbook [70], a resource commonly used in professional training. From this textbook, we selected 37 vignettes. Our task for the model was to identify the most likely diagnosis category, as well as the top three most probable categories, for each vignette. To establish a realistic benchmark for comparison, we asked three professional psychologists to determine the three most probable conditions for each vignette from the list of available condition labels. Their consensus was used to calculate the top-1 and top-3 accuracies. For a practical illustration, some examples of these vignettes and the corresponding predictions that our model generated are presented in Table 3.  
Our main objective was to determine whether the incorporation of a refinement process using a Bayesian network could enhance the initial classification results. To mimic the symptom-checking procedure, we simulated a scenario in which a user responds to the symptoms identified by the system in each interaction.