## 4 Discussion  
While the cluster analysis revealed five distinct groups based on performance metrics, in the further evaluation we classified the models into four broader groups: pessimistic (MM), optimistic (CS), sensitive (NM, TS, Llama, CGPT and Claude) and generalist (OH and Palm). These groups allow for a better understanding of model trends and biases, which can inform their appropriate application and guide model refinement or selection based on desired characteristics. Based on the provided previous analysis and the characteristics of the models (number of parameters, fine-tuning, and open-source), the following relationships can be observed:  
1. Number of parameters: The models with a larger number of parameters tend to perform better in terms of accuracy, precision, and sensitivity, as shown in Figure 2. Model CS which belongs to the optimistic group and has the highest performance, has 70 billion parameters, which is among the highest in the dataset. Similarly, CGPT (about 70 billion parameters) and Claude (about 130 billion parameters), which belong to the sensitive group, also demonstrate relatively high performance. 2. Fine-tuning: Models that have undergone specific fine-tuning generally exhibit better performance compared to those without fine-tuning. For instance, CS (the optimistic model) and NM, Llama and OH (from the sensitive and generalist groups) have been fine-tuned for various use cases, which could contribute to their better performance. 3. Open-source: The relationship between open-source and model performance is less clear from the provided information. Both open-source and proprietary models are present in the sensitive and generalist groups, suggesting that open-source alone does not necessarily determine model performance. Although each open-source model was trained by different people using different methods, there is a possibility that an open-source model could outperform more like CS if the training and fine-tuning is done correctly. It is important to note that while these relationships can be observed, model performance is likely influenced by a combination of factors, including the model architecture, training data, and the specific task or domain the model is applied to. Other recent papers have approached the use of LLMs in medical contexts. For example, in [28] the authors evaluated GPT-4's performance in predicting patient admissions from emergency department visits, comparing it to traditional machine learning (ML) models, based in a retrospective study using data from 7 NYC hospitals. Focussing only in the GPT-4 results, Naive GPT-4 had limited performance but improved significantly with real-world examples and ML probabilities. GPT-4's peak performance was slightly lower than pure ML models but offers potential for providing reasoning behind predictions. Inanother interesting work [29] the authors developed a framework using the LLAMA-2 70B with a specially crafted prompt, to allow clinicians to query patients' entire echocardiogram report history in plain language. They used 10+ years of echocardiogram reports from patients with 10+ echocardiograms at Mount Sinai Health System. The study analysed 432 reports from 37 patients for a total of 100 question-answer pairs. The LLM model demonstrated high accuracy across various aspects of echocardiogram report analysis. It achieved a 90% overall success rate in answering questions. Further analysing, the model achieved perfect (100%) accuracy in retrieving diagnoses. It also performed well in severity assessment, with 93% accuracy. The model demonstrated a good ability to identify interventions (84% accuracy) and determine timeliness (83% accuracy). These results are in line with the values we report, taking into account that we did not use any elaborated prompts or specific adjustments. Finally, it should be noted that, as with any research, this work has its limitations. Firstly, since we have used SillyTavern for the open-source models, some of them may not be available at the time the reader accesses the site.