# Google Gemini's Performance in Endodontics: A Study on Answer Precision and Reliability
## 4 Discussion  
The introduction of new technologies in the healthcare sector has had a transformative effect on the diagnosis and treatment of diseases as well as facilitating the work of healthcare professionals. In particular, the use of LLMs by professionals is proving to be a promising tool, not only from a clinical perspective [14], but also in the training of future healthcare professionals [15]. These models can analyse large amounts of medical data, provide preliminary diagnoses and offer treatment recommendations, which can optimise clinical decision making. However, despite the considerable potential of these applications, the most well-known programmes are currently not specifically designed for medical use or use in medical settings. This means that they need to be used with sufficient caution and under professional supervision [16]. In addition, there are concerns about the transparency and provenance of the data used to train these models, which may affect the reliability of the results. Although ChatGPT versions 3.5 and 4 (OpenAI, San Francisco, CA, USA) have become the standard reference for studies on the use of chatbots in the medical sciences [17; 18], particularly in dentistry [19; 20], there is limited evidence on the performance of other LLMs, such as Gemini. Figure 1: Answer obtained to question 29 in all its repetitions. \begin{table}
\begin{tabular}{c c c c c c c} \hline  & \multicolumn{2}{c}{**Incorrect (0)**} & \multicolumn{2}{c}{**Partially Correct or Incomplete (1)**} & \multicolumn{2}{c}{**Correct (2)**} \\ \hline
**Question** & **n** & **\%** & **n** & **\%** & **n** & **\%** \\ \hline
26 & 25 & 83.33 & 1 & 3.33 & 4 & 13.33 \\ \hline
27 & 0 & 0.00 & 0 & 0.00 & 30 & 100.00 \\ \hline
28 & 0 & 0.00 & 0 & 0.00 & 30 & 100.00 \\ \hline
29 & 27 & 90.00 & 3 & 10.00 & 0 & 0.00 \\ \hline
30 & 0 & 0.00 & 0 & 0.00 & 30 & 100.00 \\ \hline \end{tabular}
\end{table} 
Table 2: _Cont._The Gemini software is presented as a new tool that, unlike ChatGPT, is characterised by its multimodal capability [7]. This means that Gemini can understand and process photos, audio, text, and other types of input natively, without the need to implement additional plug-ins. Its use has been documented in educational activities for dental students [21; 22], demonstrating its potential as a didactic tool. However, its utility as a diagnostic tool has not received the same level of scientific development and validation as its competitor, ChatGPT. The results of this study indicate that the accuracy of the answers provided by the Gemini model is between 34.02% and 40.32% with 95% confidence. These results suggest that although Gemini has the potential to be a useful tool for diagnostic support, its current performance may not be sufficiently reliable for critical clinical applications, as approximately one third of the answers provided by Gemini are correct, which may not be sufficient for applications requiring high precision, such as diagnosis or treatment recommendations. Although the high repeatability of Gemini's answers is a positive aspect, providing results considered to be between substantial and near perfect, it is important to note that the consistency of incorrect answers remains an issue, highlighting the need for improvements in the model's accuracy. In this context, [PERSON] et al. [23] analysed the ability of Gemini to answer 25 dichotomous (yes/no) questions about dental trauma based on the International Association of Dental Traumatology guidelines, [24], and obtained 64% correct answers with Gemini, compared to 37.11% in the present study. However, [PERSON] et al. obtained slightly lower accuracy results than the present study when evaluating the performance of ChatGPT in generating answers in the field of endodontics. In another study [25], ChatGPT-3.5 provided more credible information compared to Bing and Google Bard. Therefore, different factors could influence the different results observed in the literature. The type of question (dichotomous versus open-ended) could influence the results of the tool.