# Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format
## 6 Results
### Resource Distribution Analysis  
Figure 3 presents a bar chart that illustrates the distribution of various resource types per bundle, where each bundle corresponds to one clinical report. The data are categorized by different models and clinics, specifically comparing the performance of Gemini and Mistral models in handling mammography and skin care reports from two distinct clinics. The chart highlights the count of different resource types, such as Condition, Observation, \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{2}{c}{} & \multicolumn{5}{c}{**Failure Rate**} \\ \hline \multirow{2}{*}{**Clinic**} & \multirow{2}{*}{**Model Name**} & \multirow{2}{*}{**Prompt Tech.**} & \multicolumn{2}{c}{**Zero-Shot**} & \multicolumn{2}{c}{**Few-Shot**} \\ \cline{3-6}  & & & **Low** & **High** & **Low** & **High** \\ \hline \multirow{5}{*}{Skin Care} & \multirow{5}{*}{Gemini} & Basic & 73\% & 82\% & 45\% & 18\% \\ \cline{3-6}  & & Two-Step & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) \\ \cline{3-6}  & & CoT & 27\% & 27\% & 18\% & \(\surd\) \\ \cline{3-6}  & & Basic & \(\times\) & 91\% & 31\% & \(\surd\) \\ \cline{3-6}  & & Two-Step & \(\surd\) & 10\% & 35\% & \(\surd\) \\ \cline{3-6}  & & CoT & 55\% & 64\% & \(\surd\) & \(\surd\) \\ \cline{3-6}  & & Basic & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\ \cline{3-6}  & & Two-Step & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\ \cline{3-6}  & & CoT & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\ \hline \multirow{5}{*}{Mammography} & \multirow{5}{*}{Gemini} & Basic & 89\% & 78\% & \(\surd\) & \(\surd\) \\ \cline{3-6}  & & Two-Step & \(\surd\) & \(\surd\) & \(\surd\) & \(\surd\) \\ \cline{3-6}  & & CoT & 56\% & 44\% & \(\surd\) & \(\surd\) \\ \cline{3-6}  & & Basic & \(\times\) & \(\times\) & 29\% & \(\surd\) \\ \cline{3-6}  & & Two-Step & \(\surd\) & \(\surd\) & 29\% & \(\surd\) \\ \cline{3-6}  & & CoT & 44\% & 40\% & \(\surd\) & \(\surd\) \\ \cline{3-6}  & & Basic & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\ \cline{3-6}  & & Two-Step & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\ \cline{3-6}  & & CoT & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation of empty bundle proportions post-verification with HAPI FHIR across various clinics, models, prompt techniques, and example presence at different temperatures. Lower proportions indicate better performance. A green tick (\(\surd\)) signifies a full conversion rate, while a red cross (\(\times\)) denotes a 100% failure rate. MedicationStatement, Procedure, AllergyIntolerance, Encounter, EpisodeOfCare, Patient, Practitioner, and Immunization, across these categories. This chart provides insights into the types of reports each clinic produces. Notably, the skin care clinic has a much greater number of MedicationStatement resources, which aim to capture the active or prescribed medications of the patient, compared to the mammography clinic, even when comparing across models. This disparity arises because the documents of the skin care clinic consist mainly of diagnoses with proposed treatments and, most of the time, include a summary of the current patient's situation, such as medications, condition, and allergies. This explains the presence of few AllergyIntolerance resources in the skin care clinic's reports, which are not used in the mammography documents. By contrast, the mammography reports contain more observation-like text, for example, findings of nodules and their characteristics and, in the majority of them, a BIRADS assessment, and in some cases, procedures that the patient will undergo due to a specific finding. Upon analyzing the charts and comparing the distribution among models, several key observations emerge. The Gemini model, particularly in skin care reports, shows a higher count of resources like Observation and MedicationStatement compared to the Mistral model. This suggests that the Gemini model might be more effective in capturing detailed clinical information in skin care reports.