# Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format
## 8 Conclusions  
This study explored using LLMs to convert unstructured clinical reports to FHIR bundles, a key step for healthcare data interoperability. We investigated prompting strategies (basic, CoT, two-step), few-shot learning, temperature, and LLM selection (Gemini, Mistral, Llama3-Med42).  
Prompt engineering proved pivotal in achieving effective FHIR conversion. Basic prompting yielded the lowest precision (4.71) and highest hallucination rates (7.25), highlighting the necessity of detailed guidance for LLMs. The two-step conversion method achieved the highest precision (7.30) and resource mapping accuracy (7.32), significantly outperforming all other approaches. Chain-of-thought (CoT) prompting offered a moderate improvement over basic prompting, with a precision of 5.98, but did not match the two-step method's effectiveness. Few-shot learning enhanced precision (7.07) and resource mapping (6.90), while reducing hallucinations (4.71), but it also introduced a potential overreliance on examples. Temperature had a less significant impact; while higher temperatures slightly increased precision (6.19 vs. 5.97), they also correlated with a minor increase in hallucination risk (5.53 vs. 5.57). Llama3-Med42 demonstrated unsuitability for this task, prioritizing diagnostic suggestions over accurate FHIR mapping. Gemini generally outperformed Mistral across all metrics.  
The key contributions of this work include: (1) demonstrating the effectiveness of the two-step conversion approach; (2) highlighting the trade-offs inherent in few-shot learning; (3) evaluating the impact of temperature and LLM selection; (4) providing a comprehensive evaluation framework for FHIR conversion using LLMs; and (5) addressing the challenge of converting unstructured EHR data to FHIR.  
In conclusion, LLMs demonstrate significant potential for automating FHIR conversion, but careful prompt engineering, appropriate use of few-shot learning, and thoughtful parameter selection are essential for optimal results. The two-step conversion strategy offers a particularly promising path toward generating accurate and valid FHIR bundles, thereby advancing healthcare data interoperability.  
Conceptualization, J.D. and J.C.; Methodology, J.D. and D.G.; Software, J.D.; Validation, D.G.; Formal analysis, D.G.; Investigation, J.D. and J.C.; Writing--review & editing, J.D. and J.C.; Supervision, J.C.; Funding acquisition, J.C. All authors have read and agreed to the published version of the manuscript.  
J.C. work was supported by the Torres Quevedo grant \"Asistente Virtual para la mejora del sistema de salud--Virtual Assistant for Better HealthCare-VA4 BHC\" (ref. PTQ2021-012147).  
Not applicable.  
**Data Availability Statement:** The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding authors.  
**Conflicts of Interest:** The authors [PERSON], [PERSON] and [PERSON] were employed by the company Top Health Tech. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.