# Multimodal AI and Large Language Models for Orthopantomography Radiology Report Generation and Q&A
## 1 Introduction
### Related Work  
A recent study by [PERSON] et al. [20] assesses the potential of AI-generated radiology reports in terms of summary quality, patient friendliness, and recommendations, highlighting the unique challenges and opportunities in evaluating report accuracy and utility. In this study, two radiologists independently performed qualitative and quantitative evaluations of the AI-generated reports, using the original reports as the reference standard. Following this, two non-physician raters independently assessed the AI-generated reports for content quality and patient-friendliness. The ChatGPT-3.5-turbo model was used to generate patient-friendly summaries and recommendations, with the original MRI reports serving as prompts. Their results indicate a 1.12% of artificial hallucinations and a 7.4% of potentially harmful translations, in the generated report. Furthermore, ref. [21] highlights the capabilities of LLMs in language generation and understanding, relevant to the healthcare domain. Their review encompasses major LLM architectures, including GPT, Llama, and Bloom, noting their scale, often comprising billions of parameters. The authors also analyze trends in medical datasets used for training these models, categorizing them by subject matter, source, and size. LLMs demonstrate the potential to enhance patient care, accelerate medical research, and improve healthcare efficiency. The paper also addresses the limitations of these models and their practical application in the medical field. [PERSON] et al. [22] deeply discussed the challenges that need to be addressed before LLMs can be safely and effectively used in healthcare, such as data privacy, the accuracy of the information, and potential bias. This work investigated the multilingual pre-trained language model mBERT using translated benchmarks named TREx and GoogleRE in 53 languages. The study explored mBERT's capability as a multilingual knowledge base, its language-dependent performance, and the impact of additional training text. Results indicated reasonable performance for 21 languages but instability across 42 languages. Ref. [23] discussed the instruction tuning of LLMs for biomedical natural language processing. The paper proposed BioInstruct, a novel method of instruction-tuning LLMs with natural language instructions and their corresponding inputs and outputs. The Section 3 further discussed self-instruction, low-rank adaptation (LoRA) [16], instruction tuning, and multi-task learning technologies that can be used in the fine-tuning process. The BioInstruct model has been able to demonstrate performance gains of 17.3% in question-answering, 5.7% in information extraction, and 96% in text generation tasks. Numerous studies have demonstrated the efficacy of vision-language models in medical imaging analysis. [PERSON] et al. proposed XrayGPT [24], a system that leverages medical vision-language models to summarize chest radiographs. Their work demonstrates the potential of vision-language models for understanding and summarizing medical images, specifically in the context of chest radiographs. By combining visual understanding with language processing, XrayGPT provides concise and informative summaries that can assist radiologists in interpreting complex chest radiographs. On the other hand, [PERSON] et al. introduced MedClip [25], a contrastive learning framework that integrates medical images and text to enhance clinical decision support. Their study showcases the effectiveness of vision-language models in capturing meaningful medical information from unpaired image-text data. MedClip offers valuable insights for improving diagnostic accuracy and aiding in treatment planning, demonstrating the potential of vision-language models in clinical decision-making. [PERSON] et al. explored the benefits of CLIP (contrastive language-image pre-training) in the medical domain, specifically in visual question-answering (VQA). Their study, PubMedCLIP, investigates how CLIP can enhance VQA tasks by utilizing medical domain-specific data from PubMed [26].