# Multimodal AI and Large Language Models for Orthopantomography Radiology Report Generation and Q&A
## 1 Introduction
### Contributions  
It should be noted that both vision language models and LLMs have the potential to play a substantial role in dental healthcare. Vision language models contribute to interpreting diagnostic images for radiology report generation in precise treatment planning, while LLMs improve communication between healthcare providers and patients, offering educational resources, and enhancing information exchange. As such, combining these models can significantly improve the quality of care, making dental healthcare more accessible, efficient, and effective. The proposed study investigates the potential of large language models (LLMs) in dentistry, focusing on tasks that require the integration of visual and textual data. Specifically, we explore two key applications: automated radiology report generation from DPT, i.e., orthopantomography images, and a question-answering (Q&A) system capable of interpreting radiographic findings and addressing general dental healthcare inquiries. A significant novelty of the proposed Q&A system is its operation in the Sinhala language. This research aims to evaluate the feasibility and effectiveness of LLMs in these critical areas, thereby facilitating their broader adoption in dental practice.  
The main contributions of this research are as follows:  
1. Radiology report generation system: A dental radiology report refers to a document generated by a radiologist or dentist after reviewing DPT images. These reports include key information such as findings, impressions, and recommendations, aiding in the diagnosis and monitoring of various dental health conditions. Currently, dental professionals conduct manual reviews of DPT images for radiology report generation. This research introduces an automated radiology report generation methodology aimed at improving diagnostic efficiency for dental professionals.
2. Sinhala language question-answering platform: The second component of this research addresses the critical need for clear and accurate communication between healthcare providers and patients, essential for effective diagnosis and treatment. The World Health Organization (WHO) has reported significant challenges in accessing adequate healthcare information, particularly in developing countries, often due to language barriers, high costs, and limited availability of dental services [30]. Facilitating access to professional medical opinions in local languages is therefore crucial. To address this, we present a robust end-to-end system for dental radiology report generation and a Sinhala language question-answering system focused on the dental domain.
3. Introduction of four new datasets: This research introduces several novel datasets to support future research in this area. Namely, the DPT Image and Caption Dataset [31], The Radiology Reports Dataset [32], the Sinhala Language Corpus Dataset [33], and the Sinhala Language Question-Answering Dataset for the Dental Domain [34].
4. Utilization of multimodal AI: In our work, we utilized dental panoramic tomograms (DPTs) and generated radiology reports to assess patient health. We then combined these findings with past patient queries to formulate answers and to address user/patient inquiries effectively. In summary, we developed a system that uses multimodal AI to generate clinical findings from images, integrating medical imaging and patient history to conduct meaningful Q&A sessions.