# Multimodal AI and Large Language Models for Orthopantomography Radiology Report Generation and Q&A
## 1 Introduction  
The convergence of multimodal artificial intelligence (AI) and large language models (LLMs) is poised to revolutionize healthcare [1]. LLMs, with their capacity to process and generate human-like text, offer exciting possibilities for enhancing clinical documentation, patient communication, and decision support [2]. While the application of these technologies in dentistry is still in its nascent stages, the potential is significant, particularly for tasks requiring the integration of visual and textual information [3]. Orthopantomography,a widely used panoramic dental imaging technique, plays a crucial role in dental and maxillofacial diagnostics [4]. These images offer a comprehensive visualization of the teeth, jaws, and adjacent anatomical structures, enabling the detection of a wide range of conditions, including dental caries, impacted teeth, bone fractures, cysts, and tumors. While orthopantomograms provide invaluable diagnostic information, their interpretation requires specialized training and expertise [4]. The increasing volume of dental imaging data in clinical practice presents a significant challenge, potentially impacting the timeliness and accuracy of interpretations by radiologists and other dental professionals. This necessitates the exploration of automated solutions to assist in the efficient and accurate analysis of these critical diagnostic images. Automated analysis has the potential to improve diagnostic accuracy, reduce clinician workload, and ultimately enhance patient care.  
In dentistry, the automatic generation of radiology reports from orthopantomograms and the ability to respond to patient inquiries regarding radiographic data and general dental health present a compelling use case for vision-language models and LLMs. A system that integrates visual analysis with natural language generation could streamline report creation, reduce diagnostic delays, and support practitioners in detecting abnormalities with greater accuracy. Additionally, such a system could serve as an intelligent virtual assistant, enhancing clinical decision-making and improving patient education. It is worthwhile to mention that numerous studies have been conducted on applying convolutional neural network (CNN)-based AI architectures for the dental healthcare sector, such as segmentation of mental foramen in dental panoramic tomography (DPT) [5] and CNN-based multiclass image classifications [6]. However, LLM-based radiology report generation represents a fundamentally different approach. Unlike traditional classification [7] and segmentation [8; 9] tasks, which focus on categorizing images into predefined classes, LLM-based systems aim to synthesize visual and textual data to generate comprehensive, human-like reports.  
Multimodal techniques are being used more and more in current AI research for medical diagnosis systems as they are capable of processing, integrating, and reasoning across multiple types of data inputs or modalities simultaneously [1; 10]. These modalities may include text, images, audio, video, and structured data, each representing information in fundamentally different ways. Unlike single-modality AI models, multimodal AI has the ability to capture complementary information across these diverse data types and their complex interrelationships. Through this integration, patient outcomes are optimized, individualized treatment is made possible, and diagnostic accuracy is increased. In the research presented, this strategy is demonstrated by the combination of medical imaging with clinical findings, which improves the accuracy of the radiology report generation.