Comparative Analysis of Pretrained Text to Image Models for Accurate Radiological Image Generation for a Single Text Prompt (Preprint)
Authors: Mookherjee Shashwat , Dutta Shruti , Maschatak Debargha , Chakraborty Sristi , Sinha Adrija 
Published: July 2023

BACKGROUND

Generative AI is a rapidly advancing field within Artificial Intelligence with wide-ranging applications. In the medical science domain, machine learning and deep learning methods have already found extensive use. This study aims to conduct a comparative analysis of seven freely available pretrained text-to-image models to generate radiological images based on a single text prompt. The primary objective is to determine which among these models produces the most accurate radiological images.

The research investigates the effectiveness of generative AI in the medical domain, particularly for generating radiological images. Several text-to-image models are tested, building on previous research that explored DALL-E 2's capabilities in understanding radiological images. By comparing the performance of different models on a single text prompt, the study provides valuable insights into their potential use in medical image generation.

Through this investigation, the study seeks to benefit medical professionals, researchers, and the wider AI community. Identifying the most accurate text-to-image model could enhance medical imaging applications, leading to improved diagnostics and treatment planning.

OBJECTIVE

The objective of this article is to conduct a comparative study of seven existing pretrained text-to-image models available freely on the internet, with the specific aim of generating radiological images based on a single text prompt. The primary goal is to determine which of these text-to-image models is capable of generating the most accurate radiological images for medical applications.

By evaluating the performance of various text-to-image models, the research aims to provide insights into the effectiveness of generative AI techniques in the medical domain. Specifically, the study seeks to identify the model that demonstrates superior capabilities in accurately translating textual descriptions into radiological images.

The article seeks to contribute to the field of Generative AI, particularly in the context of medical science and radiological imaging. By comparing different models' outcomes on a single prompt, the research aims to offer valuable information for medical professionals and researchers, highlighting the potential applications and limitations of these text-to-image models in medical image generation.

Ultimately, the objective of the article is to facilitate advancements in medical imaging technologies, leading to improved diagnostics and medical decision-making processes. Through its comparative analysis, the article endeavors to aid the AI community in selecting the most suitable text-to-image model for generating accurate and reliable radiological images.

METHODS

Model Selection: Seven existing pretrained text-to-image generative models available freely on the internet were chosen for the study. These models were specifically designed for generating images from textual descriptions.

Model Descriptions: A brief description of each text-to-image model used in the study, along with their respective results, was provided to contextualize the findings.

Image Generation: The prompt used for testing all models was: "Photorealistic MRI scan of human lungs suffering from pneumonia."

Comparative Analysis: The results obtained from each model were compared and analyzed to determine which model produced the most accurate radiological image in response to the given prompt. Keeping in mind the actual imaging of the medical condition, we consulted a physiology expert to compare with the images generated by the seven different models.

RESULTS

The following comparison results were obtained after the consultation and are presented as follows:

Dall-E 2 created the most realistic image of the lungs of a person suffering from pneumonia. It also shows the thoracic cavity with the heart in it which gives more accuracy to the image that is created. Dall-E 2 could also successfully show the difference between the left and right lung. It showed the septum which most of the models could not.
Midjourney did a good job at showing the infection even though it failed to create the image as realistically as Dall-E 2. Midjourney did provide a clear image though. It might accurately show the spread of the infection as well.
Min-dalle highlighted the infectious parts well, but it failed to give a more realistic image.
Carefree Creator did well with the image of the thoracic cavity but it is not very reliable for the detection of infections.
Big Sleep is a model which we are unsure of. If the white parts in between show the mucus congestion, then it did a nice job at showing the congestion of the lungs but did a poor job at showing the thoracic cavity.
Aphantasia used bright colours which might help the detection of infections even though it failed to show the lungs and the infection accurately.
Deep Daze produced a very complicated image which makes identifying parts of the body and the infection very difficult.

CONCLUSIONS

From the results above, we can conclude that the existing text-to-image generation models are not capable of generating radiological images with 100% accuracy. However, it must be mentioned that some of the models performed better than the others in specific cases. For eg, the image generated by DALL-E 2 was able to show the difference between the left and right lungs properly and also was able to show the thoracic cavity as compared to the image generated by Aphantasia which was able to showcase the detection of infections better than DALL-E 2 even though it failed to show the lungs accurately.
This study also indicates the importance for the need of better visualisation of medical conditions in existing radiological methods. For example the use of colours to better showcase the detection of infections as shown by the image generated by Aphantasia. Of course there are many other factors which must be considered while designing a visualisation method and we aren’t suggesting any particular method which needs to be implemented immediately. Proper consultation with an expert is always the first step.
These results surely are a starting step in the domain of image generation for radiological images.It is true that in this study we used only one prompt. Further steps would include giving a better text prompt of the medial condition and giving prompts of more varied medical conditions.
There are a variety of applications and benefits of generating radiological images. Many Machine Learning tasks like classification and segmentation require a large dataset for training respective models appropriately and an accurate radiological image generated using AI would help in making the dataset of the required size. Further developments in this field can lead to generating radiological images of specific conditions based on the particular prompt of the user.