## 2 Materials and Methods
### BiomedCLIP Methodology  
Model Selection and Fine-Tuning  
In this study, the Biomed CLIP model was employed. This model is a fine-tuned version of the CLIP (Contrastive Language-Image Pretraining) model, specifically adapted for classifying X-ray images of children's spines into three scoliosis categories: mild, moderate, and severe. The CLIP model was selected due to its superior zero-shot classification performance and proven ability to interpret complex biomedical imagery, surpassing other multimodal models like DALL-E, MedCLIP, and PubMedCLIP. The open-source availability of the Biomed CLIP model further facilitated its use in this research. The model was fine-tuned using the PMC-15 dataset, which includes 15 million biomedical image-text pairs derived from 4.4 million scientific articles. This broad and diverse dataset helped adapt the model to the specific demands of pediatric spinal imagery and enhanced its ability to interpret medical images. Hyperparameters  
The fine-tuning of Biomed CLIP followed the same hyperparameters as those used in training the original CLIP ViT-B/16 model (Table 1). These include:Hardware and Software Configuration  
All computational tasks were carried out on two NVIDIA L40S GPUs, supported by 16 vCPUs and 124 GB of RAM, with 96 GB of VRAM on the RunPod platform. The software environment utilized a Docker image (pytorch:2.1.0-py3.10-cuda11.8.0-level-ubuntu22.04) containing the necessary CUDA and PyTorch modules to support model training and evaluation. The Biomed CLIP model is based on the Vision Transformer (ViT-B/16) architecture for its vision encoder, which divides input images into 16 \(\times\) 16-pixel patches. This architecture was chosen for its ability to capture long-range dependencies in medical images. Each patch is treated as a token and processed using a self-attention mechanism. The model supports images of 224 \(\times\) 224 pixels, though experiments with a higher resolution (\(384\times 384\)) suggested improved performance for certain tasks, albeit with increased training time. The ViT-B/16 model features approximately 86 million trainable parameters. To handle the specific demands of medical text, the text encoder was adjusted from the standard 77 tokens to a context size of 256 tokens, accommodating longer and more detailed biomedical descriptions. Training Setup  
Training was performed on large-scale hardware configurations using 16 NVIDIA A100 or V100 GPUs. To optimize memory usage and computational efficiency, techniques such as gradient checkpointing and automatic mixed precision were employed. The fine-tuning process was carried out for 32 epochs, with a batch size of 32,768, and the Adam optimizer was used with weight decay and a learning rate of 4 \(\times\) 10\({}^{-4}\). Data Handling and Evaluation  
The evaluation was performed using a database of anonymized X-ray images, enabling iterative assessments of the model's classification performance across various stages of scoliosis. For this task, descriptive text labels were prepared to represent the classification of each category:  
1. First Category: 1. \"This is an image of scoliosis\". 2. \"This is not an image scoliosis\". 2. Second Category: 1. \"This is an image of single curve scoliosis\" 2. \"This is an image of double curve scoliosis\" 3. Third Category: 1. \"This is an image of mild scoliosis\". \begin{table}
\begin{tabular}{l l} \hline
**Hyperparameter** & **Value** \\ \hline Learning Rate & 4 \(\times\) 10\({}^{-4}\) \\ Weight Decay & 0.2 \\ [PERSON] \(\beta\)1 & 0.9 \\ [PERSON] \(\beta\)2 & 0.98 \\ [PERSON] \(\varepsilon\) & 1.00 \(\times\) 10\({}^{-6}\) \\ Batch Size & 32,768 \\ Training Epochs & 32 \\ Warm-up Iterations & 2000 \\ Gradient Clipping & Max temperature of 100 \\ \hline \end{tabular}
\end{table} 
Table 1: Hyperparameters used for fine-tuning BioMed CLIP, following the same setup as the original CLIP ViT-B/16 model. * \"This is an image of moderate scoliosis\". * \"This is an image of severe scoliosis\".