## 3 Vision Capacity of GPT-4o
### Image Classification: Glaucoma Detection  
The early detection of glaucoma is critical for preventing vision loss and ensuring timely treatment. The glaucoma detection dataset used for this evaluation consisted of retinal fundus images from the ACRIMA database ([[https://www.kaggle.com/datasets/chetangediredla/glaucoma-dataset](https://www.kaggle.com/datasets/chetangediredla/glaucoma-dataset)]([https://www.kaggle.com/datasets/chetangediredla/glaucoma-dataset](https://www.kaggle.com/datasets/chetangediredla/glaucoma-dataset)), accessed on 15 June 2024). A subset of 100 images was sampled, evenly split between glaucomatous and normal cases. These images were collected at FISABIO Oftalmologia Medica in Valencia, Spain, and were annotated by experienced glaucoma experts. GPT-4o was tasked with classifying each image into either glaucoma or normal based solely on the visual information provided. The model's predictions were compared against the expert-annotated ground truth labels to assess its performance in detecting glaucoma from retinal fundus imagery.  
Figure 10: Confusion matrix (**top**) and performance comparison (**bottom**) for crop disease classification.  
As shown in Table 11, GPT-4o achieved an average **precision of 0.65**, an average **recall of 0.62**, and an average **F1-score of 0.59**. For the glaucoma class, the model demonstrated a precision of 0.58, a recall of 0.86, and an F1-score of 0.69. In contrast, the normal class had a higher precision of 0.73 but a significantly lower recall of 0.38, resulting in an F1-score of 0.50.  
The confusion matrix in Figure 11 reveals that the model correctly identified 42 out of 49 glaucoma cases but struggled more with normal cases, correctly classifying only 19 out of 50. The plot shows the model's relatively balanced precision and recall for the glaucoma class but highlights a pronounced discrepancy for the normal class, with precision substantially higher than recall. GPT-4o is effective at identifying glaucomatous images but has difficulty in correctly classifying normal cases.  
Few-shot learning allows models to make accurate predictions with only a small number of training examples. This approach is particularly beneficial in scenarios where data are scarce. Figure 12 illustrates the F1 scores for both classes across different numbers of shots, indicating how the model's performance evolves with the number of examples provided during training. The glaucoma class maintains a relatively high F1 score across all shot levels, showing slight improvement with additional examples. This consistency suggests that GPT-4o effectively learns to identify glaucomatous features even with a limited number of examples. In contrast, the normal class exhibits significant improvement in the F1-score from zero shots to one shot but then plateaus. This indicates that while the initial provision of examples significantly enhances the model's ability to recognize normal cases, further increases in the number of examples yield diminishing returns.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Class** & **Precision** & **Recall** & **F1-Score** \\ \hline Glaucoma & 0.58 & 0.86 & 0.69 \\ Normal & 0.73 & 0.38 & 0.50 \\ \hline \hline \end{tabular}
\end{table}
Table 11: GPT-4o performance on glaucoma detection.  
Figure 11: Confusion matrix (**left**) and performance comparison (**right**) for glaucoma detection.