# Detailed Image Captioning and Hashtag Generation
## 4 Experiments and Results
### Results  
#### 4.4.1 Image Captioning Performance  
The performance of various configurations of the image captioning model, which were fine tuned using different LoRA adapters, was evaluated and compared to the base Florence-2 model. The results, presented in Table 1, demonstrate the effectiveness of theLoRA adaptation technique in improving the performance of the Florence-2 model for detailed image captioning.  
The DOCCI adapter showed the most significant improvements across all metrics, with particularly notable gains in the CAPTURE score, which is specifically designed to evaluate detailed image captions. The DOCCI adapter improved the CAPTURE score from 0.5458 (base model) to 0.5757--a 5.48% increase. This improvement suggests that the DOCCI dataset, with its focus on detailed and contrasting image descriptions, provides valuable training data for generating more nuanced captions. The Pixelprose and Recap DataComp adapters also showed improvements over the base model, but to a lesser extent than the DOCCI adapter.  
To further assess the performance of the proposed model, it was compared to other state-of-the-art vision-language models (VLMs). The results of this comparison are presented in Table 2.  
These results demonstrate that, despite having significantly fewer parameters (232 million) compared to models like Llava-v1.6-mistral-7b-hf (7 billion) and Phi-3-vision-128k-instruct (4.2 billion), our model achieves competitive performance, particularly in the CAPTURE metric. The proposed model demonstrates remarkable efficiency, with an average processing time of 0.688 s, which is second only to the base Florence-2 model. This is significantly faster than larger models like Llava-v1.6-mistral-7b-hf (17.983 s) and Phi-3-vision-128k-instruct (7.358 s).  
Figure 2 provides a visual comparison of the processing times for different models. Both the base Florence-2 model and our adapted model demonstrated remarkably quick processing times below one second. The PaliGemma model, while quick on average, shows some inconsistency with occasional processing times reaching 3 s. The Phi-3-Vision model performed decently in terms of speed but lagged behind our model. The Llava-mistral model, being the largest, was significantly slower and would require more powerful hardware for deployment.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model Configuration** & **METEOR** & **BLEU** & **ROUGE-L** & **CIDEr** & **CAPTURE** \\ \hline Base Florence 2 Model & 0.2128 & 0.1100 & 0.2753 & 0.0312 & 0.5458 \\ DOCCIadapter & **0.2671** & **0.1850** & 0.2874 & **0.0863** & **0.5757** \\ Pixelprose Adapter & 0.2501 & 0.1552 & **0.2982** & 0.0388 & 0.5554 \\ Recap DataComp Adapter & 0.2397 & 0.1501 & 0.2942 & 0.0348 & 0.5530 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance metrics for various configurations of the image captioning model.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **METEOR** & **BLEU** & **ROUGE** & **CIDEr** & **CAPTURE** & **AvgTime(s)** \\ \hline Paligemma-3b-mix-448 & 0.1242 & 0.0530 & 0.1947 & 0.0157 & 0.4038 & 1.201 \\ Llava-v1.6-mistral-7b-hf & **0.3346** & **0.2754** & **0.3231** & 0.0883 & 0.5674 & 17.983 \\ Phi-3-vision-128k-instruct & 0.3144 & 0.2359 & 0.3230 & **0.0955** & 0.5509 & 7.358 \\ Florence-2-base-ft & 0.2128 & 0.1100 & 0.2753 & 0.0312 & 0.5458 & **0.523** \\ Our Model & 0.2671 & 0.1850 & 0.2874 & 0.0863 & **0.5757** & 0.688 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance comparison with other vision-language models.