# A Unified Visual and Linguistic Semantics Method for Enhanced Image Captioning
## 5 Experiments
### Ablation Studies  
To validate the specific strengths of our 'Ours; MLP+GPT2 tuning' model in precise language generation and semantic alignment, we conduct an ablation study delineated in Table 4. This experiment is designed to highlight the effectiveness of each component in our model by comparing it with other configurations and models. Detailed Analysis of Model Performance: The 'Ours; MLP+GPT2 tuning' model demonstrates exceptional performance, as evidenced by its leading scores in the majority of the metrics. It surpasses other models with the highest BLEU@1 score of 79.7%, indicating its superior capability in capturing frequently used words. This trend continues with the highest BLEU@2, BLEU@3, and BLEU@4 scores (68.9%, 59.1%, and 50.7%, respectively), underscoring its proficiency in constructing longer, more complex linguistic structures. Additionally, the model achieves the top ROUGE score (66.2%), reflecting its effectiveness in summarizing the essential content of images. However, it slightly falls behind in the METEOR score (40.1%) and CIDEr score (164.2%) compared to the 'Ours; Transformer' variant, which could be attributed to the Transformer's slightly better grasp of syntactic variation and distinctiveness in caption generation. Interpreting Strengths and Addressing Limitations: Despite the minor shortfall in METEOR and CIDEr, the 'Ours; MLP+GPT2 tuning' model's overall performance indicates a robust balance between linguistic fluency, n-gram coherence, and content relevance. The slight underperformance in METEOR and CIDEr may be due to the model's focus on grammatical and syntactical structure, which, while leading to high BLEU and ROUGE scores, might not capture the more nuanced semantic aspects as effectively as the 'Ours; \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **BLEU@1** & **BLEU@2** & **BLEU@3** & **BLEU@4** & **METEOR** & **ROUGE** & **CIDEr** \\ \hline CLIPCAP & 76.7 & 65.1 & 55.0 & 46.4 & 38.8 & 64.0 & 151.2 \\ \hline SegmentCLIP & 74.2 & 63.4 & 53.8 & 45.2 & 37.6 & 63.7 & 150.4 \\ \hline Ours; Transformer & 79.2 & 68.4 & 58.6 & 50.1 & **40.3** & 66.1 & **164.9** \\ \hline Ours; MLP+GPT2 tuning & **79.7** & **68.9** & **59.1** & **50.7** & 40.1 & **66.2** & 164.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: UVLS model validity experiment. Top scores for each metric are bold. All values are reported as percentage (%). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Params (M)** & **Training Time (h)** \\ \hline CS-NIC & 31 & 40 (GTX3090 (NVIDIA, Santa Clara, CA, USA)) \\ \hline CIC & 42 & 56 (GTX3090) \\ \hline SC & 64 & 80 (GTX3090) \\ \hline Oscar & 135 & 200 (GTX3090) \\ \hline CLIPCAP & 156 & 20 (GTX3090) \\ \hline Ours; MLP+GPT2 tuning & 175 & 24 (GTX3090) \\ \hline Ours; Transformer & 63 & 23 (GTX3090) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Params and training times of different models on ICC dataset. Transformer' variant. Nevertheless, the model's strong performance across most metrics, particularly in BLEU and ROUGE, highlights its capability to generate contextually rich and detailed image captions, suggesting its usefulness as a method in the realm of image captioning. In our empirical investigation of prefix length's influence on model performance for image captioning, we present an in-depth analysis through Figure 3, which comprises Figure 3a,b, corresponding to two different model configurations: MLP and Transformer. These graphs illustrate the effects of prefix length variation on the captioning performance across the AIC-ICC dataset. Figure 3a delineates the results from the MLP-based model configuration. It is evident that the BLEU@4 and CIDEr scores, which represent the model's linguistic accuracy and the salience of the generated captions, respectively, exhibit a substantial increase as the prefix length progresses from 2.5 to around 10.