# NuCap: A Numerically Aware Captioning Framework for Improved Numerical Reasoning
## 1 Introduction  
Image captioning is a core task in vision-language research that involves generating natural language descriptions for a given image. It plays a crucial role in multi-modal learning, as it requires the integration of visual understanding and linguistic expression. A widely adopted framework is the encoder-decoder architecture, in which an image encoder extracts visual features and a language decoder produces the corresponding caption [1]. Initially, convolutional neural networks (CNNs) [2] were commonly used as feature extractors, but the introduction of attention mechanisms [3] led to a shift toward self-attention-based models [4, 5, 6, 7, 8]. More recently, large-scale vision-language models (VLMs) have demonstrated strong performance by leveraging extensive pretraining on diverse datasets. However, despite these advances, current captioning models struggle to accurately represent numerical information, particularly object counts. Captions often omit count information or resort to vague expressions such as \"a group of\" or \"several\", even when the number of objects is visually clear. For example, rather than stating \"Five planes flyingin the sky\", many models generate the less informative caption \"A group of planes flying in the sky\". This limitation becomes more pronounced as the number of objects increases, reducing model effectiveness in applications that require numerical precision. This issue is illustrated in Figure 1, which compares caption outputs from existing models and NuCap on images with countable objects. This issue stems primarily from the limitations of existing training datasets, which lack emphasis on numerical reasoning. Models trained on datasets such as COCO [9] or Visual Genome [10] learn to describe scenes generally but fail to incorporate explicit count information. As a result, they rely on ambiguous quantifiers or plural nouns, which may suffice in casual contexts but fall short in domains such as scientific analysis, data annotation, and surveillance systems. Although VLMs show relatively improved numerical awareness due to large-scale pretraining, their computational demands hinder deployment in real-time or resource-constrained environments. To address these limitations, we propose NuCap--a numerically aware captioning framework that enhances object counting in caption generation. NuCap integrates both local and global visual representations through two key modules: a Region-Attended Object Encoder (RAOE) and a Spatially Attended Grid Encoder (SAGE). We further introduce a Number-Focused Cross-Entropy (NFCE) loss, which emphasizes numeric tokens during training to strengthen numerical reasoning. To support fine-tuning, we construct CountCOCO, a numerically enriched version of the COCO dataset that provides structured captions containing explicit count values. This setup allows the model to learn both semantic fluency and numerical grounding. The overall training process of NuCap, including both pretraining and fine-tuning, is summarized in Figure 2.  
Figure 1: Comparison of captions generated by different models on images containing countable objects. Vague quantifiers such as “a group of” or “several” are marked in red, while correct numerical expressions are highlighted in yellow. NuCap produces captions with precise numerical values, demonstrating improved numerical reasoning in comparison to existing models. Figure 2: Overview of the training process of NuCap. The model is first pretrained on the COCO2014 dataset using NFCE loss, and subsequently fine-tuned on CountCOCO with structured captions to enhance its numerical reasoning capabilities. Our key contributions are summarized as follows:  
* We propose NuCap, a unified captioning framework for enhancing numerical reasoning. NuCap integrates SAGE, the NFCE loss function, and a count-enriched dataset (CountCOCO) to improve the model's ability to represent and reason about quantities in image captions. * We design a Structured Caption Format that explicitly encourages early number generation.