# Detailed Image Captioning and Hashtag Generation
## 2 Background
### Image Captioning  
Image captioning, the task of automatically generating natural language descriptions for visual content, has significantly progressed due to deep learning techniques [1]. Early approaches utilized encoder-decoder architectures with convolutional neural networks (CNNs) encoding visual features and recurrent neural networks (RNNs) generating captions [2]. These models, however, often produced simplistic captions lacking detail. To address this limitation, attention mechanisms were introduced to focus on salient image regions when generating each word, improving caption quality [3]. The \"bottom-up and top-down\" method proposed by [PERSON] et al. [4] used Faster R-CNN to propose salient regions, implementing a \"hard\" attention mechanism. Graph-based methods also emerged, aiming to better capture the relationships between image elements by representing images as scene graphs [5]. The Scene Graph Auto-Encoder (SGAE) introduced by [PERSON] et al. [5] incorporated the inductive bias of language generation into the encoder-decoder framework. The introduction of Transformer architectures led to substantial improvements in image captioning. The Meshed-Memory Transformer [6] introduced a mesh-like connectivity between encoder and decoder layers, allowing for more nuanced caption generation. X-Linear Attention Networks [7] employ spatial and channel-wise bilinear attention to extract second-order interactions, facilitating more in-depth reasoning. Other Transformer-based models like Grid- and Region-based Image captioning Transformer (GRIT) [8] used both grid- and region-based features, while PureT [9] implemented a fully Transformer-based architecture for end-to-end training. Developments in vision-language pre-training have shown promising results. Contrastive Language-Image Pre-Training (CLIP) [10] has emerged as a powerful pre-trained model, providing a shared representation for both image and text. ClipCap [11] leverages CLIP embeddings to produce a prefix for each caption, which is then used by a pre-trained language model (GPT-2) to generate captions. Object-Semantics Aligned Pre-training (OSCAR) [12] uses object tags as anchor points to align image and language modalities in a shared semantic space. VinVL [13] and mPLUG [14] further refined these approaches, with mPLUG introducing novel cross-modal skip-connections to improve computational efficiency and address information asymmetry. Recent research has explored novel architectures and training paradigms. Expansion-Net v2 [15] introduces a Block Static Expansion layer to address potential performance bottlenecks related to input length. COS-Net [16] aims to unify semantic comprehension and ordering in the captioning process. VIVO (Visual VOCabulary pre-training) [17] learns a joint presentation of visual and text input using image-tag pairs for pre-training, allowing for zero-shot generalization to novel visual objects. Cross-modal Generative Pre-Training (XGPT) [18] uses a cross-modal encoder-decoder architecture that is directly optimized for generation tasks. Despite these advancements, current captioning systems face several limitations. A key issue is the tendency to produce short, generic descriptions rather than detailed analyses of image contents [1]. The problem of object hallucination, where models detect objects not present in the input image, persists [19]. Additionally, most advanced modelsrequire powerful hardware that is unavailable to average users, limiting their accessibility. Newer models like Florence-2 [20] aim to bridge this gap between speed and performance by training their model on high-quality data like FLD-5B, which consists of 5.4 billion comprehensive visual annotations on 126 million images. The field of image captioning continues to evolve, with researchers exploring various techniques to improve caption quality, efficiency, and applicability.