# Detailed Image Captioning and Hashtag Generation
## 4 Experiments and Results
### Evaluation Metrics  
Image captioning research employs various metrics to assess the quality of generated captions. The most commonly used metrics include BLEU [42], ROUGE [43], METEOR [44], CIDEr [45], and SPICE [46]. BLEU, a pioneer in evaluating machine-generated texts, compares n-grams in the generated caption with reference captions. It is widely used due to its simplicity, language independence, and comparability with human judgment [42]. ROUGE, originally designed for text summarization, measures the overlap of n-grams, word sequences, and word pairs between generated and reference captions [43]. METEOR attempts to address some limitations of BLEU by incorporating stemming and synonymy matching, aiming to correlate better at the sentence or segment level [44]. CIDEr, which is designed specifically for image captioning, uses term frequency-inverse document frequency (TF-IDF) weighting to encode how often n-grams in the candidate sentence are present in the reference sentences [45]. SPICE employs scene graphs to evaluate semantic propositional content, measuring how well objects, attributes, and the relations between them are covered in image captions [46].  
These metrics, while useful for comparative evaluation, have several limitations when assessing detailed captions. Key issues include their lack of semantic understanding, bias towards common phrases, inability to assess factual accuracy, limited evaluation of descriptive richness, and difficulty in assessing coherence and fluency [1]. Most metrics focus on lexical similarity rather than semantic meaning, potentially missing the nuances of detailed descriptions and frequently favoring occurring phrases in the training data.  
The limitations of these metrics highlight the need for more sophisticated evaluation methods that can better assess the quality, accuracy, and richness of detailed image captions. As the field of image captioning continues to advance, with models capable of generating increasingly detailed and nuanced descriptions, the development of more robust evaluation metrics becomes crucial [1]. Future research could focus on developing metrics that incorporate deeper semantic understanding and can better evaluate the nuanced aspects of detailed descriptions, potentially bridging the gap between current evaluation methods and the complexities of human-like image understanding and description. One attempt at solving these limitations was proposed by [PERSON] et al. [40] and is called CAPTURE (CAPtion evaluation by exTracing and coUpling coRE information), which is specially designed for detailed caption evaluation and works by extracting visual elements such as objects, attributes, and relations for better feature representation of the image.  
CAPTURE calculates precision and recall for each type of visual element, combines these into F1 scores, and produces a final weighted score:  
\[CAPTUREScore=\frac{5\cdot F1_{obj}+5\cdot F1_{attr}+2\cdot F1_{rel}}{5+5+2}, \tag{3}\]  
where \(F1_{obj}\), \(F1_{attr}\), and \(F1_{rel}\) are the F1 scores for objects, attributes, and relations, respectively.  
For the hashtag generation model, performance was evaluated using precision, recall, and F1-score. These metrics assess the relevance and accuracy of the generated hashtags.  
* Precision: Measures the proportion of generated hashtags that are relevant.
* Recall: Measures the proportion of relevant hashtags that were successfully generated.
* F1-score: The harmonic mean of precision and recall, providing a balanced measure of the model's performance.  
Additionally, the average processing time for each method was measured to assess their suitability for real-time applications.