# Storytelling with Image Data: A Systematic Review and Comparative Analysis of Methods and Tools
## 8 Task 6: Quality Assessment and Dataset
### Automatic Assessment
Using a dependency tree structure, the semantic image graph represents the objects' properties and relationships [131]. The SPICE value is thereafter defined as the F1-score depending on the correspondence between the candidates and reference description tuples [132]. **Consensus-Based Image Description Evaluation (CIDEr). ** CIDEr [128] is intended as a specialized measure for evaluating picture captioning, although it operates in a cognitive,technical manner and only expands current metrics by weighting tf-idf across n-grams [132]. It assesses the agreement between an anticipated sentence and the image's reference phrases. It executes stemming and transforms all candidate and reference phrases' words into the root types [131]. Each phrase is represented by a collection of n-grams comprising one to four words via CIDEr [131]. To encode the agreement between the anticipated phrase and the reference phrase, the co-occurrence rate of n-grams in both phrases is measured [131]. Therefore, for the purpose of computing this measure, initial stemming is performed, and each phrase is represented by a set of 1 to 4 grams [132]. As a summary, we have discussed various metrics that are commonly used to evaluate the quality of text generated by machine translation and image captioning systems. The metrics discussed include BLEU, METEOR, ROUGE, SPICE, and CIDEr. BLEU is a precision-based measure that calculates the overlap between the generated text and a reference text using n-grams [133]. METEOR, on the other hand, is also a measure for machine translation; it was designed to overcome the limitations of BLEU by using semantic matches instead of requiring a precise lexical matches [131]. ROUGE is a metric developed for evaluating summarization programs; it is performed by comparing overlapping n-grams, word sequences, and word pairs [132]. SPICE is a measure that is recently developed for measuring the similarity of picture captions [132]. Lastly, CIDEr is a specialized measure intended for evaluating image captioning; it operates in a cognitive and technical manner [132]. In general, each metric has its own strengths and weaknesses, and it is more appropriate to use one depending on the specific task and its requirements. In addition to the objective metrics (described above) that several storytelling studies appear to employ in their evaluations, subjective metrics such as fidelity and coherence are also necessary. For a work involving creative generation, such as story generation, there are no valid automatic evaluation criteria for assessing qualities such as interestingness and coherence [129]. There are four primary subjective metrics: fidelity (if the story corresponds to the title), coherence (if the story is logically valid and cohesive), interestingness (if the story is engaging), and overall user satisfaction (how readers enjoy the story) [129]. Then, a cohesive narrative should arrange its sentences in the proper sequential sequence and maintain the same theme between adjacent sentences [117]. In addition, one technique to determine the relevance between an image and its produced explanation is to match the things specified in the explanation to the image's bounding boxes [117].