# Generalized Image Captioning for Multilingual Support
## 3 Filter Captioning Model Algorithm
### Image Caption Fine-Tuning
Compared to the noise-free general OSCAR [18] model of input data, we aimed to ensure that the evaluation metrics do not fall significantly. ### Filter Captioning Learning and Brotherhood Indicators

In this paper, BLEU [1], METEOR [2], ROUGE [3], and CIDEr [5] are used as evaluation methods of image captions generated by the model during fine-tuned learning. \[BLEU_{k}=min(1,\frac{predictsentence(wordlength)}{labelsentence(wordlength)})*( \prod_{i=1}^{K}precision_{i})^{\frac{1}{k}} \tag{3}\]

The Bilingual Evaluation Understudy Score (BLEU Score) is a measurement method created to evaluate language machine translation. This is a method of evaluating predictive sentences based on the precision regarding how many ordered pairs overlap based on n-gram. K stands for n in n-gram. \[METEOR=\frac{10 PR}{R+9P}(1-p),\qquad p=0.5(\frac{c}{u_{m}})^{2},\qquad P=\frac {m}{w_{t}},\qquad R=\frac{m}{W_{r}} \tag{4}\]

Metric for Evaluation with Explicit Ordering (METEOR) is a measurement method designed to evaluate language machine translation like BLEU. Unlike BLEU, recall is also considered, and predictive sentences are evaluated based on uni-gram based on aligned text. In Equation (4), \(P\) refers to precision, \(m\) is the number of uni-grams of the correct answer sentences found in the uni-gram of the model's predicted sentence, \(w\_t\) is the number of uni-grams in the predicted sentence, \(R\) is recall, and \(u\_m\) is the number of uni-grams in the correct sentence. Since precision and recall describe only word matching, we use \(p\) to calculate the penalty for sorting. Finally, \(c\) is the number of adjacent mapping of sentences

Figure 5: COCO captioning dataset: a chart comparing the frequency of word occurrence in image captions in training data. and correct answer sentences predicted by the number of chunks, and \(u\_m\) is the number of mapped uni-grams. \[ROUGE\_L=\frac{2 PR}{P+R},\qquad P=\frac{LCS(predict,label)}{n},\qquad P=\frac{LCS (predict,label)}{m} \tag{5}\]

Recall-Originated Understudy for Giving Evaluation (ROUGE) is a measurement method designed to evaluate machine translation language and text summaries. This is a method of evaluating the predictive sentence based on the recall of how much the ordered pairs overlap based on n-gram. This paper describes the bi-gram-based ROUGE_L evaluation method. The ROUGE_L evaluation method measures the longest matching string using the longest common subsequence (LCS) technique. In the above formula, \(m\) and \(n\) are the length of the correct answer sentence and the length of the predicted sentence, respectively, and _prediction_ and _label_ are the predicted and correct sentences, respectively. \[g_{k}(s_{ij})=\frac{h_{k}(s_{ij})}{\sum_{w_{ij}\in\Omega h_{l}(s_{ij})}}\log( \frac{|I|}{\sum_{t_{p}\in I}MIN(1,\sum_{q}h_{k}s_{pq})}) \tag{6}\]

\[CIDEr_{n}(c_{i},S_{i})=\frac{1}{m}\sum_{j}\frac{g^{n}(c_{i})\cdot g^{n}(s_{ij} )}{\|g^{n}(c_{i})\|\|g^{n}(s_{ij})\|} \tag{7}\]

Consensus-based Image Description Evaluation (CIDEr) is a measurement method designed to evaluate sentences made for image descriptions. It uses a method of determining whether one sentence is similar to another in generating an absolute indicator for evaluating a sentence. For the comparison of sentences, all words in the sentence are mapped with a word root and a word stem, where each sentence is set with n-gram. Since the n-gram may have little information, the frequency inverse document frequency (TF-IDF) [32] weight is determined in Equation (6) to lower the weight. Here, \(\Omega\) is an n-gram vocabulary and \(l\) is all image datasets. Additionally, \(w\_k\) is the TF-IDF weight calculated by referring to the number of each image set \(|l|\). Figure 2:0

Here, the weight for the n-gram of n is calculated in Equation (7) using cosine similarity, and the vector denominator \(c\_i\) of \(g^{n}\) corresponds to all \(g\_k\) of length \(n\) in size. Using this value, the CIDEr value is calculated in Equation (8), where \(w\_n\) is \(1/N\) divided by the weight, and \(N\) is 4.