# A Systematic Literature Review on Image Captioning
## 2 SLR Methodology
### Search Query  
To become acquainted with the \"image caption generation\" topic we first conducted a quick review of articles under it. We obtained an idea of the technologies and models, which are popular under this topic so that our research would be relevant and correct. Moreover, we did not narrow the search query to small details in order to get enough results and an appropriate number of articles from the search--keywords are presented identically to how they were submitted for the search query. The query questions together with the number of articles found in each library are presented below in Tables 1-4. Libraries were read in the order as they are listed in the table--first the query was searched in ArXiv, then in IEEE Xplore, and finally in WOS. If the article found had already been previewed from the previous library or from a previous query, it was not added to the total number of relevant articles but identified in brackets.  
dataset or evaluation metric was used in the article, a short note was provided. i.e., Ref. [10] used the Lifelog Dataset for model training and evaluation, Ref. [20]--the Visual Genome Dataset, Ref. [39] evaluated results using F-1 score metrics, Ref. [47]--R metrics. To present the results to be easier understood, we first presented five articles from each one which achieved the most results--Tables 5-8. There were only three articles from 2016 which were evaluated on MS COCO, so only those were presented. Other tables have five articles with the top five results on MS COCO based on the highest CIDEr metric results.  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Image Encoder**} & \multicolumn{4}{c}{**Image Decoder**} & \multicolumn{4}{c}{**Method**} & \multicolumn{4}{c}{**MS COCO**} & \multicolumn{4}{c}{**Flickr30k**} \\ \hline  & & & & & & & & & & & & & & & & & & & & & \\
2017 [34] & & & x & & & & & & & & & & & & & & & & & \\
2017 [33] & & & x & & & & & & & & & & & & & & & & & \\
2017 [32] & x & & & x & & & & & & & & & & & & & & & \\
2017 [31] & x & & & x & & & & & & & & & & & & & & & \\
2017 [30] & x & & x & & & & & & & & & & & & & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 6: Top 5 results in 2017 (sorted by CIDEr result on MSCOCO).  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Image Encoder**} & \multicolumn{4}{c}{**Image Decoder**} & \multicolumn{4}{c}{**Method**} & \multicolumn{4}{c}{**MS COCO**} & \multicolumn{4}{c}{**Flickr30k**} \\ \hline  & & & & & & & & & & & & & & & & & & & & & \\
2016 [13] & & x & & & & & x & & & & & & & & & & & & & \\
2016 [12] & & & x & & & & & x & & & & & & & & & & & & & \\
2016 [11] & x & x & & & & & x & & & & & & & & & & & & & & \\
2017 [31] & x & & & & & & & & & & & & & & & & & & & & \\
2017 [30] & x & & & & & & & & & & & & & & & & & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 5: Top 3 results in 2016 (sorted by CIDEr result on MSCOCO).  
Figure 4: Bleu-4 results by year on MSCOCO (**left**) and Flickr30k (**right**).  
Figure 5: CIDEr results by year on MSCOCO (**left**) and Flickr30k (**right**).  
Figure 3: Bleu-3 results by year on MSCOCO (**left**) and Flickr30k (**right**).  
Figure 6: Meteor results by year on MSCOCO (**left**) and Flickr30k (**right**).  
Tables 9 and 10 present results based on different techniques used in image captioning--which combinations of encoder and decoder were used every year and which methods were the most popular. Those tables help to understand which techniques work best together, and which combinations have probably not been successful or have not been explored at all up to now.