# Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
## 6 Multimodal Benchmark
### Text-to-Image Generation
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Task and Dataset** & **Model** & **Year** & \multicolumn{4}{c}{**Metric**} \\ \hline \multicolumn{5}{l}{**Image Captioning**} & \multicolumn{4}{c}{**BLEU-4 [194]**} & **METEOR [195]**} & **CIDEr [196]**} & **SPICE [197]** \\ \hline COCO Captions [189] & OFA [199] & 2022 & 43.5 & 31.9 & 149.6 & 26.1 \\  & SimVL [201] & 2021 & 40.6 & 33.4 & 143.3 & 25.4 \\  & Oscar [200] & 2020 & 41.7 & 30.6 & 140.0 & 24.5 \\ COCO [104] & M2 Transformer [213] & 2019 & 39.1 & 29.2 & 131.2 & 22.6 \\ Flickr30k [188] & Unified VLP [198] & 2019 & 30.1 & 23.0 & 67.4 & 17.0 \\ npcaps [193] & LEMON-large [214] & 2021 & 34.7 & 31.3 & 114.3 & 14.9 \\  & SimVLM-huge [201] & 2021 & 32.2 & 30.6 & 110.3 & 14.5 \\ TextCaps [191] & LSTM-R [215] & 2012 & 22.9 & 21.3 & 100.8 & 13.8 \\ VizWiz [192] & CASIA-IVA & 2020 & 28.3 & 22.1 & 79.1 & 17.9 \\ Local Narratives [190] & LoopCAG [216] & 2021 & 27.0 & 26.0 & 114.0 & - \\ SciCap [217] & CNN+LSTM [217] & 2021 & 21.9 & - & - & - \\ \hline \multicolumn{5}{l}{**Visual Question**} & \multicolumn{4}{c}{Overall} & \multicolumn{1}{c}{Y/N} & \multicolumn{1}{c}{Number} & \multicolumn{1}{c}{Other} \\ \multicolumn{5}{l}{**Answering**} & \multicolumn{4}{c}{Overall} & \multicolumn{1}{c}{Y/N} & \multicolumn{1}{c}{Number} & \multicolumn{1}{c}{Other} \\ \hline VQA v1 [119] & AAAA [203] & 2017 & 64.5 & 82.2 & 39.1 & 55.2 \\  & DAN [202] & 2016 & 64.3 & 83.0 & 39.1 & 53.9 \\ VQA v2 [125] & VLMo [218] & 2021 & 81.3 & 94.7 & 67.3 & 72.9 \\  & OFA & 2022 & 80.5 & 92.9 & 67.0 & 72.7 \\ VQA-CP [140] & CSS [219] & 2020 & 58.9 & 84.4 & 49.4 & 48.2 \\ VQA-CE [220] & RandImg [220] & 2021 & 63.3 & - & - & - \\ COCO & MCB 7 att. [135] & 2016 & 66.5 & - & - & - \\ VVC [221] & VL-BERT-Large [79] & 2019 & 75.5 & - & - & - \\ GQA [131] & NSM [222] & 2019 & 63.2 & 78.9 & - & - \\ CLEVR [127] & NS-VQA [136] & 2018 & 99.8 & - & - & - \\ Ioora [223] & Patch-TRM [223] & 2021 & 82.7 & - & - & - \\ MSRVT-QA [224] & Just Ask [225] & 2020 & 41.5 & - & - & - \\ \hline \multicolumn{5}{l}{**Multimodal Machine**} & \multicolumn{4}{c}{BLEU} & \multicolumn{4}{c}{METEOR} \\ \hline Multi30k [152] & DCCN [205] & 2020 & 39.7 & 56.8 & & \\  & Caglayan [226] & 2019 & 39.4 & 58.7 & & \\  & MM Transformer [160] & 2020 & 38.7 & 55.7 & & \\ \hline \multicolumn{5}{l}{**Text-Image Retrieval**} & \multicolumn{4}{c}{Recall@1} & \multicolumn{4}{c}{Recall@5} & \multicolumn{4}{c}{Recall@10} \\ \hline COCO Captions & VisualSparta [206] & 2020 & 68.2 & 91.8 & 96.3 & \\ COCO & Oscar & 2020 & 78.2 & 95.8 & 98.3 & \\ Flickr30k & VisualSparta & 2020 & 57.4 & 82.0 & 88.1 & \\ RoDI-ML [227] & ADAPT-12T [227] & 2021 & 19.0 & 30.0 & 45.0 & \\ WIT [228] & WIT-ALL [228] & 2021 & 34.6 & 64.2 & - & \\ Fashion IQ [229] & RTIC-GCN [230] & 2021 & - & - & 40.6 & \\ \hline \multicolumn{5}{l}{**Text-to-Image**} & \multicolumn{4}{c}{FID [211]} & \multicolumn{4}{c}{IS [210]} \\ \multicolumn{5}{l}{**Generation**} & \multicolumn{4}{c}{} \\ \hline COCO & Lafite [212] & 2021 & 8.1 & 32.3 & & \\ CUB [207] & Lafite & 2021 & 10.5 & 6.0 & & \\ CelebA-HQ [209] & Lafite & 2021 & 12.5 & 2.9 & & \\ Oxford Flower [208] & StackGAN++ [187] & 2018 & 48.7 & 3.3 & & \\ \hline \multicolumn{5}{l}{**Text Generation**} & \multicolumn{4}{c}{BLEU-2} & \multicolumn{4}{c}{BLEU-3} & \multicolumn{4}{c}{BLEU-4} & \multicolumn{4}{c}{BLEU-5} \\ \hline COCO Captions & LeakGAN [231] & 2017 & 0.95 & 0.88 & 0.78 & 0.69 \\  & RankGAN [232] & 2017 & 0.85 & 0.67 & 0.56 & 0.54 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Multimodal benchmark.