## 4 Experiments
### Quantitative Results  
#### 4.2.1 Discriminative Setting  
We compare the results of our proposed model with previously published results on the VisDial v1.0 dataset for the following methods: Late Fusion (LF) [7], Hierarchical Recurrent Encoder (HRE) [7], Memory Network (MN) [7], Graph Neural Network (GNN) [12], Co-reference Neural Module Network (CorefNMN) [4], Recursive Visual Attention (RVA) [6], Synergistic Network [9], Dual Encoding Visual Dialogue (DualVD) [33], Context-Aware Graph (CAG) [14], History-Aware Co-Attention (HACAN) [34], Consensus Dropout Fusion (CDF) [11], Dual Attention Network (DAN) [5], and Factor Graph Attention (FGA) [13].  
Table 1 reports the quantitative results on the VisDial v1.0 under the discriminative decoder setting. For VisDial v1.0, our MVAN model outperforms the previous state-of-the-art methods with respect to NDCG and AVG and shows competitive results in non-NDCG metrics. Specifically, MVAN achieves significant improvements in NDCG from 57.59 to 59.37 and is ranked first as a result of the average rank of NDCG and MRR, compared to the state-of-the-art baseline. We also report the results for an ensemble of 10 independent models that were trained with random initial seeds. This leads to further performance improvements in NDCG from 59.37 to 60.92 and MRR from 64.84 to 66.38. These results indicate that MVAN not only has an accurate prediction ability, as indicated by the non-NDCG metric results (i.e., MRR, Rek, and Mean), but it has a powerful generalization capability given the result of NDCG score because this metric considers several relevant answers to be correct.  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Model** & **AVG \(\downarrow\)** & **NDCG \(\uparrow\)** & **MRR \(\uparrow\)** & **R@1 \(\uparrow\)** & **R@5 \(\uparrow\)** & **R@10 \(\uparrow\)** & **Mean \(\downarrow\)** \\ \hline LF [7] & 12 & 45.31 (13) & 55.42 (12) & 40.95 & 72.45 & 82.83 & 5.95 \\ HRE [7] & 12 & 45.46 (12) & 54.16 (13) & 39.93 & 70.45 & 81.50 & 6.41 \\ MN [7] & 11 & 47.50 (11) & 55.49 (11) & 40.98 & 72.30 & 83.30 & 5.92 \\ GNN [12] & 10 & 52.82 (10) & 61.37 (10) & 47.33 & 77.98 & 87.83 & 4.57 \\ CorefNMN [4] & 9 & 54.70 (9) & 61.50 (9) & 47.55 & 78.10 & 88.80 & 4.40 \\ RVA [6] & 8 & 55.59 (8) & 63.03 (7) & 49.03 & 80.40 & 89.83 & 4.18 \\ DualVD [33] & 7 & 56.32 (7) & 63.23 (5) & 49.25 & 80.23 & 89.70 & 4.11 \\ Synergistic [9] & 6 & 57.32 (3) & 62.20 (8) & 47.90 & 80.43 & 89.95 & 4.17 \\ CAG [14] & 5 & 56.64 (6) & 63.49 (4) & 49.85 & 80.63 & 90.15 & 4.11 \\ DAN [5] & 4 & 57.59 (2) & 63.20 (6) & 49.63 & 79.75 & 89.35 & 4.30 \\ HACAN [34] & 3 & 57.17 (4) & 64.22 (1) & 50.88 & 80.63 & 89.45 & 4.20 \\ FGA [13] & 2 & 56.90 (5) & **66.20** (1) & **52.75** & **82.92** & **91.07** & **3.80** \\ MVAN (ours) & 1 & **59.37** (1) & 64.84 (2) & 51.45 & 81.12 & 90.65 & 3.97 \\ \hline Synergistic \({}^{\dagger}\)[9] & 5 & 57.88 (4) & 63.42 (5) & 49.30 & 80.77 & 90.68 & 3.97 \\ CDF \({}^{\dagger}\)[11] & 2 & 59.49 (2) & 64.40 (4) & 50.90 & 81.18 & 90.40 & 3.99 \\ DAN \({}^{\dagger}\)[5] & 2 & 59.36 (3) & 64.92 (3) & 51.28 & 81.60 & 90.88 & 3.92 \\ FGA \({}^{\dagger}\)[13] & 2 & 57.20 (5) & **69.30** (1) & **55.65** & **86.73** & **94.05** & **3.14** \\ MVAN \({}^{\dagger}\) (ours) & 1 & **60.92** (1) & 66.38 (2) & 53.20 & 82.45 & 91.85 & 3.68 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Results on VisDial v1.0 (test-std). \({}^{\dagger}\) denotes ensembles. Existing works [17; 18] based on pre-trained models are not reported for the fair comparison. The numbers in the brackets indicate rankings of the models in Normalized Discounted Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR) metrics.