SlowGuess/NAACL_2022_OCR huggingface dataset:
Models	Item	CMU-MOSI	CMU-MOSEI
MISA	Learning rate	1e-5	4e-5
Optimizer	RMSprop	Adam
Activation	hardtanh	relu
BBFN	Learning rate	1e-4	5e-5
MMIM	Batch size	32	64
learning rate ηlld	4e-3	1e-3
learning rate ηmain	1e-3	5e-4
α	0.3	0.1
β	0.1	0.05
V-LSTM hidden dim	32	64
A-LSTM hidden dim	32	16
MulT	Batch size	128	16
Learning rate	1e-4	1e-4
Optimizer	Adam	Adam
Transformers Hidden Unit Size d	30	30
No. of Crossmodal Attention Heads	10	10
No. of Crossmodal a Blocks D	4	4
Textual Embedding Dropout	0.3	0.3
Crossmodal Attention Block Dropout	0.2	0.1
Output Dropout	0.2	0.1
Gradient Clip	0.8	1.0
No. of Epochs	100	20
Use Bert	Yes	Yes
Table 2: Hyper-parameter config used to train the models.
B Reproducing Results
For each model we train the models to achieve performances close to reported in the respective papers. Table 2 presents the hyper-parameters we used to reproduce their results.
C Additional Results on Modality-Perturbation
We also analyze with varying proportions of perturbations in the training and testing phase, respectively. As seen in Table 3, as the noise gradually increases from  $5\%$  to  $15\%$ , the drop of Corr in MOSI is gradually reduced, which shows the robustness is getting better, until it reaches the optimum at  $30\%$  perturbation (15% missing + 15% noise). In other models,  $30\%$  perturbation is also advantageous. For example, in Table 7, (Mult, MOSEI) reduces Corr drop while improving F1 performance in  $30\%$  perturbation. Although it is only a small improvement at present, we believe that there will be more meaningful improvements in the future.
$^{3}$ https://github.com/declare-lab/MISA/tree/ec42faddde0d210cf7368aebf2118fe9570e7102 $^{4}$ https://github.com/declare-lab/BBFN/tree/be15f947ed7539b3c54381e453f09439466ed91 $^{5}$ https://github.com/thuiar/Self-MM $^{6}$ https://github.com/declare-lab/Multimodal-Infomax $^{7}$ https://github.com/yaohungt/Multimodal-Transformer
Robust Training	Diagnostic	MOSI	MOSEI
Corr	F1	Corr	F1
MISA	-	0.737	82.40	0.765	85.76
missing 5%	↓0.011	↓1.18	↓0.042	↓1.57
noise 5%	↓0.022	↓0.75	→0	→0
missing 10%	↓0.031	↓2.19	↓0.073	↓2.41
noise 10%	↓0.047	↓2.56	↓0.031	↓1.75
missing 15%	↓0.050	↓3.99	↓0.126	↓4.10
noise 15%	↓0.076	↓4.05	↓0.068	↓3.88
missing 30%	↓0.122	↓11.53	↓0.186	↓8.45
noise 30%	↓0.210	↓9.96	↓0.147	↓8.26
5% missing 5% noise	0.714	80.99	0.765	85.68
missing 5%	↓0.010	↓1.38	↓0.042	↓1.57
noise 5%	↓0.032	↓1.37	→0	→0
missing 10%	↓0.030	↓2.17	↓0.074	↓2.41
noise 10%	↓0.055	↓2.44	↓0.041	↓2.22
missing 15%	↓0.044	↓4.52	↓0.126	↓0.41
noise 15%	↓0.094	↓3.65	↓0.058	↓3.53
missing 30%	↓0.121	↓11.08	↓0.186	↓8.37
noise 30%	↓0.201	↓8.71	↓0.150	↓8.12
10% missing 10% noise	0.734	81.40	0.765	85.68
missing 5%	↓0.011	↓1.04	↓0.022	↓1.58
noise 5%	↓0.028	↓1.20	→0	→0
missing 10%	↓0.030	↓2.05	↓0.042	↓2.54
noise 10%	↓0.045	↓2.27	↓0.001	↓0.02
missing 15%	↓0.049	↓3.54	↓0.071	↓4.04
noise 15%	↓0.073	↓3.42	↓0.001	↓0.43
missing 35%	↓0.118	↓11.11	↓0.135	↓8.17
noise 30%	↓0.181	↓9.55	↓0.035	↓0.54
15% missing 15% noise	0.736	81.42	0.767	85.97
missing 5%	↓0.012	↓1.50	↓0.021	↓1.46
noise 5%	↓0.027	↓1.50	→0	→0
missing 10%	↓0.033	↓2.71	↓0.041	↓2.43
noise 10%	↓0.032	↓2.12	↓0.001	↓0.03
missing 15%	↓0.052	↓4.40	↓0.021	↓4.11
noise 15%	↓0.073	↓4.99	↓0.001	↓0.12
missing 30%	↓0.122	↓11.67	↓0.136	↓8.36
noise 30%	↓0.163	↓10.10	↓0.002	↓0.19
Table 3: MISA Robust Training. Results are averaged over three random runs.
Robust Training	Diagnostic	MOSI	MOSEI
Corr	F1	Corr	F1
BBFN	-	0.754	83.12	0.764	85.70
missing 5%	↓0.013	↓1.51	↓0.020	↓1.97
noise 5%	↓0.034	↓0.75	↓0.050	↓1.21
missing 10%	↓0.028	↓2.59	↓0.038	↓3.34
noise 10%	↓0.093	↓2.90	↓0.121	↓3.54
missing 15%	↓0.032	↓3.79	↓0.055	↓5.45
noise 15%	↓0.080	↓2.28	↓0.154	↓4.18
missing 30%	↓0.127	↓10.55	↓0.139	↓10.57
noise 30%	↓0.232	↓8.58	↓0.308	↓9.62
5% missing 5% noise	0.743	82.39	0.765	85.48
missing 5%	↓0.020	↓0.94	↓0.017	↓1.00
noise 5%	↓0.002	→0	→0	↓0.08
missing 10%	↓0.039	↓2.17	↓0.032	↓1.95
noise 10%	→0	→0	→0	↓0.06
missing 15%	↓0.045	↓2.18	↓0.050	↓3.57
noise 15%	↓0.002	↓0.15	↓0.001	→0
missing 30%	↓0.049	↓1.92	↓0.122	↓7.60
noise 30%	→0	↓0.20	↓0.001	↓0.04
10% missing 10% noise	0.742	81.66	0752	85.15
missing 5%	↓0.018	↓1.66	↓0.018	↓1.62
noise 5%	↓0.001	→0	↓0.022	↓0.05
missing 10%	↓0.034	↓2.87	↓0.035	↓2.87
noise 10%	↓0.001	→0	↓0.003	↓0.09
missing 15%	↓0.036	↓3.63	↓0.050	↓4.47
noise 15%	↓0.001	↓0.01	↓0.004	↓0.07
missing 30%	↓0.126	↓9.75	↓0.125	↓9.40
noise 30%	↓0.003	↓0.46	↓0.079	↓0.31
15% missing 15% noise	0.754	83.28	0.763	85.43
missing 5%	↓0.020	↓1.50	↓0.018	↓0.96
noise 5%	↓0.001	→0	↓0.001	→0
missing 10%	↓0.031	↓2.26	↓0.036	↓2.08
noise 10%	↓0.002	→0	→0	↓0.05
missing 15%	↓0.035	↓2.71	↓0.057	↓3.62
noise 15%	↓0.003	↓0.14	↓0.001	→0
missing 30%	↓0.119	↓7.28	↓0.124	↓7.88
noise 30%	↓0.046	↓0.16	↓0.003	↓0.23
Table 4: BBFN Robust Training. Results are averaged over three random runs.
Robust Training	Diagnostic	MOSI	MOSEI
Corr	F1	Corr	F1
Self-MM	-	0.794	85.61	0.759	84.62
missing 5%	↓0.023	↓1.93	↓0.018	↓1.99
noise 5%	↓0.009	↓0.91	↓0.022	↓1.32
missing 10%	↓0.046	↓3.57	↓0.040	↓3.51
noise 10%	↓0.039	↓2.43	↓0.058	↓2.91
missing 15%	↓0.051	↓4.77	↓0.056	↓4.51
noise 15%	↓0.050	↓2.71	↓0.069	↓3.65
missing 30%	↓0.099	↓11.74	↓0.126	↓9.04
noise 30%	↓0.154	↓8.35	↓0.172	↓9.48
5% missing 5% noise	0.798	83.97	0.789	0.837
missing 5%	↓0.022	↓1.73	↓0.021	↓1.83
noise 5%	↓0.002	→0	↓0.002	↓0.05
missing 10%	↓0.046	↓3.32	↓0.046	↓3.28
noise 10%	↓0.012	↓0.14	↓0.011	↓0.44
missing 15%	↓0.050	↓4.63	↓0.053	↓4.60
noise 15%	↓0.018	↓0.09	↓0.018	↓0.51
missing 30%	↓0.119	↓9.57	↓0.124	↓9.77
noise 30%	↓0.046	↓0.33	↓0.043	↓1.04
10% missing 10% noise	0.789	83.67	0.764	0.849
missing 5%	↓0.021	↓1.83	↓0.017	↓0.63
noise 5%	↓0.002	↓0.05	↓0.005	↓0.03
missing 10%	↓0.046	↓3.28	↓0.038	↓1.93
noise 10%	↓0.011	↓0.44	↓0.011	↓0.15
missing 15%	↓0.053	↓4.60	↓0.057	↓3.16
noise 15%	↓0.018	↓0.51	↓0.019	↓0.13
missing 30%	↓0.124	↓9.77	↓0.126	↓7.42
noise 30%	↓0.043	↓1.04	↓0.056	↓1.01
15% missing 15% noise	0.790	84.73	0.754	84.67
missing 5%	↓0.022	↓1.91	↓0.017	↓0.60
noise 5%	↓0.003	→0	↓0.004	↓0.01
missing 10%	↓0.046	↓3.39	↓0.038	↓1.91
noise 10%	↓0.010	↓0.44	↓0.011	↓0.14
missing 15%	↓0.049	↓4.58	↓0.055	↓3.08
noise 15%	↓0.018	↓0.28	↓0.016	↓0.16
missing 30%	↓0.120	↓9.66	↓0.122	↓6.86
noise 30%	↓0.041	↓0.58	↓0.051	↓1.02
Table 5: Self-MM Robust Training. Results are averaged over three random runs.
Robust Training	Diagnostic	MOSI	MOSEI
Corr	F1	Corr	F1
MMIM	-	0.796	86.02	0.758	84.89
missing 5%	↓0.028	↓1.34	↓0.016	↓0.93
noise 5%	↓0.035	↓1.52	↓0.031	↓1.71
missing 10%	↓0.067	↓4.16	↓0.034	↓2.43
noise 10%	↓0.058	↓2.10	↓0.070	↓2.64
missing 15%	↓0.056	↓2.78	↓0.056	↓4.13
noise 15%	↓0.078	↓4.65	↓0.094	↓4.67
missing 30%	↓0.117	↓9.37	↓0.122	↓8.15
noise 30%	↓0.197	↓9.55	↓0.191	↓9.18
5% missing 5% noise	0.797	85.13	0.755	0.836
missing 5%	↓0.057	↓1.19	↓0.016	↓0.62
noise 5%	↓0.021	↓1.20	↓0.025	↓1.07
missing 10%	↓0.035	↓1.80	↓0.046	↓2.29
noise 10%	↓0.075	↓2.25	↓0.056	↓2.43
missing 15%	↓0.057	↓4.37	↓0.168	↓2.26
noise 15%	↓0.072	↓3.96	↓0.063	↓2.75
missing 30%	↓0.117	↓8.24	↓0.120	↓6.52
noise 30%	↓18.75	↓9.31	↓0.178	↓7.34
10% missing 10% noise	0.794	84.76	0.758	84.81
missing 5%	↓0.020	↓1.50	↓0.016	↓1.07
noise 5%	↓0.006	↓0.46	↓0.024	↓1.28
missing 10%	↓0.032	↓2.09	↓0.043	↓2.79
noise 10%	↓0.060	↓2.57	↓0.045	↓2.58
missing 15%	↓0.060	↓3.27	↓0.058	↓3.84
noise 15%	↓0.062	↓3.65	↓0.063	↓3.42
missing 30%	↓0.110	↓8.40	↓0.119	↓7.50
noise 30%	↓0.163	↓9.18	↓0.177	↓8.16
15% missing 15% noise	0.784	84.67	0.751	83.15
missing 5%	↓0.014	↓1.79	↓0.020	↓0.89
noise 5%	↓0.028	↓1.04	↓0.016	↓0.69
missing 10%	↓0.038	↓2.18	↓0.030	↓2.24
noise 10%	↓0.050	↓1.30	↓0.028	↓0.62
missing 15%	↓0.061	↓4.91	↓0.059	↓2.77
noise 15%	↓0.071	↓4.08	↓0.040	↓1.58
missing 30%	↓0.146	↓10.48	↓0.115	↓6.62
noise 30%	↓0.180	↓8.88	↓0.096	↓4.41
Table 6: MMIM Robust Training. Results are averaged over three random runs.
Robust Training	Diagnostic	MOSI	MOSEI
Corr	F1	Corr	F1
MuIT	-	0.747	82.25	0.738	83.37
missing 5%	↓0.032	↓3.76	↓0.019	↓0.84
noise 5%	↓0.053	↓1.67	→0	→0
missing 10%	↓0.046	↓3.92	↓0.031	↓1.85
noise 10%	↓0.069	↓3.03	→0	→0
missing 15%	↓0.052	↓4.84	↓0.051	↓3.29
noise 15%	↓0.172	↓5.00	↓0.152	↓4.50
missing 30%	↓0.113	↓12.17	↓0.109	↓6.60
noise 30%	↓0.295	↓8.99	↓0.263	↓7.95
5% missing 5% noise	0.748	81.90	0.748	84.59
missing 5%	↓0.031	↓1.50	↓0.018	↓0.93
noise 5%	↓0.021	↓1.67	→0	→0
missing 10%	↓0.043	↓2.43	↓0.030	↓1.99
noise 10%	↓0.021	↓1.96	→0	→0
missing 15%	↓0.050	↓3.36	↓0.050	↓3.51
noise 15%	↓0.067	↓3.30	↓0.173	↓3.97
missing 30%	↓0.077	↓5.12	↓0.074	↓4.90
noise 30%	↓0.094	↓5.17	↓0.198	↓5.50
10% missing 10% noise	0.741	81.31	0.746	84.13
missing 5%	↓0.034	↓2.57	↓0.019	↓0.96
noise 5%	↓0.012	↓0.09	→0	→0
missing 10%	↓0.046	↓3.06	↓0.030	↓0.019
noise 10%	↓0.031	↓2.42	→0	→0
missing 15%	↓0.053	↓5.52	↓0.048	↓3.19
noise 15%	↓0.045	↓4.08	↓0.153	↓4.08
missing 30%	↓0.077	↓6.57	↓0.072	↓4.62
noise 30%	↓0.056	↓4.57	↓0.202	↓5.58
15% missing 15% noise	0.744	82.21	0.745	83.95
missing 5%	↓0.034	↓2.56	↓0.018	↓0.86
noise 5%	↓0.023	↓1.52	→0	→0
missing 10%	↓0.047	↓3.93	↓0.032	↓1.86
noise 10%	↓0.039	↓3.62	→0	→0
missing 15%	↓0.052	↓5.15	↓0.051	↓3.27
noise 15%	↓0.046	↓3.62	→0	↓0.08
missing 30%	↓0.117	↓9.63	↓0.113	↓7.03
noise 30%	↓0.068	↓6.13	↓0.001	↑0.02
Table 7: MulT Robust Training. Results are averaged over three random runs.