SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we present empirical results addressing the following questions: (1) How effective and robust is the OOD detection layer? (2) What improvement does the Jacobian-Free backpropagation (JFB) provide in computational cost within our VK-OOD architecture? (3) How do external knowledge resources influence the performance of multimodal pipelines? (4) Can VK-OOD work with different backbones in large-scale settings for various downstream tasks? To answer (1) and (2), we performed experiments on VQA task with various setups, comparing memory and training costs with unrolled EM (vanilla), Jacobian-based-EM, and JFB-EM. To answer (3), we evaluated the influences of knowledge resources by explicitly (or intentionally) introducing outliers in the pipeline. Finally, for (4) we utilized ViLT [27], CLIP [49], and BLIP [31] as the backbone model and illustrate performance on downstream tasks, including natural language for visual reasoning and image-text retrieval. More implementation and training details are described in the Appendix C.1. For ablation studies, we fixed the backbone model to be ViLT with pretrained parameters.
4.1 Results and analysis of OOD detection layer
Backprop methods. We used different backpropagation methods in OOD detection layer with ViLT as the backbone. As shown in Table 1, compared to vanilla and Jacobian-based methods, JFB-EM
Method	#Param(M)	#FLOPs(G)	Time(m)/epoch	Max Mem(Mb)	VQAv2
Forward	Backward
Vanilla-EM	152.6	185.2	39.6	26.8	18673	76.6
JB-EM	125.2	115.7	39.6	12.7	14512	76.6
JFB-EM	124.8	108.6	39.6	6.3	13674	76.8
Table 1: Experimental results of different backpropagation method in the dense OOD detection layer. JFB-EM is much more efficient in backward pass and use less memory. It also outperforms on the VQAv2 task in terms of accuracy.
k=1
Figure 2: Visualization of the multimodal feature space. Here,  $k$  denotes the number of clusters.
k=4
k=5
outperformed them in term of accuracy on VQAv2 task. It is trained with significantly less memory and time cost per epoch, which is beneficial for large-scale settings. Additionally, we conducted experiments on  $T$ -number of iterations, which is a critical parameter to compute fixed points. The results are shown in Figure 3a. We found the improvements to be marginal for  $T \geq 5$ , and JFB achieves faster performance on all settings. Based on the results in Figure 3a, we fixed  $T = 10$  for the subsequent experimental settings. Additionally, we have provided the fixed point error plot over iterations in Figure 3b. We observed that the squared euclidean distance between successive iterates indeed went to zero showing convergence of forward pass.
Number of Clusters. We investigated the effects of the number of clusters on optimizing the GMM process, and the results are presented in Figure 3c (see green line). The general trend indicates that performance improves with an increase in the number of clusters. Qualitatively, Figure 2 shows that learned output features using U-MAP embeddings [42]. Feature embedding spaces of multiple modalities on COCO val dataset are shown in different colors representing different clusters. We provide example images of the clusters in Appendix C.3. The results show that our VK-OOD model can identify clusters over the extracted multimodal features – our layer can accurately detect outliers.
Robustness of OOD layer. Usually OOD features are not present within the training datasets themselves. However, we may encounter outliers when integrating external knowledge triplets into the training pipeline. If we denote  $M$  as the number of external knowledge triplets, then  $M = 0$  corresponds to the ID setup – no outliers. To delve into further quantitative analysis, we considered two setups: one with  $M = 0$ , i.e., ID setup, and OOD setup with  $M = 5$  (so possibly 5 outliers per language caption) that corresponds to augmenting features from external knowledge. We can see from the results in Table 2b that there is not a significant difference from the rate of convergence perspective — as indicated by squared norm of successive iterates  $\| \mu_t - \mu_{t + 1} \|_2^2$  — in both setups. However, from the accuracy (Acc) column in Table 2b, we can conclude that the performance in VQA tasks has significant improvements over iterations when considering external knowledge.
(c)
Figure 3: Ablation studies results on VQA task. (a) Results on accuracy with different  $T$  iterations in EM on VQAv2 dataset. (b)  $\| \mu_t - \mu_{t + 1}\| _2^2$  over EM iterations in forward pass.(c) Results on accuracy with different numbers of external triplets and the number of clusters in the GMM process (see Equation (2)) on OKVQA dataset.
Incomplete Knowledge Triplets. To evaluate the sensitivity of models to OOD detection performance, we conducted experiments of incomplete knowledge triplets with missing values. Note that, we only dropped language features inputs due to computational reasons - augmenting image patches requires more resources such as GPUs. However, since  $x$  is used for both visual and language features, the implementation remains the same as that of dropping patches in language features. So, it is equivalent to drop either the visual or the language feature since the computational effort involved in running EM algorithm depends only on the total number of features. We now present more results of  $\| \mu_t - \mu_{t + 1} \|_2^2$  and OKVQA performance in term of accuracy over optimization iterations in Table 2a. With higher level of incompleteness, the rate of convergence is slower as expected. Once again, as in previous robustness experiments, we found accuracy gain over iterations here also.
25%	50%	75%
T	Err	Acc	Err	Acc	Err	Acc
1	2.213	48.2	2.448	47.3	2.735	44.6
3	0.174	48.9	0.214	47.7	0.208	45.1
5	0.065	50.6	0.092	48.6	0.244	46.0
8	0.057	51.2	0.108	49.1	0.112	46.6
10	0.051	51.8	0.084	49.4	0.167	46.9
(a)  $\mu$  error and OKVQA accuracy
ID(M=0)	OOD(M=5)
T	Err	Acc	Err	Acc
1	1.94	73.6	2.15	73.1
3	0.059	73.8	0.089	74.8
5	0.038	73.9	0.051	76.1
8	0.042	73.9	0.036	76.5
10	0.036	74.1	0.034	76.8
Table 2: Fixed point error is measured using consecutive iterate distance  $\left\| {\mu }_{t} - {\mu }_{t + 1}\right\| /2$  . (a)  $\mu$  error and OKVQA accuracy over the EM optimization iterations in different level of incompleteness. (b)  $\mu$  error and VQA accuracy over the EM optimization iterations in ID and OOD setups.
4.2 Ablation study on VK-OD Components and External Knowledge
Effectiveness of Each Component. To compare the impact of the proposed OOD layer in VK-OOD, we considered different combinations of inclusion and exclusion of knowledge graph representations (KG) and OOD detection layer. The results are shown in Table 3. The results show that our model achieves the best performance when both the components are included in the model. Moreover, comparing the results on VQAV2 and OKVQA datasets, the results imply that external knowledge triplets (KG) can be beneficial to improve the performance especially on VQA task. Furthermore, using OOD layer even when there are no external knowledge triplets has good performance. This shows that including OOD layer in our model is helpful and able to capture the noise of multiple modalities, such as missing or mismatching modalities.
Numbers of external knowledge triplets. We conducted experiments to explore the impact of the amount of the knowledge triplets. We evaluate this on VQA tasks using OKVQA dataset. Figure 3c shows the experimental results. As expected, we observed that increasing the number of retrieved knowledge triplets improve the accuracy of predicted answers. We achieved the best accuracy of  $52.4\%$  when the number of triplets is  $\approx 5$ .
Knowledge resources. We evaluated different knowledge resources, i.e., different embeddings of implicit and/or explicit knowledge. Table 4 shows benefits provided by different external resources in our pipeline. We queried
(b)  $\mu$  error and VQA accuracy
Method	Downstream tasks
KG	OOD	VQAV2	OKVQA
73.9	45.5
✓	74.6	48.3
✓	74.1	46.2
✓	✓	76.8	52.4
Table 3: Ablation studies of different components of VK-OOD. "KG" and "OOD" denote knowledge graph and OOD detection layer respectively.
knowledge from Wikidata [59] and used BERT to get embeddings  $l_{j} \in \mathbb{R}^{d}$ . Our model produced  $1.8\%$  and  $18.7\%$  more accurate results than the best and worst performing baselines respectively. Moreover, using ConcepNet embeddings solely, our multimodal training pipeline also learn implicit knowledge in the multimodal fusion encoder. We compared our model performance with the models using external knowledge resources. As results shown in Table 4, all the different external knowledge resources brought improvements in the OKVQA tasks. Our proposed model takes advantages of implicit knowledge from vision-language models and integrates explicit knowledge prior information, thus outperforms other models using external knowledge resources without OOD detection.
4.3 Scalability of VK-OD
In this section, we incorporated the proposed OOD detection layer to various backbone models with different architectures and model sizes. We initialized the parameters  $(\mu$  and  $\sigma)$  of our proposed
OOD detection layer in two different ways, scalar  $\sigma$  or the dense one which  $\sigma$  is a  $d\times d$  matrix, where  $d$  is the dimension of input embeddings. As shown in Table 5, the number of parameters in scalar VK-OOD are approximately similar to the baselines. Specifically, comparing to other baseline models, while our scalar VK-OOD increases the #-parameters slightly  $-\approx 0.4$  million more parameters (since  $d\approx 700$ ) - it significantly improved the performance in downstream tasks.
Method	Knol wedge resources	OKVQA
ConceptBERT	CN	33.7
KRISP	Wiki + CN	38.4
MAVEx	Wiki + CN + GI	39.4
KAT-B	Wiki + GPT3	50.6
UnifER	CN + ViLT	42.1
VK-OOD (Ours)	CN - w/o BERT	51.1
Wiki	51.9
Wiki + CN	52.2
CN	52.4
We then conducted experiments on several downstream tasks. In all these experiments, our model consistently achieved the best and second-best performance compared to five state-of-the-art (SOTA) vision-language models across three downstream tasks on various datasets(see Table 5). We trained on open-source data and compare the results with the models having similar number of parameters. In particular, for VQA tasks, we evaluated on the VQAv2 test set. As shown in Table 5, our model outperformed all the baselines on this dataset, yielding an accuracy of  $77.9\%$ .
Furthermore, in the Natural Language for Visual Reasoning (NLVR) task, VK-OOD achieved the best and second-best result in terms of accuracy with different backbone models, respectively. We also evaluated our model, along with the baseline models, on the test set of COCO and F30K dataset for image-text retrieval task. Our model produced the best performance and outperformed the best and worst-performing baselines, with the exception of BLIP, by up to  $0.5\%$  and  $11.6\%$  respectively on the COCO dataset (Table 5). In other settings and on the F30K dataset, the results are similar. Further details are provided in Appendix C.2.
The performance of our model demonstrates its ability for visual reasoning while integrating both implicit and explicit knowledge. Moreover, we are able to show that the proposed OOD detection layer can scale with different models and contribute improvements to various tasks. For inference, since  $\mu$  and  $\sigma$  are fixed, the inference time is similar to the backbone models considered – the average inference time for one VQA instance is around 57 ms on a single 2080Ti GPU. Therefore, we believe that our implicit layer can function as a plug-and-play module and can be easily integrated with other vision-language models.
Visualizing Attention with Multimodal Information. Figure 4 is an example of multimodal alignment results from our VK-OOD representations. We use Grad-cam [52] to visualize the multimodal maps of the models on the image corresponding to knowledge triplets. As shown in
Table 4: Results on the different knowledge resources on OKVQA dataset. "CN" and "GI" denote ConceptNet and Google Images respectively.
Model	#Params	VQAv2	NLVR2	COCO	Flickr30k
TR R@5	IR R@5	TR R@5	IR R@5
ViLT	87	70.3	74.6	86.2	72	95.6	86.8
UNITER	155	72.7	75.8	87.4	78.5	97.1	92.4
ALBEF	314	74.5	80.5	91.4	81.5	99.4	96.7
VinVL	157	75.9	83.1	92.6	83.2	-	-
BLIP*	346	77.5	82.8	95.2	85.4	99.8	97.5
VK-OOD-s(ViLT)	87.4	76.7	84.3	90.9	81.6	97	94.3
VK-OOD-s(CLIP)	113.4	76.2	83.8	92.8	83.4	99.6	96.7
VK-OOD-s(BLIP)	346.4	77.8	84.1	95.4	85.2	99.8	97.2
VK-OOD-l(ViLT)	125	76.8	84.6	91.7	81.3	97.2	94.5
VK-OOD-l(CLIP)	151	76.1	83.9	93.1	83.6	99.6	96.8
VK-OOD-l(BLIP)	412	77.9	84.5	95.1	84.8	99.6	97.1
Table 5: Overall performance on multiple downstream tasks. We demonstrate VK-OOD scale with different model backbones and achieve the best and second-best results. VK-OOD-s is the scalar case, and VK-OOD-1 is the dense case. *our implementation.
A man riding bicycle
Car locates on street
Bus locates on street
Traffic lights locate on street
Buildings locate on street
Figure 4: Visualization of the attention maps of image and triplets alignment. The original sample caption is "A man riding a bicycle down a city street". We highlight areas in the image corresponding to different knowledge triplets. Our model learns different objects and localize those objects correctly.
Figure 5: Example case studies with OKVQA dataset. We show the examples of retrieved knowledge triplets. Our model is able to detect outliers of retrieved triplets shown in orange. The predicted answers are finetuning on OKVQA dataset. Comparing with the baseline results, our model provides more correct answers. Note that, the baseline model here is our implementation without KG and OOD components.
Figure 4, our model has the capability to attend to the extracted knowledge concepts, such as buildings and traffic lights. Thus, our model can detect more objects to provide the ability for answering open questions. We discuss more user studies on interactive OOD detections by feeding in domain knowledge with different distributions in the Appendix C.3.
Furthermore, we present prediction results in Figure 5 with VK-OOD model on OKVQA dataset along with the extracted knowledge triplets based on the captions. In the example, the caption  $\langle$  apple, used for, making apple pie  $\rangle$  is useful to obtain correct answers. This observation validated that explicit knowledge provides more reasoning capability than implicit knowledge. Moreover, our model detected OOD triplets by combining modalities, i.e., the apple fruit is in image, thus is not used for computing. The last one is a failure case, because the ground truth caption is not sufficiently specific. Therefore, it might be beneficial to consider more inference and reasoning abilities in multimodal analysis, such as chain of thoughts [62].
Limitations. A potential limitation of our proposed implicit layer arises when the covariance matrix  $\sigma$  is dense. In such cases, a fast linear system solver would be required to evaluate the likelihood. We will consider to explore sparse approaches to further saccelerater OOD detection layer in the future. Additionally, given that we have demonstrated that explicit knowledge can serve as supervision in vision-language training, we believe that various knowledge bases, such as medical knowledge graphs, can provide user-desired domain distributions.
Conclusions. In this work, we presented a training framework designed to facilitate multimodal analysis under distribution shifts and/or the presence of outlier distributions within the input feature space. Several other models have been proposed to exploit special structures in the available modalities for faster training purposes, as mentioned in [61], and for tasks involving egocentric vision [64]. While the approaches have been shown to perform well in large-scale settings, they may not be sufficient on their own. For instance, most frames in a video have low semantic information content and may require complex processing pipelines [8]. We argued that handling outliers within the context of multimodal analysis is an crucial topic as more models are integrated or fused. Our implicit OOD detection layer can be directly instantiated within such complex pipelines, possibly allowing us to intervene and accelerate the training process while reducing computation costs. We demonstrated through extensive empirical analysis across various setups that incorporating OOD detection in the training pipeline can significantly enhance the performance on downstream tasks.
[55] R. Speer, J. Chin, and C. Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In Thirty-first AAAI conference on artificial intelligence, 2017.
[56] A. Suhr, S. Zhou, A. Zhang, I. Zhang, H. Bai, and Y. Artzi. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491, 2018.
[57] J. Sun, Y. Li, H.-S. Fang, and C. Lu. Three steps to multimodal trajectory prediction: Modality clustering, classification and synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13250-13259, 2021.
[58] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008, 2017.
[59] D. Vrandecic and M. Krötzsch. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78-85, 2014.
[60] H. Wang, Y. Zhang, Z. Ji, Y. Pang, and L. Ma. Consensus-aware visual-semantic embedding for image-text matching. In European Conference on Computer Vision, pages 18–34. Springer, 2020.
[61] Z. Wang, J. Yu, A. W. Yu, Z. Dai, Y. Tsvetkov, and Y. Cao. Simvlm: Simple visual language model pretraining with weak supervision. arXiv preprint arXiv:2108.10904, 2021.
[62] J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022.
[63] J. Wu, J. Lu, A. Sabharwal, and R. Mottaghi. Multi-modal answer validation for knowledge-based vqa. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2712-2721, 2022.
[64] A. Zeng, A. Wong, S. Welker, K. Choromanski, F. Tombari, A. Purohit, M. Ryoo, V. Sindhwani, J. Lee, V. Vanhoucke, et al. Socratic models: Composing zero-shot multimodal reasoning with language. arXiv preprint arXiv:2204.00598, 2022.
[65] C. Zhang, Z. Yang, X. He, and L. Deng. Multimodal intelligence: Representation learning, information fusion, and applications. IEEE Journal of Selected Topics in Signal Processing, 14(3):478-493, 2020.
[66] P. Zhang, X. Li, X. Hu, J. Yang, L. Zhang, L. Wang, Y. Choi, and J. Gao. Vinvl: Revisiting visual representations in vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5579-5588, 2021.
A More details of Multimodal Pipeline
A.1 Image Encoder
We extracted features from image patches and used vision transformer based models(ViTs) [58] as our encoder for the patches. To do so, we split input image into a sequence of patches and project them using linear maps to obtain embedding of patch features  $v_{q}$ , which simplifies the step for fusing with text embedding. Following the vision-language models, we trained our model with multiple popular ViTs to examine the influence of image encoder in OOD detection backpropagation process.
A.2 Knowledge Retrieval Module
Given the caption  $S$ , we first parsed it into triplets in the form of  $T^{id} = \langle o(c), r(c), o'(c) \rangle$ , where  $o(c)$  and  $o'(c)$  are concepts  $\in C^{id}$ , and  $r(c)$  is the relation(s) between them, i.e.,  $\langle \text{man}, \text{riding}, \text{bicycle} \rangle$ . In our example, the seed triplets (ID triplets) parsed from the caption are  $\langle \text{man}, \text{riding}, \text{bicycle} \rangle$  and  $\langle \text{bicycle}, \text{down}, \text{street} \rangle$ . Then we augmented knowledge by matching these triplets with external open knowledge including domain and commonsense knowledge graphs, i.e., ConceptNet [55]. ConceptNet provides a large scale commonsense knowledge with over 21 million edges by 36 type of relations, i.e., IsA, UsedFor, AtLocation, connecting 8 million nodes. To complete our knowledge graph based augmentation, we collected concepts by querying from ConceptNet using  $o(c)$ ,  $o'(c)$  and  $rel_i$  where  $i \in [0,36]$  and concatenated them to seed triplets. For example, given "street" as  $o(c)$  and "AtLocation" as  $rel_i$ , we will extract the related concepts are located at street to form triple  $T_i$ . Specifically, we queried explicit knowledge triplets of  $o(c)$  and  $o'(c)$  from ConceptNet to form  $T^{cn}$ , i.e.,  $\langle \text{bicycle}, \text{used for}, \text{transport} \rangle$ . Finally, these knowledge triplets  $\in T = T^{cn} \cup T^{id}$  are encoded as language features, such as  $l_j$  using a language encoder (i.e., BERT[14]).
A.3 Multimodal Fusion Encoder
We used our proposed OOD detection layer as a "plug-and-play" module that can be used in different vision-language architectures. In ViLT and CLIP, we added the visual and textual embeddings with  $s(\cdot)$  as in Equation (3), and passed them to a standard L-depth transformer. While using the BLIP backbones, we implemented a two-stream transformer pipeline consisting of stacked multiple layers to joint vision and knowledge textual representations. For each layer, we used self-attention unit and merged cross-attention unit, thus integrating vision and knowledge semantic information and the alignments across them.
Multimodal Attention Module. As in the standard transformer architecture [58], the attention function computes identical learnable parameters (weights) as in Equation (8) and Equation (9), where  $d$  is the dimension of the inputs, a query Q, key K, and value V. We used fusion encoder to compare similarity among the image-text pairs given by,
$$
\operatorname {A t t n} \left(Q _ {I}, K _ {L}, V _ {L}\right) =
$$
$$
\operatorname {s o f t m a x} \left(\frac {Q _ {I} K _ {L} ^ {T}}{\sqrt {d}}\right)) V _ {L}, \tag {8}
$$
$$
\operatorname {A t t n} \left(Q _ {L}, K _ {I}, V _ {I}\right) =
$$
$$
\operatorname {s o f t m a x} \left(\frac {Q _ {L} K _ {I} ^ {T}}{\sqrt {d}}\right) V _ {I} \tag {9}
$$
where  $I$  and  $L$  correspond to image modality and language modality.
In implementation of multimodal encoder, we updated the image and language embedding outputs themselves from previous layer as queries and concatenate them together to get keys and values. As a further improvement of attention function, we used a multi-head attention which is composed by multiple paralleled attention function in each head. The feed-forward layers transform the outputs of multi-head attention through two fully-connected layers with GeLU activation.
B More details on formulating Training objectives
We provide more details on our training objectives in our pipeline in this section, including image text matching (ITM) and masked language modeling (MLM).
B.1 Image Text Matching (ITM)
To incorporate both the vision and the language representations, we used ITM which is widely used in previous VL studies. Given an image and text of triple pair  $\langle v_q, l_j \rangle$ , ITM predicts whether they are matched as positive examples or not, and it is a binary classification problem with the loss function in Equation (5) and Equation (6). We assumed that each image and ID triple pair  $\langle v_q, l_j \rangle$ , as a positive example. The negative pairs are constructed through batch-sampling.
B.2 Masked Language Modeling (MLM)