Implicit Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis
Authors: Zhu Wang
Sourav Medya
Sathya N. Ravi
Venue: NeurIPS 2023 poster (2023)
Keywords: Implicit layer
Out-of-distribution detection
multimodal learning

Abstract: Deep network models are often purely inductive during both training and inference on unseen data. When these models are used for prediction, but they may fail to capture important semantic information and implicit dependencies within datasets. Recent advancements have shown that combining multiple modalities in large-scale vision and language settings can improve understanding and generalization performance. However, as the model size increases, fine-tuning and deployment become computationally expensive, even for a small number of downstream tasks. Moreover, it is still unclear how domain or prior modal knowledge can be specified in a backpropagation friendly manner, especially in large-scale and noisy settings. To address these challenges, we propose a simplified alternative of combining features from pretrained deep networks and freely available semantic explicit knowledge. In order to remove irrelevant explicit knowledge that does not correspond well to the images, we introduce an implicit Differentiable Out-of-Distribution (OOD) detection layer. This layer addresses outlier detection by solving for fixed points of a differentiable function and using the last iterate of fixed point solver to backpropagate. In practice, we apply our model on several vision and language downstream tasks including visual question answering, visual reasoning, and image-text retrieval on different datasets. Our experiments show that it is possible to design models that perform similarly to state-of-the-art results but with significantly fewer samples and less training time. Our models and code are available here: https://github.com/ellenzhuwang/implicit_vkood

Implicit Differentiable Outlier Detection Enables Robust Deep Multimodal Analysis
Zhu Wang
Sourav Medya Sathya N. Ravi
Department of Computer Science, University of Illinois at Chicago {zwang260, mendya,,,mmdda,,. wathaaaasccccccccaccoooo edi
Abstract
Deep network models are often purely inductive during both training and infer- ence unseen data. When these models are used for prediction, but they may fail capture important semantic information and implicit dependencies within datasets. Recent advancements have shown that combining multiple modalities in large-scale vision and language settings can improve understanding and gen- eralization performance. However, the model size increases, fine-tuning and deployment become computationally expensive, even for a small number of down- stream tasks. Moreover, it is still unclear how domain prior modal knowledge can be specified in a backpropagation friendly manner, especially in large-scale and noisy settings. To address these challenges, propose simplified alternative of combining features from pretrained deep networks and freely available semantic explicit knowledge. In order remove irrelevant explicit knowledge that does not correspond well to the images, we introduce an implicit Differentiable Out- of of-Distribution (OOD) detection layer. This layer addresses outlier detection by solving for fixed points of differentiable function and using the last iterate of fixed point solver backpropagate. In practice, we apply our model on several vision and language downstream tasks including visual question answering, visual reasoning, and image-text retrieval on different datasets. Our experiments show that is possible to design models that perform similarly to state-of sttte-of-the-ar results but with significantly fewer samples and less training time. Our models and code are available here: httts://githuu
Introduction
Numerous neural network models constructed via the stacking of explicit layers that transform inputs into outputs through a sequence of operations associated with the designated layers. Although these explicit layers expressive, they may be unnecessary in many large-scale applications where only the function value and its gradients are required. For instance, it is possible to employ implicit layers for diverse tasks, including hyperparameter optimization 77 meta learning [50]. and solving inverse problems in image processing (2I). Indeed, if a desired input-output correspondence within network can be expressed as an optimization problem, gradients can be computed efficiently (30).
Recently, JFB 117 proposed an efficient method to backpropagate through implicit layers by imple- menting them Network Operator (V..). Here, A is defined sequential application of map F(:), which is guaranteed to converge in < oo iterations. Formally, (xx) can be represented as,
((;; θ) Fo Fo... (w;
T times
where & corresponds to parameters of potential multimodal model. The convenience of A in Equation 4 lies its ability to backpropagate through A without requiring its full sequence (or
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Traffic light
MLM
True
ITM
Multimodal Encoder
VQA
Yes
NEVR Image text ival
Nx; ① = Fo F o
F(x; +
Compute fixed point
O(x:0)
T
F(x: 6)
F(x: T-1
F(x; O
1=1
f
f =
4
8)
Feedforward
Vanilla backprop
lacobian-based backprop
Jacoblan-Free backprop
Caption: riding a bicycle down city street.
Visual Encoder
KG- Textual Encoder
Question: person crossing illegally or legally?
Figure 1: Our Proposed Architecture. The inputs are images and captions/questions. Image patches are processing by a visual encoder, while the caption, integrated with a knowledge graph, is passed to a language encoder. The extracted features are then fed into the OOD detection layer. This implicit layer identifies noise concepts before forwarding the features to the multimodal encoder. The ITM and MLM are training objectives, and VQA, NLVR, and image-text retrieval are downstream tasks.
trajectory) or solving an inverse problem with its Jacobian matrix for gradients. In Vision tasks, implicit maps like clustering with fixed parameters - for instance, N could be k-means with fixed means have been shown to enhance performance [57]. To enable small-scale custom applications or to fine-tune large-scale models, fixed parameters are insufficient, so we consider the problem of evaluating gradients with respect to the means efficiently.
Introducing "Benign" noise in Multimodal Pipelines. Recent advancements 65 have demon- strated that the combinations of multiple modalities improve understanding and boost generalization performance. However, these models, trained by a simple relation (matched or unmatched) between the image and text pair, often falter in capturing important semantic information and implicit de- pendencies within datasets. For example, as depicted in Figure [ these models may recognize the green light but fail to infer the legality of person crossing at the green light. Thus, the incorporation of "noisy" external knowledge presents a potential solution. this work, we explore the design of an implicit Network Operator A to manage feature-level noise during training. Specifically, / is employed to integrate external knowledge in multimodal learning, which is a deep network with parameters & developed to align multiple modalities 44 46).
Using External Knowledge for Reasoning Purposes. A knowledge graph collects information, organizes it in a specific structure, i.e., ontology, and is utilized to deduce novel insights and knowledge from the integrated information [6]. Often, external knowledge may be beneficial for reasoning tasks (22). Most recent knowledge-based multimodal models [22 36 53 focus on entity retrieval methods from external knowledge resources such as structural knowledge graphs and large language models. Unfortunately, these methods become impractical when combined with features derived from external knowledge resources that are prone to incompleteness and noise. Furthermore, lasion-language models like those in [18, 66) often struggle filter noise pairs during training, resulting in slow convergence. For instance, in Figure I given an object "strect" and relation "IocatedAt", noisy objects like "fire hydrant" are returned with high confidence by ConceptNet [55]. However, the "fire hydrant" absent the input image, which is undesirable for training objectives, such image-text matching.
We propose an implicul out-o--distributiin (OOD) detection layer to handle outliers from external knowledge as discussed above. We demonstrate how such a layer can be expressed as a network operator N Equation (T) for memory saving and efficient training. Specifically, the ongoing example in Figure I our proposed approach approximates the density of in-distribution features,

---

which are extracted from the triplets in the caption (question) corpus, by fixed point iterative layer. Thus, F trained with implicit differentiation at the fixed point according to OOD scores 44 for the retrieved knowledge triplets. Overall, we provide end-to-end training framework with knowledge graphs and multimodal models in an implicitly differentiable manner, as illustrated in Figure [] showcase the effectiveness of the proposed implicit layer based on N in large-scale multimodal pipelines, We conducted several experiments on multiple vision and language tasks, such as visual question answering and image text retrieval. We have included extensive evaluations and ablation studies to demonstrate the improvements in memory usage and training time compared to vanilla backpropagation.
We summarize our three primary contributions as follows: (1) Our proposed implicit layer can perform outlier detection tasks efficiently with less memory costs during backpropagation in practical settings. This layer can scale to different datasets and multimodal backhones. (2) We demonstrated that integrating explicit knowledge enhances multimodal fusion models. Our model learned from visual and textual modalities while using external knowledge seamlessly that may available in certain situations.
(3) Resulting model after training and/or fine-tuning, outperforms baseline models on various downstream tasks. Moreover, we provided various ablation studies and an interactive simulation based user study, which shows that users can sustain their desired in-distribution inputs or features while being able to update model parameters simultaneously.
2 Related Work
Deep implicit layers. Implicit differentiation has been an emerging and competitive alternative to explicit unrolling for backpropagation. From the practical point of view, it is often memory- efficient and numerically stable compared to vanilla unrolling 7 [55]. Recent works apply implicit functions and layers in various problems with deep neural network, such optimization layers 2, convolutional sparse modeling [33] and object representations (9. Moreover, implicit layers can be used to produce representations that are robust to perturbations,
well better empirical performance in downstream predictions. For example, 3 S used path independence as a technique to improve the OOD generalization. The main argument is that path independence can be guaranteed while training using implicit layers instead of unrolling which is crucial for our applications.
(OOD) detection. Out-of-distribution detection is usually studied from the statistical point of view, mainly under Robust Statistics (26). We will summarize recent developments focusing on our cases. For OOD detection, Maximum Mahalanobis Distance [29] and energy score (37] are statistical methods for detecting features that may not be obtained using training data. Recently, GEM
44 has introduced provable and competitive OOD detection method to estimate outliers for deep networks. GEM score can be used to recognize irrelevant distributions of concepts during training. The primary focus is the detection performance with simulated outliers, whereas our focus in this paper is on utilizing them in real-world noisy data for training purposes efficiently.
Vision-and- ann--anggaageeeee transformers and knowledge-basee VQA. Most recent vision-and-langee languuag models have shown improved performance by optimizing self-supervision based losses. For example, (5] introduce multimodal architectures wherein cross-modal features are used to learn combined representations of visual and textual contents that can be used to improve predictions. In Vision and Language Processing, knowledge representations in the form of extracted consensus or semantic concepts are aligned with visual concepts to construct concepts vocabulary 24 19 660. In knowledge-based VQA, different explicit knowledge bases be integrated for answering purposes In this work, we explore the effectiveness of utilizing external knowledge in deep multimodal models during training.
10
27
32
3
40 63
53 36
3 Implict OOD Detection Layer for Multimodal Analysis
Basic Notations. We denote input features as I; E R= 1,..., N including features of extracted knowledge triplets 1, E IR 1...., M. The goal of OOD layer to compute a score a(I,) E R,, for 1j to estimate density of in-distribution (ID) data. In this section, we explain how to estimate the distribution of the ID features as (normalized) linear combination of simpler distributions using an implicit layer to detect outliers and backpropagate for gradients, efficiently.
3

---

3.1 Finding Fixed Points for Forward Pass
Finite mixtures with components are conceptually simple, and computationally attractive. Math- ematically. they can produce accurate approximations to most density functions [43]. So, approximated the density of ID features using a Gaussian Mixture Model (GMM) and used Mi IRd o% Raxd l,..., K to denote the optimal means and covariance matrices of K components.
To obtain optimal GMM, we solved μ² and o² by using Expectation Maximization (EM) algorithm 45]. Our main observation that the update rule used in an EM-based algorithm of μgg on the current iterate & E T can be written as fixed point iteration as follows:
Σ d= exp( ((p )az
Pk
(2)
E exp( w (2f))
where the weight of current iterate 14 on D, denoted by w((() := Ek 10, 0.5 14) We updated a similarly using (x; -
100-(( (1; 1²)²² the numerator of Equation (2). Therefore, we obtained an approximation of in-distribution density that can be used for further outlier detection.
3.2
GEM score for Outlier Detection
After computing the optimal means μ* and covariances o² using the above described fixed point function, the final ingredient we need is a samplewise and memory efficient forward pass OOD detection method. After obtaining the parameters, we can compute a score for each 1,. the features derived from external knowledge triplet. In deep network context, the recently introduced GEM score 44 has already been tested on few classification applications. Specifically, we used GEM score to filter anomalous triplets. GEM score is also beneficial to obtain memory efficient gradients, details will be explained in Section 3.4 Given a derived feature from knowledge triple j E M, its GEM score s(1)) is defined using energy function as,
1 211 -1x) (1 11)) (-
s(1y) log
exp
Am
where 1ú is the textual feature of retrieved external triplets, 14 is the optimal means and og is the optimal covariance matrix of ID features.
Do we require fixed point iterations Equation 2 to exact? In our training pipeline, we only require that the gradients provided by the proposed OOD detection layer to be descent direction with respect to the loss as a function of network parameters. Thus, the only use of μ² and o* is to calculate OOD score in Equation (3). so the approximate μ²,,, o² may be sufficient enough for the training purpose. To see this, we considered a simplified setup in which language features in Equation
first processed by ReLU based upstream layer parameterized by Wup- followed by energy calculation using the output means 14% of EM algorithm.
Formally, we considered the loss function given by E(Wupi i n²) )= == log((xp((0 relu(W -t) μ (1l)) where we assumed number of components to be one i.e., K = 1 for simplicity. We now look the gradients computed by using the approximate output of EM algorithm. Now, the gradients of the energy score (Equation itself is given by Chain rule as follows:
V Wip E(W n²) a( 2(-log(exp(0.5 relu 1) - p² 1(1)))
owm up
-rellu((() n²)) C relu(sign((o)) 1"
(4)
where to := Wup 1, and @ denotes the Hadamard or Elementwise product. Impor- tantly, E(pp n²) is linear in μ²" By definition of a descent direction, it is possi- ble to reduce the loss using an approximate ji computed with finite iterations as long as tr (Vw, W. F(W (²" Ww, E(Wap; (i) < 0. Our calculation above considered only one upstream layer which is rarely the case in practice. We leave extensions to more upstream layers as future work.

---

Algorithm Fixed Point Network Operator based OOD Detection Layer for Language Features /
Input
avii [N],yj [M] p [K], of [
Output
Feature-wise OOD scores and Gradients
Begin Forward Pass
while T T do Update uk using Equation
end while
Set fi to be the last iterate of juf.
OOD Detection. Compute s(4) in Equation
Begin Backward Pass
Jacobian Free Backpropagation. Output gradient VI, (l,) ut. of)) by computing derivative
of composition of log-sum seg-sum-xpp and ReLU functions Equations using Chain rule.
3.3 Implementing Differentiable OOD layer in Multimodal Pipeline
To implement our implicit OOD detection layer discussed in previous sections in multimodal pipelines, we propose novel architecture called VK-OOD by fusing Vision and external Knowledge features. In essence, we used an OOD detector the network operator N and detected the retrieved knowledge triplets which can potentially lead slow convergence of upstream and/or downstream layers during training.
Architecture. Given an image with caption (question), our pipeline consists of the following steps: (1) Transform the image to visual features using the vision encoder, such as ViT-based [15]. (2) Retrieve knowledge triplets using external knowledge bases, and transform to textual features with the language encoder, such as BERT-based [14], more details of retrieval module are introduced in Appendix A (3) Approximate density of in in-distribution features, and filter outliers the image- triplet pairs with the proposed implicit OOD detection layer, (4) Finally, we learn vision and textual representations by multimodal encoder with multiple training objectives.
Image-text Matching Loss with GEM scores. We aimed to incorporate the calculated GEM scores the widely used image-text matching (ITM) loss. Specifically, we consider the model is encouraged to not only match images and texts correctly, but also to map OOD pairs farther away from ID pairs in the feature space. Given image-text pairs D {(u } based on s(-) is derived as:
the similarity of image-text pair
exp(m(Ug: L)/T r B sign(m(v,..4)) 44) s(l))
bg) E exp(m(V,)) (4) + B (m s(lw))
(5)
where m(-) is the cosine similarity of image-text pair, ,3 is learnable weight parameter, sign(-) is sign function. Then, the modified ITM loss is written
Cim DE (yttm p)u,))
(6)
where H. denotes the cross-entropy, Vitm corresponds to the image-text matching label. Note that, s(.) of the same triplets may vary on random masks and different tasks. Hence, we also modified other training objectives losses based on s(-), which are similar to Lim- See more details in
3.4 Efficient Backpropagation for OOD Detection Layer
Having found ugg og in the forward pass, as the EM algorithm. It outputs the optimal parameters of ID feature density given by n., a. However, EM viewed tricky since the Jacobian of such map will be daddd matrix, practically infeasible for training purposes even when d ze 100 is not very large. Hence, in this section, we describe how we adopted Jacoian-Free backpropagation for the implicit OOD detection layer.
view the initial part of our OOD network operator '
operator from Rdd -> Rd (mapping ly E I to Po Rd) makes backpropagation
Applying Chain Rule with Limited Unrolling for Gradients. Since each iteration in GMM as in Equation 2 is differentiable, can easily backpropagate through few iterations of EM algorithm
5

---

to compute gradients. First we used Chain rule to illustrate the necessary components to obtain gradients. Note that from Equation ( we know that v,L E R is scalar, and moreover for fixed Hr, itself is a smooth scalar function Sμ Rd + R. Thus, we consider the following composition of maps to compute v E Rd gradient with respect to z; (denoted generically as 2:),
L
Rd
R
R R.
where j²" is the output of our OOD layer from forward pass. The key difficulty is in approximating the Jacobian Jz.u- E Rd and/or its inverse for backpropagation since it maybe dense. We now describe three standard ways that can be used to backpropagate through implicit layers. First, in the vanilla backpropagation often called as "unrolling" is used where each iterate stored, and a "path" gradient approximation is calculated. That is, since the parmeters of each unrolled layer fixed, we can simply use them along with the closed form gradients available for Equation (2) for backpropagation. Second, Jacobian based methods form approximation to the Jacobian inverse in other words, solve a linear system formed using the fixed point condition in Equation (2) itself. Often, this approach yields better gradients. Third, Jacobian- Backpropagation (JFB) combines both these approaches, that is, in which we simply use or unroll the last few iterates of the EM algorithm for gradients. As can see, JFB enables train with fixed memory costs (1) and efficient backpropagation without computing gradients at each iterations. The time complexity is (n) for forward and (1) for backprop. We provide empirical results of the above three methods in Section
Benefits of EM algorithm. EM updates are provably convergent in various settings, and it takes few iterations when it is guaranteed to converge [13]. This also implies that we can simply initialize M4 randomly and perform few more iterations to find a fixed point. However, since the denominator in Equation ( contains terms that are also unknown parameters, such update schemes may be numerically unstable. In such cases, we can simply use recently proposed gradient based EM algorithms as the network operator, for fine-tuning deep networks in large scale settings, see [51 for convergence analysis.
Interaction via Outlier Detection. The applications which require a high number of image patches (or concepts), the likelihood that one of the patch features or text features to be an outlier also increases dramatically. In high dimensional settings, this increases the training time taken by first order methods significantly, especially when mini-batches are used to compute gradients 20) Alternatively, when features 1, are computationally easy to extract, say using CLIP [49], it is reasonable to expect that a certain fraction of the 1j are outliers, and should not be used for backpropagation purposes. In more optimistic scenario, we consider to customize our predictions, and handle "on-the-fly" fly" integration of explicit knowledge. In our framework, this corresponds to treating yu in Equation (3) as trainable parameters. We can update the initialization μ4 without storing the trajectory, or forming the full Jacobian which can be expensive, as in our Algorithm
4 Experiments
In this section, we present empirical results addressing the following questions: (1) How effective and robust is the OOD detection layer? (2) What improvement does the Jacobian-Fres Free backpropa- gation (JFB) provide in computational cost within our VK-OOD architecture? (3) How do external knowledge resources influence the performance of multimodal pipelines? (4) Can VK-OOD work with different backbones in large-scale settings for various downstream tasks? To answer (1) and (2), we performed experiments on VQA task with various setups, comparing memory and training costs with unrolled EM (vanilla), Jacobian seaobian-based-Eene based-EM, and JFB-EM. To answer (3), we evaluated the influences of knowledge resources by explicitly (or intentionally) introducing outliers in the pipeline. Finally, for (4) we utilized ViLT [27), CLIP [49, and BLIP [1 as the backbone model and illustrate performance downstream tasks, including natural language for visual reasoning and image-tex
retrieval. More implementation and training details are described in the Appendix C.T For ablation studies, we fixed the backbone model to be ViLT with pretrained parameters.
4.1 Results and analysis of OOD detection layer
Backprop methods. We used different backpropgation methods OOD detection layer with ViLT as the backbone. As shown in Table I
compared to vanilla and Jacobian-based methods. JFB-EM

---

Time Time(m)e
Method
#Param(M)
#FLOPs(G)
Max Mem(Mb)
VQAv2
Forward
Backward
Vanilla-EM
152.6
185.2
39.6
26.8
18673
76.6
JB-EM
125.2
115.7
39.6
12.7
14512
76.6
JFB-EM
124.8
108.6
39.6
6.3
13674
76.8
Table 1: Experimental results of different backpropagation method in the dense OOD detection layer.
JFB-EM much more efficient in backward pass and use less memory. It also outperforms on the
VQAv2 task in terms of accuracy.
k=1
k=2
k=3
k=4
k=5
Figure 2: Visualization of the multimodal feature space. Here, k denotes the number of clusters.
outperformed them in term accuracy on VQAv2 task. is trained with significantly less memory and time cost per epoch, which is beneficial for large-scale settings. Additionally, we conducted experiments T' number of iterations, which critical parameter compute fixed points. The results are shown in Figure 3a We found the improvements to be marginal for T 2 5. and JFB achieves faster performance on all settings. Based on the results in Figure 3a we fixed T 10 for the subsequent experimental settings. Additionally, we have provided the fixed point error plot over iterations Figure 3b] We observed that the squared euclidean distance between successive iterates indeed went to zero showing convergence of forward pass.
Number of Clusters. We investigated the effects of the number of clusters optimizing the GMM process, and the results are presented in Figure (see green line). The general trend indicates that performance improves with an increase in the number of clusters. Qualitatively, Figure shows that learned output features using U-MAP embeddings 42] Feature embedding spaces of multiple modalities COCO val dataset are shown different colors representing different clusters. We provide example images of the clusters in Appendix C33 The results show that our VK-OOD model can identify clusters over the extracted multimodal features our layer can accurately detect outliers.
Robustness of OOD layer. Usually OOD features are not present within the training datasets themselves. However, may encounter outliers when integrating external knowledge triplets into the training pipeline. corresponds to the ID setup no outliers. To delve into further quantitative analysis, we considered two setups: one with M = 0, i.e., ID setup, and OOD setup with M 5 (so possibly 5 outliers per language caption) that corresponds from the results in Table 26 that there is not a significant difference from the the rate of convergence perspective as indicated by squared norm of successive iterates I|mr M 1 in both setups. However, from the accuracy (Acc) column in Table 2b we can conclude that the performance in VQA tasks has significant improvements over iterations when considering external knowledge.
we denote M as the number of external knowledge triplets, then M =
augmenting features from external knowledge. We can see
61-
0/1
Tocle
Nasbe ThipeeChaiiss
(a)
(b)
Figure 3: Ablation studies results on VQA task. (a) Results on accuracy with different T iterations in
EM on VQAv2 dataset. (b) || - M++1l over EM iterations in forward (ass.(c) Results on accuracy with different numbers of external triplets and the number of clusters in the GMM process (see Equation 2 on OKVQA dataset.
7

---

Incomplete Knowledge Triplets. To evaluate the sensitivity of models to OOD detection perfor- mance, we conducted experiments of incomplete knowledge triplets with missing values. Note that, we only dropped language features inputs due to computational reasons augmenting image patches requires more resources such as GPUs. However, since used for both visual and language features, the implementation remains the same as that of dropping patches in language features. So, it equivalent to drop either the visual or the language feature since the computational effort involved in running EM algorithm depends only the total number of features. We now present more results of lite - - ll and OKQQA performance in term of accuracy over optimization iterations in Table With higher level of incompleteness, the rate of convergence is slower as expected. Once again,
2a
as
in previous robustness experiments, we found accuracy gain over iterations here also.
25%
50%
75%
ID(--0)
OOD(M=5)
T
Err
Acc
Err
Acc
Err
Acc
T
Err
Acc
Err
Acc
1
2.213
48.2
2.448
47.3
2.735
44.6
1
1.94
73.6
2.15
73.1
3
0.174
48.9
0.214
47.7
0.208
45.1
3
0.059
73.8
0.089
74.8
5
0.065
50.6
0.092
48.6
0.244
46.0
5
0.038
73.9
0.051
76.1
8 0.057
51.2
0.108
49.1
0.112
46.6
8
0.042
73.9
0.036
76.5
10
0.051
51.8
0.084
49.4
0.167
46.9
10
0.036
74.1
0.034
76.8
(a) ja error and OKVQA accuracy
(b) error and VQA accuracy
Table 2: Fixed point error is measured using consecutive iterate distance ||/4 - 1 1| (a) μ error and OKVQA accuracy over the EM optimization iterations in different level of incompleteness. (b) error and VQA accuracy over the EM optimization iterations in ID and OOD setups.
4.2 Ablation study VK-OOD Components and External Knowledge
Effectiveness of Each Component. To compare the impact of the proposed OOD layer in VK-OOD, we considered different combinations of inclusion and exclusion knowledge graph representations (KG) and OOD detection layer. The results are shown in Table 3 The results show that our model achieves the best performance when both the components are included in the model. Moreover, comparing the results on VQAV2 and VKVQA datasets, the results imply that external knowledge triplets (KG) can be beneficial to improve the performance especially on VQA task, Furthermore, using OOD layer even when there are external knowledge triplets has good performance. This shows that including OOD layer in our model is helpful and able to capture the noise of multiple modalities, such missing or mismatching modalities.
Numbers of external knowledge triplets. We conducted experiments to explore the impact of the amount of the knowledge triplets. We evaluate this on VQA tasks using OKVQA dataset. Figure shows the experimental results. As expected, we observed that increasing the number of retrieved knowledge triplets improve the accuracy of pre- dicted answers. We achieved the best accuracy of 52.4% when the number of triplets is R 5.
Method KG OOD
Downstream tasks
VQAV2
OKVQA
73.9
45.5
74.6
48.3
74.1
46.2
76.8
52.4
Table Ablation studies of different components of VK-OOD. "KG" and "OOD" denote knowledge graph and OOD detection layer respectively.
Knowledge resources. We evaluated different knowledge resources, i.e., different embeddings of implicit and/or explicit knowledge. Table 4 shows benefits provided by different external resources in our pipeline. We queried knowledge from Wikidata [59 and used BERT to get embeddings 1, E Rdd. Our model produced 1.8% and 18.7% more accurate results than the best and worst performing baselines respectively. Moreover, using ConcepNet embeddings solely, our multimodal training pipeline also learn implicit knowledge in the multimodal fusion encoder. We compared using external knowledge resources. As results shown in Table 4 all the different external knowledge resources brought improvements in the OKVQA tasks. Our proposed model takes advantages of implicit knowledge from languae models and integrates explicit knowledge prior information, thus outperforms other models using external knowledge resources without OOD detection.
model performance with the models
4.3 Scalability of VK-OOD
In this section, we incorporated the proposed OOD detection layer to various backbone models with different architectures and model sizes. We initialized the parameters (μ and σ) of our proposed
8

---

OOD detection layer in two different ways, scalar a or the dense one which is a d X d matrix, where d is the dimension of imput embeddings. As showin Table the number of parameters in scalar VK-OOD are approximately similar to the baselines. Specifically, comparing to other baseline models, while our scalar VK-OOD increases the # #-parameters slightly - 0.4 million more parameters (since d ~ 700) significantly improved the performance in downstream tasks.
We then conducted experiments on sev- eral downstream tasks. In all these experiments, our model consistently achieved the best and second-best per- formance compared to five state-of- the-art (SOTA) vision-language models across three downstream tasks on vari- ous datasets(see T). We trained on open-source data and compare the results with the models having similar number of parameters. tasks, we evaluated on the VQAv2 test set. As shown in Table B our model outperformed all the baselines dataset, yielding an accuracy of 77.9%. Furthermore, in the Natural Language for Visual Reasoning (NLVR) task, VK-OOD achieved the best and second-best result terms of accuracy with different backbone models, respectively. We also evaluated model, along with the baseline models, the test of COCO and F30K dataset for image-text retrieval task. Our model produced the best performance and outperformed the best and worst-performing baselines, with the exception of BLIP, by up 0.5% and 11.6% respectively on the COCO dataset (Table 5 In other settings and on the F30K dataset, the results are similar. Further details are provided in Appendix C.2
Method
Knolwedge resources
OKVQA
ConceptBERT
CN
33.7
KRISP
Wiki CN
38.4
MAVEX
Wiki GI
39.4
KAT-B
Wiki GPT3
50.6
UnifER
CN + ViLT
42.1
CN w/o BERT
51.1
Wiki
51.9
VK-OOD (Ours)
Wiki + CN
52.2
CN
52.4
particular, for VQA
Table 4: Results on the different knowledge resources on OKVQA dataset. "CN" and "GI" denote ConceptNet and Google Images respectively.
this
The performance of our model demonstrates its ability for visual reasoning while integrating both implicit and explicit knowledge. Moreover, layer can scale with different models and contribute improvements to various tasks. For inference, since js and inference time for one VQA instance is around 57 ms on single 2080Ti GPU. Therefore, we believe that our implicit layer function plug-and-pla, module and can be easily integrated with other vision-language models.
able to show that the proposed OOD detection
fixed, the inference time is similar to the backbone models considered the average
4.4 Qualitative Analysis
Visualizing Attention with Multimodal Information. Figure is example of multimodal alignment results from our VK-OOD representations. We use Grad-cam [52] to visualize the multimodal maps of the models on the image corresponding to knowledge triplets. As shown in
COCO
Flickr30k
Model
#Params
VQAv2
NLVR2
TR R@5
IR R@S
TR R@5
IR R
ViLT
87
70.3
74.6
86.2
72
95.6
86.8
UNITER
155
72.7
75.8
87.4
78.5
97.1
92.4
ALBEF
314
74.5
80.5
91.4
81.5
99.4
96.7
VinVL
157
75.9
83.1
92.6
83.2
BLIP*
346
77.5
82.8
95.2
85.4
99.8
97.5
VK-OOD-
87.4
76.7
84.3
90.9
81.6
97
94.3
CKLOOD-sCLLIP)
113.4
76.2
83.8
92.8
83.4
99.6
96.7
VK-OOD-s(BLIP)
346.4
77.8
84.1
95.4
85.2
99.8
97.2
VK-OOD-I(ViLT))
125
76.8
84.6
91.7
81.3
97.2
94.5
VK-OOD-I(CLIP)
151
76.1
83.9
93.1
83.6
99.6
96.8
VK-OOD-IBLLIP
412
77.9
84.5
95.1
84.8
99.6
97.1
Table Overall performance on multiple downstream tasks. We demonstrate VK-OOD scale with different model backbones and achieve the best and second-best results. VK-OOD-s is the scalar case, and VK-OOD-1 the dense case. *our implementation.

---

Original
man riling bicycle Car locales on abrect
Bus locales strect Traffic lights locase sireet Buikdings locate atroet
Figure 4: Visualization of the attention maps of image and triplets alignment. The original sample caption "A man riding a bicycle down a city street". We highlight areas in the image corresponding to different knowledge triplets. Our model learns different objects and localize those objects correctly.
Caprlons:
Cantoos:: loying an hedge
will
doestiic
apples.
thic canyos vole thoe?
Wh sho beatn
Rrewingye rellee spain soott, CCD0
Kooweeder Segeete:
Aonaloage Hobeet: Hobile, hanee
unad cantrutte .010
Croud (nath clettrctt Mide ralo
Drea vetrette
Cround Pruh cote Bardiing fire
Figure 5: Example case studies with OKVQA dataset. We show the examples of retrieved knowledge triplets. Our model is able to detect outliers of retrieved triplets shown in orange. The predicted answers are finetuning OKVQA dataset. Comparing with the baseline results, our model provides more correct answers. Note that, the baseline model here is our implementation without KG and OOD components.
Figure 4 our model has the capability to attend to the extracted knowledge concepts, such as buildings and traffic lights. Thus, our model can detect more objects to provide the ability for answering open questions. We discuss more user studies on interactive OOD detections by feeding in domain knowledge with different distributions in the Appendix
Furthermore, we present prediction results in Figure 5 with VK-OOD model on OKVQA dataset along with the extracted knowledge triplets based the captions. In the example, the caption (apple, used for, making apple pic) is useful to obtain correct answers. This observation validated that explicit knowledge provides more reasoning capability than implicit knowledge. Moreover, our model detected OOD triplets by combining modalities, i.c, the apple fruit is in image, thus is not used for computing. The last one is a failure case, because the ground truth caption not sufficiently specific. Therefore, it might be beneficial to consider more inference and reasoning abilities in multimodal analysis, such as chain of thoughts (62.
5 Discussion
Limitations. A potential limitation of our proposed implicit layer arises when the covariance matrix is dense. In such cases, a fast linear system solver would be required to evaluate the likelihood. We will consider to explore sparse approaches to further saccelerater OOD detection layer the future. Additionally, given that we have demonstrated that explicit knowledge can serve as supervision in lision-language training, we believe that various knowledge bases, such as medical knowledge graphs, can provide user-desired domain distributions.
Conclusions. In this work, we presented training framework designed facilitate multimodal analysis under distribution shifts and/or the presence of outlier distributions within the input feature space. Several other models have been proposed exploit special structures in the available modalities for faster training purposes, as mentioned in [61]. and for tasks involving egocentric vision [64]. While the approaches have been shown perform well in large-scale settings, they may not be sufficient on their own. For instance, most frames in a video have low semantic information content and may require complex processing pipelines [ We argued that handling outliers within the context of multimodal analysis is an crucial topic as more models are integrated or fused. Our implicit OOD detection layer can be directly instantiated within such complex pipelines. possibly allowing us to intervene and accelerate the training process while reducing computation costs. We demonstrated through extensive empirical analysis across various setups that incorporating OOD detection in the training pipeline can significantly enhance the performance on downstream tasks.
10

---

References
[1] J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, al. Flamingo: visual language model for few-shot learning. Advances in Neural Information Processing Systems, 12-2366,,,7,,, 2022.
[2] B. Amos and J. Z. Kolter. Optnet: Differentiable optimization as layer in neural networks, In International Conference on Machine Learning, pages 136-145. PMLR, 2017.
(3] C. Anil, A. Pokle, Liang, J. Treutlein, Wu, S. Bai, Z. Kolter, and R. B. Grosse. Path independent equilibrium models can better exploit test-time computation. Advances in Neural Information Processing Systems, 35:7796-7809, 2022.
[4] S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference computer vision, pages 2425-2433 2015.
[5] S. Bai, Z. Geng. Y. Savani, and J. Z. Kolter. Deep equilibrium optical flow estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 620-630 2022.
[6] S. Bai, J. Z. Kolter, and V. Koltun. Deep equilibrium models. Advances in Neural Information Processing Systems, 32, 2019.
[7] M. Blondel, Q. Berthet, M. Cuturi, R. Frostig, S. Hoyer, F. Llinares-López, F. Pedregosa, and J.-P. Vert. Efficient and modular implicit differentiation. arXiv preprint 105. 2021.
[8] A. Cavallaro, O. Steiger, and T. Ebrahimi. Semantic video analysis for adaptive content delivery and automatic description. JEEE Transactions on Circuits and Systems for Video Technology. 1 15(00)1200-12000 2005.
[9] M. Chang, T. Griffiths, and S. Levine. Object representations fixed points: Training iterative refinement algorithms with implicit diffcrentiation. Advances in Neural Information Processing Systems, 35:32694-32708, 2022.
[10] Y.-C. Chen, L. Li, I.. Yu, A. El Kholy, F. Ahmed, Z. Gan, Y. Cheng, and J. Liu. Uniter: Universal image-text representation learning. In European conference on computer vision, pages 104-120. Springer, 2020.
[11] Z. Chen, Q. Zhou, Shen, Hong, H. Zhang, and C. Gan. See, think, confirm: Interactive prompting between vision and language models for knowledge-basee visual reasoning. arXiv preprint arXiv:2301 005226, 2023.
[12] E. D. Cubuk, B. Zoph, J. Shlens, and Q. Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pages 702-703, 2020.
[13] C. Daskalakis, C. Tzamos, and M. Zampetakis. Ten steps of em suffice for mixtures of two gaussians. In Conference on Learning Theory, pages 704-710. PMLR, 2017.
[14] Devlin, M.-W. Chang, K. Lee, and Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint
2018.
[15] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 1 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929,, 2020.
[16] L. Ehrlinger and W. Wöß. Towards definition of knowledge graphs. SEMANTICS (Posters, Demos, SuCCESS), 48(1-4):2, 2016.
[17] S. W. Fung, H. Heaton, Q. Li, D. McKenzie, S. Osher, and W. Yin. Jfb: Jacobian-free backpropagation for implicit networks. In Proceedings of the AAAI Conference on Artificial 2022.
Intelligence, volume 36, pages

---

[18] Z. Gan, Y.-C. Chen, L. Li, C. Zhu. Y. Cheng, and J. Liu. Large-scale adversarial training for aasion-and-languaage representation learning. Advances in Neural Information Processing Systems, $3:6616-6666 2020.
[19] F. Gardéres, M. Ziaeefard, B. Abeloos, and F. Lecue. Conceptbert: Concept- representation for visual question answering. In Findings of the Association for Computational Linguisties: EMNLP 2020, pages 489 4988, 2020.
[20] J. Geiping, M. Goldblum, P. E. Pope, M. Moeller, and T. Goldstein. Stochastic training is not necessary for generalization. arXiv preprint arXiv:2109. 14119, 2021.
[21] D. Gilton, G. Ongie, and R. Willett. Deep equilibrium architectures for inverse problems in imaging. IEEE Transactions on Computational Imaging, 7:1 1:1123-1133, 2021.
[22] L. Gui, B. Wang, Q. Huang, A. Hauptmann, Y. Bisk, and J. Gao. Kat: A knowledge augmented transformer for vision-and-lange ann-langu In NAACL, 2022.
[23] Y. Guo, L. Nie, Y. Wong. Y. Liu, Z. Cheng, and M. Kankanballi. A unified end-to-end retriever- reader framework for knowledge-baat vqa. In Proceedings of the 30th ACM International Conference Multimedia, pages 2061-2069, 2022.
[24] Y. Huang. Q. Wu, C. Song, and L. Wang. Learning semantic concepts and order for image and sentence matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6163-6171, 2018
[25] Z. Huang, S. Bai, and J. Z. Kolter. (1ttextrm(Implicit))) representations. In A. Beygelzimer, Y. Dauphin, P. Liang, and W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
Implicit layers for implicit
[26] P. J. Huber. Robust statistics. In International encyclopedia of statistical science, pages 1248-1251. Springer, 2011.
[27] W. Kim, Son, and I. Kim. Vilt: fision-and-langusee transformer without convolution or region supervision. In International Conference on Machine Learning, pages 5583-5594. PMLR, 2021.
[28] R. Krishna, Zhu, O. Groth, J. Johnson, Hata, Kravitz, S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123(1):32-73, 2017.
[29] K. Lee, K. Lee, H. Lee, and Shin. A simple unified framework for detecting -ittr--distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
[30] G. Li, M. Müller, B. Ghanem, and V. Koltun. Training graph neural networks with 1000 layers. In International conference on machine learning, pages 6437-6449. PMLR, 2021.
[31] J. Li, D. Li, C. Xiong, and S. Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, pages 12888-12900. PMLR, 2022.
[32] J. Li, R. Selvaraju, A. Gotmare, S. Joty, C. Xiong, and S. C. H. Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in Neural Information Processing Systems, 34, 2021.
[33] M. Li, P. Zhai, S. Tong, X. Gao, S.-L. Huang, Z. Zhu, C. You, Y. Ma, et al. Revisiting sparse convolutional model for visual recognition. Advances in Neural Information Processing Systems, 11:10492-10004 2022.
[34] Z. Li, Z. Chen, F. Yang. W. Li, Y. Zhu, C. Zhao, R. Deng, L. Wu, R. Zhao, M. Tang, et al. Mst: Masked self-supervised transformer for visual representation. Advances in Neural Information Processing Systems, 34:13165-13176 2021.
[35] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014.
12

---

[36] Y. Lin, Xie, D. Chen, Y. Xu, C. Zhu, and L. Yuan. Revive: Regional visual representation matters in knowledge-baad based visual question answering. arXiv preprint arXiv:2206.01201, 2022.
[37] W. Liu, X. Wang, J. Owens, and Y. Li. Energy- out-of-distribution detection. Advances in neural information processing systems, 11221444-14777 2020.
[38] Y. Liu, M. Out N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zeulemoyer, and V. Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11622 2019.
[39] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. International Conference on Learning Representations, 2018.
[40] K. Marino, X. Chen, D. Parikh, A. Gupta, and M. Rohrbach. Krisp: Integrating implicit and symbolic knowledge for open-domain knowledge-base based vqa. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 14111-14121 2021.
[41] K. Marino, M. Rastegari, A. Farhadi, and Mottaghi. Ok-vqa: visual question answering benchmark requiring external knowledge. In Conference Computer Vision and Pattern Recognition (CVPR), 2019.
[42] L. McInnes, J. Healy, N. Saul, and L. Grossberger. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018
[43] G. McLachlan, S. X. Lee, and S. I. Rathnayake. Finite mixture models. Annual review of statistics and its application, 6:355-378, 2019.
[44] P. Morteza and Y. Li. Provable guarantees for understanding out-of-distribution detection. Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
[45] K. P. Murphy. Probabilistic machine learning: an introduction. 2022.
[46] J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng. Multimodal deep learning. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 689-696, 2011.
[47] V. Ordonez, G. Kulkarni, and T. Berg. Im2text: Describing images using I million captioned photographs. Advances in neural information processing systems, 24, 2011.
[48] B. A. Plummer, L. Wang. C. M. Cervantes, J. C. Caicedo, J. Hockenmaier, and S. Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, pages 2641-2649, 2015.
[49] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748-8763. PMLR, 2021.
[50] A. Rajeswaran, C. Finn, S. M. Kakade, and S. Levine. Meta-learning with implicit gradients. Advances in neural information processing systems, 32, 2019.
[51] N. Segol and B. Nadler. Improved convergence guarantees for learning gaussian mixture models by em and gradient em. Electronic journal of statistics, 15(2):5100 4544 2021.
[52] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual explanations from deep networks via gradient- localization. In Proceedings of the IEEE international conference computer vision, pages 618 8--66,,222 2017.
[53] Z. Shao, Z. Yu, M. Wang, and J. Yu. Prompting large language models with answer heuristics for knowledge-based visual question answering. arXiv preprint (rXiv:2303.01900 2023.
[54] A. Singh, R. Hu, V. Goswami, G. Couairon, W. Galuba, M. Rohrbach, and D. Kiela. Flava: A foundational language and vision alignment model. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 15638-15651 2022.
13

---

[55] R. Speer, J. Chin, and C. Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In Thirty-first AAAI conference on artificial intelligence, 2017.
[56] A. Suhr, S. Zhou, A. Zhang, I. Zhang, H. Bai, and Y. Artzi. corpus for reasoning about natural language grounded in photographs. arXiv preprint
http:/811.004990 2018.
[57] J. Sun, Y. Li, H.-S. Fang, and C. Lu. Three steps to multimodal trajectory prediction: Modal- ity clustering, classification and synthesis. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 13250-13259, 2021.
[58] A. Vaswani, N. Shazeer, N. Parmar, Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and
I. Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008, 2017.
[59] D. Vrandečić and M. Krötzsch. Wikidata: free collaborative knowledgebase. Communications of the ACM, 57(10))78-88) 85, 2014.
[60] H. Wang, Y. Zhang, Z. Ji, Y. Pang, and L. Ma. Consensus- aware visual-semantic embedding for image-text matching. In European Conference on Computer Vision, pages 18-34. Springer, 2020.
[61] Z. Wang, J. Yu, A. W. Yu, Z. Dai, Tsvetkov, and Y. Cao. Simvlm: Simple visual language model pretraining with weak supervision. arXiv preprint arXiv:2108. 10904, 2021.
[62] J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning large language models. arXiv preprint arXiv:2201.11903, 2022.
[63] Wu, Lu, A. Sabharwal, and R. Mottaghi. Multi-modal answer validation for knowledge- based vqa. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages
2712-2721, 2022.
[64] A. Zeng, A. Wong, S. Welker, K. Choromanski, Tombari, A. Purohit, M. Ryoo, V. Sindhwani, J. Lee, V. Vanhoucke, et al. Socratic models: Composing zero-shot multimodal reasoning with language. arXiv preprint arXiv:2204.00599 2022.
[65] C. Zhang, Z. Yang, X. He, and L.. Deng. Multimodal intelligence: Representation learning, information fusion, and applications. IEEE Journal of Selected Topics in Signal Processing, 14(3):478-44) 493, 2020.
[66] P. Zhang, X. Li, X. Hu, J. Yang, L. Zhang, L. Wang, Y. Choi, and J. Gao. Vinvl: Revisiting visual representations in vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5579-5588, 2021.
14

---

A More details of Multimodal Pipeline
A.1 Image Encoder
We extracted features from image patches and used vision transformer based models(ViTs) [58] as our encoder for the patches. To do so, we split input image into a sequence of patches and project them using linear maps to obtain embedding of patch features lgg, which simplifies the step for fusing with text embedding. Following the vision- -langugg models, we trained our model with multiple popular ViTs to examine the influence of image encoder in OOD detection backpropagation process.
A.2 Knowledge Retrieval Module
Given the caption S, we first parsed it into triplets in the form of (o(c). o'))) where o(c) and d'(c) are concepts Old ,nd r(c) is the relation(s) between them, i.e., (man, riding, bicycle). In our example, the seed triplets (ID triplets) parsed from the caption are (man, riding, bicycle) and (bicycle, down, street). Then we augmented knowledge by matching these triplets with external open knowledge including domain and commonsense knowledge graphs, i.e., ConceptNet (55]. ConceptNet provides large scale commonsense knowledge with over 21 million edges by 36 type of relations, i.c., IsA, UsedFor, AtLocation, connecting 8 million nodes. To complete our knowledge graph based augmentation, we collected concepts by querying from ConceptNet using o(c), (cc and rel, where [0, 36] and concatenated them to seed triplets. For example, given "street" as o(c) and "AtLocction"" as rel,, we will extract the related concepts are located at street to form triple T; Specifically, we queried explicit knowledge triplets of o(c) and o' (c)) from ConceptNet to form T
i.e. (bicycle, used for, transport). Finally, these knowledge triplets E T = Tep U are encoded as language features, such as l, using language encoder (i.e., BERT 14
A.3 Multimodal Fusion Encoder
We used our proposed OOD detection layer as a "plug-and-play module that can be used in different vision-language architectures. In ViLT and CLIP, we added the visual and textual embeddings with x(.) as in Equation (3). and passed them to a standard L-depth transformer. While using the BLIP backbones, we implemented two-stream transformer pipeline consisting of stacked multiple layers to joint vision and knowledge textual representations. For each layer, we used self-attention unit and merged cross-ates coos-atttntion unit, thus integrating vision and knowledge semantic information and the alignments across them.
Multimodal Attention Module. As in the standard transformer architecture 58 the attention function computes identical learnable parameters (weights) as in Equation where d is the dimension of the inputs, a query Q, key K, and value V. We used fusion encoder to compare similarity among the image-text pairs given by,
(8) and Equation P
Attn (Q1, KL. VL) QIKT Vd
(8)
softmax
VVE
and
Attn (QI, Kr, V1)
softmax Q2K?
(9)
where 1 and L correspond to image modality and language modality.
In implementation of multimodal encoder, we updated the image and language embedding outputs themselves from previous layer as queries and concatenate them together to get keys and values. As further improvement of attention function, we used multi-head attention which is composed by multiple paralleled attention function in each head. The feed-forward layers transform the outputs of multi-head attention through two fully-connected layers with GeLU activation.
15

---

B More details on formulating Training objectives
We provide more details on our training objectives in our pipeline in this section, including image text matching (ITM) and masked language modeling (MLM).
B.1 Image Text Matching (TTM)
To incorporate both the vision and the language representations, we used ITM which is widely used in previous VL studies. Given image and text of triple pair (vy, 1y), ITM predicts whether they matched as positive examples or not, and it a binary classification problem with the loss function in Equation (5) and Equation (6). We assumed that each image and ID triple pair (Ugs 4j), as positive example. The negative pairs are constructed through batch-sampling.
B.2 Masked Language Modeling (MLM)
MLM utilizes vision features and text features of ID concepts and relations to predict the masked tokens in the caption sentence S. Masking tokens is form of self supervurreisionn and is well known to improve performance [4]. Here, we randomly masked some tokens in S replacing as y m* and predicted them with their visual and textual features. Since some tokens are replaced with "[mask]", the OOD score s(.) becomes function of the random masks. With this, we used s(-) to calculate the predicted probability for masked token as in Equation 33. Our final MLM loss can be written as,
mmim =
- H p(u,
(10)
where H. denotes the cross-entropy, Ymaim is a one-hot vector where the ground truth tokens are with probabilities of 1, denotes the masked text.
C Additional Experiments
In this section, we provide details experimental setups that we used to conduct ablation studies, and more qualitative analysis of our proposed VK-OOD multimodal pipeline.
C.1 Implementation details
Datasets. Following standard practice in Vision, used training strategies wherein pre-trained on a fixed, large dataset and then fine-tuned on datasets specific to downstream tasks. We pre-trained on three datasets, including COCO [35). Visual Genome (28), and SBU Captions 447 with total of 1M images and 6.8M image-caption pairs, as approximate 30% less than the baseline ViLT). Each caption is parsed to 1 3 triplets and augmented with 5 external knowledge triplets. For downstream datasts, we used Flickr30k 48 and COCO for image-text retrieval, VQAv2 4 and OK VKVQA [41] for visual question answering (and ablation studies), and NLVR2 (56) for visual reasoning. We resized each image to the size of 224 x 224 by center-cropping. In the merged attention module, each multimodal encoder layer consists of one multi-head self-attentio block and one feedforward block. and total number of identical layers is equal to 12.
Encoder backbones. First, retrieved explicit knowledge triplets pre-processing, by using ConceptNet Numberbatct]] Next, for ViLT, we used RoBERTa (38] as text encoder and ViT-B/32 44 as visual encoder. To scale with CLIP, we used CLIP-ViT-B/22 (49 as both backbones. Then, we followed the BLIP design with BERT-base as text encoder and ViT-B/16 as visual encoder.
Network training. We pre-trained the model for 10 epochs using Adam W optimizer (39] with learning rate of 1e - 4 and weight decay of - 2. We chose the warm-up phase of learning rate to be 10% of the total training steps, and the learning rate decayed lincarly to 0 afterwards. Then, fine-tuned our model for epochs with learning rate of 2e for all downstream tasks. In addition, applied CandAugment [12] augmentation strategy in fine-tuning steps. We pre-trained and fine-tuned both on 8 NVIDIA RTX 2080Ti GPUs, and for inference we used 1 NVIDIA RTX 2080Ti GPU.
16

---

COCO
F30K
Model
TR
IR
TR R@5
TR
R@1 1
R@5
R@ 10
R@1
R@5
R@ 10
R@1
R@ 10
R@1 1
R@5 5
R@ 10
ViLT
61.8
86.2
92.6
41.3
72.0
82.5
81.4
95.6
97.6
61.9
86.8
92.8
UNITER
64.4
87.4
93.1
50.3
78.5
87.2
85.9
97.1
98.8
72.5
92.4
96.1
ALBEF
73.1
91.4
96.0
56.8
81.5
89.2
94.3
99.4
99.8
82.8
96.7
98.4
VinVL
74.6
92.6
96.3
58.1
83.2
90.1
BLIP
80.6
97.6
63.1
85.3
91.1
96.6
99.8
87.2
98.8
Ourst ViL.T)
73.8
91.4
96
52.4
81.3
90.1
85.9
97.1
97.6
80.1
94.6
96.7
CLIIP
69.8
93.6
48.8
78.5
82.5
92.3
98.4
99.5
79.8
92.1
96.4
Ours(BLIP)
80.7
96.8
96.4
99.6
86.3
98.8
Table 6: Detailed results of image-text retrieval tasks on COCO and Flickr30k datasets. Our model
with different backbones outperforms other models and achieve the best and second-best results.
VQA
Flickr30k TR@1
Model
Objectives
test-dev
IR@1
ViLT
ITM
70.6
82.1
65.6
ViLT
MLM
72.8
ViLT
ITM+MLM
74.2
88.1
74.1
VK-OOD-S
ITM
72.1
84.5
69.8
VK-OOD-s
MLM
73.4
VK-OOD-S
ITM+MMMM
74.8
89.0
77.2
Table Ablation study experiment results of VK-OOD model. ViLT is our implementation without explicit knowledge and OOD detection layer. ITM is image-text matching, and MLM is masked language modeling. Results on VQA are on test-dev set. Both downstream results are in zero-shot settings. The bold values mean the best model in the table. Comparing with the baselines, our model with OOD detection layer outperforms on all objectives with two datasets. Training on combinations of objectives improves model performance.
C.2 Experimental Results
Details of image-text retrieval tasks. We now discuss detailed results on COCO and F30K datasets, as shown in Table 6 We can see that the model of OOD detection layer with ViLT has significant improvements in image retrieval and text retrieval tasks. Overall, we once again see that our model achieved the best and second-best results both datasets comparing other SOTA models.
Training Objectives with OOD detection Layer. To precisely characterize the performance benefits of using OOD layer, performed more ablations with the default training settings of the baseline and our model mentioned in Section 4.2 We considered different combinations train objectives and evaluated them in zero-shot settings and observed our model performance training objectives. Note that ViLT we compare to in Table 7 is our implementation with the same subset of training datasets. Our raw results presented in Table 7 Here, trained pre-train datasets with Lim in Equation (6), mmlm in Equation (10). and the sum of Lim and Cmlm losses as our training objectives. We that training on image-text matching and masked language modeling beneficial for both downstream tasks comparing to the baseline model. Specifically, there is improvements in image retrieval and text retrieval tasks. Thus, it beneficial train on both ITM and MLM for filtering outlier concepts and improved performance on downstream tasks.
C.3 Qualitative Analysis
As shown in Figure 2 the feature maps obtained using our VK-OOD pipeline can be clustered that are easily identified. This means that detect outliers and group images closest to the corresponding 24 with image and explicit knowledge triplets. Figure 6a and Figure 6b are examples with the nearest images in each cluster.
Figure Figure&sshov show more qualitative examples of multimodal alignment results of our models.
Here, we visualized the multimodal attention maps images corresponding to concept triplets using Grad-cam [52]. Following our model architecture, the caption parsed and integrated with
knowledge triplets. The right bottom sub-figures of our model in Figure 7 and Figure g are the
17

---

(a) Example cluster
(b) Example cluster
Figure 6: Example images in the clusters COCO val set.
drive animal park
Original
giraffe locate in
zebras locate in zoo
giraffe and zebras mingle as cars drive out of an animal park
cars crive tree locate in 200
zebras locate in
animal park
Our model
Baseline
Figure 7: Visualization of the attention maps of image feature e, and language features Ij of external knowledge alignment. The results are from VK-OOD-s (ViLT) model. The original sample caption is "a giraffe and zebras mingle as cars drive out of an animal park". We highlight areas in the example image corresponding to different knowledge triplets. Comparing with the attention maps of the baseline model, our model learns object shapes such as zebras and localize those objects correctly.
multimodal alignment of original captions from MSCOCO dataset 35 alignments of extracted knowledge triplets on the image.
Other sub-figures show the
Interestingly, we found that VK-0OD model is able to capture concept "plug" as part of "refrigerator" or "microwave" Figure 7 The heatmap area of "plug" and "microwave" in Figure suggest that our model has the capability to exploit different relevance between visual and corresponding conceptual text features. and microwave. Figure 8 shows that VK-OOD can detect three zebras comparing with baseline, but counting cars is not performing well as we expected - since the size (or scale) cars is not sufficiently high, and moreover some parts of them are occluded.
contrast, the baseline results did not show the relation between plug
18

---

Original
microwave locate room
refrigerator locate in room
refrigerator locate in room
plug is part of refrigerator
plug is part
a refrigerator freezer microwave oven
plug is part of microwave
microwave
Our model
Baseline
Figure Visualization of the attention maps of image and knowledge concept triplets alignment. The results are from our VK-OOD-s (ViLT) model. The original sample caption is "a metallic refrigerator freezer next to a microwave oven". We highlight areas in the example image corresponding to different knowledge triplets. Comparing with the attention maps of the baseline model, our model learns the relations between the parts (i.e., plug) the objects correctly.
19


Reviews:
[{"review_id": "SaWBn53baJ", "reviewer": "Reviewer_uxWt", "summary": "This paper proposes a framework for multi-modal (text and image) analysis subject to a differentiable framework for out-of-distribution detection in the input space. The method proposes a pipeline to predict the modes of the in-distribution data using a Gaussian mixture model and then leverages it to predict an out-of-distribution (OOD) score for knowledge triplets. These OOD scores are then used to weigh image-text matching as the training objective. The multi-modal encoded features are then used for down-stream applications such as VQA.", "questions": "I went over the paper a couple of times to fully understand it, some parts were not fully clear\n\n- In line 105, x is described as input features including triplets. I am assuming that x only contains the text data and not visual data. Given x_i, mean and standard deviation (std) of the in-distribution (ID) data is computed. Knowing the mean and std directly depends on x_i, are there out-of-distribution data in x_i when computing these values? If so, what's their percentage and how much they can impact the obtained mean and std? \n\n- What is the matching score function m() in Eq. (4)? This is not defined.\n\n- Given the image v_q and triplet l_j similarity score p(v_q, l_j) in Eq. (4), how the final p(v,l) is computed in Eq. (5)? Is it for a specific triplet or some sort of aggregation over all triplets is taken?\n\n- Given the above description my understanding is that the OOD is measured only on the textual data in Eq. (3) of the paper. However, the training objective is to obtain an image-text matching score. Why OOD is computed only on text and not on the joint representation of image and text? Isn't the text-image score m() already capturing the ODD cases which is more relevant for the objective (image-text matching)?\n\nTypos:\n\n- abstract (The following line should be fixed -- remove but): When these models are used for prediction, but they may fail to capture important semantic information and implicit dependencies within datasets.\n\n- start (*) for sigma in Eq. (3) is missing.", "limitations": "No concerns", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper leverages the GEM [33] out-of-distribution detection (in a differenetiable setup) in the context of text-anomaly-detection, which it then used for image-text score matching.", "weaknesses": "While the method seems to have potentials for OOD detection, there are a couple of shortcomings regarding its motivation and its usage for the downstream applications:\n\n- One of the motivations of the paper in the abstract is the difficulty of fine-tuning and deployment as the model size increases. However, compared to other baselines in Table 4, the model has higher number of parameters. I was expecting a model much lighter but with a comparable or slight degraded performance. Compared to simple EM baselines in Table 1, the approach has about 12% faster training time, but has almost similar FLOPS and params (compared to JB-EM). I think the motivation is not well justified for this approach.\n\n- While the OOD detection component should theoretically boost the performance compared to other baselines, in table 4, there is not a big gap in-between methods. By using a ViLT backbone the model gains a reasonable boost compared to the same model (still having more params). Using BLIP backbone this is not the case. The question is whether the OOD component is needed or the performance gain is due to other architectural and training components? Some of these baselines do not seem to be using any OOD component yet they obtain decent/similar results.\n\n- (related to above) It is not clear how much OOD samples exist in the training data and how much it is needed in a model. While the approach seems interesting, it is not clear how much it is useful given the application.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read and review our paper. We would like you to kindly read our general responses for the clarifications on the model parameters and performance. Here we will address your specific concerns and/or questions:\n>One of the motivations of the paper in the abstract is the difficulty of fine-tuning and deployment as the model size increases. However, compared to other baselines in Table 4, the model has higher number of parameters. I was expecting a model much lighter but with a comparable or slight degraded performance. Compared to simple EM baselines in Table 1, the approach has about 12% faster training time, but has almost similar FLOPS and params (compared to JB-EM). I think the motivation is not well justified for this approach.\n\n**What are our motivations?** Our goal is to integrate existing external knowledge into multimodal pipelines. However, upon further investigation, we noticed that external knowledge could be noisy and can hinder training. With this motivation, we developed a differentiable outlier detection layer with decent theoretical and practical advantages. For example, please see the general response 1 and 2, and the rebuttal PDF for the updated Table 4.\n\n>While the OOD detection component should theoretically boost the performance compared to other baselines, in table 4, there is not a big gap in-between methods. By using a ViLT backbone the model gains a reasonable boost compared to the same model (still having more params). Using BLIP backbone this is not the case. The question is whether the OOD component is needed or the performance gain is due to other architectural and training components? Some of these baselines do not seem to be using any OOD component yet they obtain decent/similar results.\n\n**Why there is not a big gap in-between methods?** Please refer to the general response 2. Moreover, as the results shown in Table 2 indicate, both KG and OOD component provide nontrivial gain in understanding and generalization performance in large-scale vision and language settings. Furthermore, we demonstrate the scalability of VK-OOD across different backbone models with diverse architectures and model sizes. Our proposed implicit layer can be regarded as a plug-and-play module for the multimodal pretrained models, making it easy to incorporate with other vision-language models, **as can be seen in our submitted code**. \n\n>It is not clear how much OOD samples exist in the training data and how much it is needed in a model. While the approach seems interesting, it is not clear how much it is useful given the application.\n\n**What are OOD samples and why we need them?**\nThe OOD samples are not present within the training datasets themselves. Instead, in our proposed method, we encounter outliers when integrating external knowledge triplets into the training pipeline. For example, when extracting external knowledge triplets using the given captions, certain concepts/objects might be out-of-distribution (OOD) w.r.t textual/visual features present in training datasets. For a detailed illustration of the OOD examples, please refer to Figure 5. In the experimental setup, we perform ablation studies on VK-OOD components as detailed in Section 4.2. As results shown in Table 2, the models with OOD detection components consistently  outperformed those lacking OOD detection layers in both settings.\n\n**Q1: Does $x_i$ include visual features and textual features? What is the fraction of OOD samples?** \n\nYes! So, $x_i$ represents the union of features from text and image data. Our key idea is to use a mixture model to approximate the distribution of \"In-distribution\" features as a mixture model parameterized ($\\mu_k^*$ and $\\sigma_k^*$) using this union of features. Intuitively, if an image/language feature $x_i$ is close to at least one of these components, then they will be considered in-distribution.  In equation (3) we show only score calculation for $l_j$ for simplicity, but we are happy to clarify this under the equation.\n\nIn our experiments, we interpret the features from images and captions existing in the training dataset to be In-Distribution (ID) samples, and then we augment the captions from external knowledge which may be OOD samples. In particular, if all external triplets are OOD, the training set will contain $\\approx 70\\%$ OOD samples at most in the experiments since we augment $\\approx5$ external triplets for every textual input.\n\n**Q2:** The matching score function $s$ in equation (4) is the calculation of cosine similarity between image and textual(triplet) features. We will clarify this in the revised version.\n\n**Q3:** The calculation of $p(v_q, l_j)$ is given in equation (4), and then we average all image-triplet pairs, as denoted by Expectation ($\\mathbb{E}$) in Eq. 5, during the training.\n\n**Q4: Is it possible to use OOD visual feature in your framework?** \n\nYes, of course! In our current implementations, we use textual external knowledge resources which are easy to incorporate. Thus, we consider to filter OOD samples from the textual triplets. Our framework can indeed be modified to handle visual OOD features as well, and we will include this feature in our code release. In the future work, we will consider to directly integrate visual knowledge bases, such as Google Images."}, {"review_id": "Tkc7GbQnrr", "reviewer": "Reviewer_oiir", "summary": "This paper introduces a novel approach to integrating explicit knowledge graphs into deep networks for multimodal analysis. To filter noise brought by external knowledge, the authors propose an implicit differentiable Out-of-Distribution (OOD) detection layer with efficient backpropagation. This implicit layer comprises the Expectation Maximization (EM) steps of Gaussian Mixture Models (GMM). To efficiently differentiate through the OOD detection implicit layer, Jacobian-free backpropagation was applied in the final optimization step. The proposed OOD detection layer demonstrates state-of-the-art results with significantly fewer samples and less training time.", "questions": "- How was the OOD detection layer initialized for the EM algorithm? Given that EM may fall into local optima, how does this influence the model's performance in the OOD settings?\n- The accuracy gain from iterations doesn't appear to plateau in Figure 3(b). Is the EM algorithm convergent at the maximum iterations? Would it benefit from additional iterations?\n- Could you plot the fixed point error $|\\mu_{t} - EM(\\mu_{t})|$ alongside the optimization steps and include this plot near Figure 3(b)?\n- Given the paper's claim of the robustness of the OOD detection layer, I am curious about the adversarial robustness of the proposed VK-OOD in the VQA settings.\n- Specifically, could any input perturbations/attacks impact the convergence or stability of the implicit layer? It would be useful for authors to monitor the convergence using the aforementioned fixed point error.", "limitations": "The paper discusses the limitations of the proposed OOD detection implicit layer, stating that it could infer inefficiently if the covariance matrix is parameterized densely. A fast linear system solver could potentially rectify this inefficiency. However, these extensions was left for future works.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The experiment section is well-structured. Clarity of this section is a plus.\n- The ablation study is carefully conducted, revealing the influence of hyperparameters and external knowledge resources and the effectiveness of efficient backpropagation.\n- Despite having 30% incomplete data, the proposed implicit layer outperforms previous models in the OOD settings. This demonstrates the OOD generalization of implicit layers.\n- Informative qualitative results demonstrate how the proposed model performs in the real VQA datasets.", "weaknesses": "- Missing references. The current manuscript doesn't effectively integrates previous research into the discussion. The following references are highly relevant and should be acknowledged:\n  - Paper [1] is a landmark work in implicit models.\n  - Paper [2] proposed the one-step gradient for backpropagating through iterative algorithms, substantially overlapping with and earlier than the efficient differentiation in this work. Both paper [2] and this work leverage non-convex optimization layers of multiple variables.\n  - Paper [3] systematically studies the inexact gradient for implicit models in theory and practice.\n  - Papers [4,5] delve into the Out-of-Distribution (OOD) generalization of implicit layers and the benefits from path-independence/convergence.\n  - Paper [6] discusses the adversarial robustness of implicit models.\n\n- The method section can be rewritten for better clarity. The paper could benefit from an improvement in notation clarity.\n\nTypos:\n\n- Line 314: \"saccelerater\" should be \"accelerate\".\n\n[1] Bai, Shaojie, J. Zico Kolter, and Vladlen Koltun. \"Deep equilibrium models.\" Advances in Neural Information Processing Systems 32 (2019).\n\n[2] Geng, Zhengyang, Meng-Hao Guo, Hongxu Chen, Xia Li, Ke Wei, and Zhouchen Lin. \"Is Attention Better Than Matrix Decomposition?.\" In International Conference on Learning Representations. 2020.\n\n[3] Geng, Zhengyang, Xin-Yu Zhang, Shaojie Bai, Yisen Wang, and Zhouchen Lin. \"On training implicit models.\" Advances in Neural Information Processing Systems 34 (2021).\n\n[4] Anil, Cem, Ashwini Pokle, Kaiqu Liang, Johannes Treutlein, Yuhuai Wu, Shaojie Bai, J. Zico Kolter, and Roger B. Grosse. \"Path Independent Equilibrium Models Can Better Exploit Test-Time Computation.\" Advances in Neural Information Processing Systems 35 (2022).\n\n[5] Bai, Shaojie, Zhengyang Geng, Yash Savani, and J. Zico Kolter. \"Deep equilibrium optical flow estimation.\" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[6] Yang, Zonghan, Tianyu Pang, and Yang Liu. \"A Closer Look at the Adversarial Robustness of Deep Equilibrium Models.\" Advances in Neural Information Processing Systems 35 (2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read and review our paper. We are glad that you found our proposed method with OOD detection implicit layer novel and has efficiency benefits. We have started working on integrating suggested previous related research in the manuscript and will incorporate all the modifications into the revised version. Please check our general responses for the clarification on the model parameters and performance. We now address your  specific questions:\n\n**Q1: How did you initialize your EM algorithm?** \n\nWe initialized the OOD detection layer parameters $\\mu$ and $\\sigma$ using random $k$ number of vision features of the inputs. Our OOD detection layer may fail during inference. However, for training purpose, approximating $\\mu^*$ and $\\sigma^*$ might be sufficient. With an approximation $\\tilde{\\mu}$ and $\\tilde{\\sigma}$, we use the GEM score to filter OOD features in multimodal pipeline. In the experimental results, Table 2 in the paper illustrates that the performance of the model with OOD detection layer is better than the models w/o OOD detection layer in downstream tasks.\n\n**Q2: Is the EM algorithm convergent at the maximum iterations? Would it benefit from additional iterations?** \n\nPlease see the general response 3. Perhaps there is a misunderstanding, but we could not think of any reason why accuracy should plateau in Figure 3(b), because the x-axis in Figure 3(b) is the number of triplets/clusters. We think the reviewer may refer to Figure 3(a) which x-axis is $T$ iterations where we can observe plateauing.  \n\nYes, we believe more iterations may be beneficial. Although, considering the training costs, we conduct ablation studies on $T \\leq 10$ and the improvements in term of accuracy are slow after $T \\geq 5$. Moreover, the main goal of OOD detection layer is to find the approximate clusters for the inputs. \n\n\n**Q3: Can you provide empirical convergence plot of $\\mu,\\sigma$ of In-Distribution (ID) features using EM algorithm?** \n\nWe have provided the fixed point error plot over iterations in the rebuttal pdf. We find that the squared euclidean distance between successive iterates indeed goes to zero. We will add this plot in Figure 3 in the revised version, thank you for the suggestion.\n\n**Q4: How is Adversarial Training related to OOD detection?** \n\nThank you for the question. We can use one of components $\\tilde{\\mu}_k$ and $\\tilde{\\sigma}_k$ to be one of the mixtures to handle adversarial samples.  To calculate this component we can use an appropriate adversarial sample generation or attack algorithm at the feature level. We leave this for future work.\n\n**Q5: Could any input perturbations/attacks impact the convergence or stability of the implicit layer?**\n\nYes, it  can impact convergence behavior of EM algorithm, and hence our layer. Designing robust outlier detection is a great suggestion and still remains an open problem, so we will consider the problem for future work. "}, {"review_id": "7Lq605Lf6P", "reviewer": "Reviewer_h1Px", "summary": "The paper presents an approach that combines the features from pre-trained deep networks and freely available semantic explicit knowledge. It proposes an implicit out-of-distribution (OOD) detection layer to address outlier detection and thus further improve understanding and generalization performance in large-scale vision and language settings. It offers comprehensive explanations of the proposed approach, along with experimental results and comparisons against other methods.", "questions": "1. In Table 4, how do you interprete that VK-OOD(BLIP) is worse than BLIP in COCO and Flickr30k (especially yours has more params)?\n2. How much more training resource/time were added when adding VK-OOD to other SOTA vision-language models?\n \n", "limitations": "The iterative method, though optimized for efficiency, still has concerns in training time.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method is utilizing outlier detection to improve model training, instead of solely for outlier detection.\n2. The proposed method is differentiable, efficient and can be applied to different datasets and multimodal backbones.\n3. Comprehensive experiments.", "weaknesses": "1. The iterative method, though optimized for efficiency, still has concerns in training time.\n2. Adding the proposed method sometimes causes degradation as shown in Table 4. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper. We will address your concerns/questions regarding the training time and model performances as follows:\n\n**Q1:** Please refer to general response 1 and 2, and check the updated Table 4 in the rebuttal PDF. During training, we augmented each captions with 5 external knowledge triplets. Thus, considering the ***noise*** level of textual feature space, the recall drop shown in Table 4 is **marginal**.\n\n**Q2:**\nWe pre-train on three datasets with total of 1M images and 6.8M image-caption pairs, which is approximately **30\\% less** than what the baselines such as ViLT used in their training. Each caption is augmented with 5 external knowledge triplets. We trained VK-OOD-l (ViLT) on the aforementioned training set with 50k steps on 8 NVIDIA 2080Ti GPU and it took around 2.5 days only. Thus, the overall training time and resources are much more less than ViLT and BLIP. However, our proposed model obtains significant improvements in all downstream tasks with training on less samples. Each sample in VK-OOD-l (ViLT) **only** requires $\\approx$ 3ms more than the baseline(ViLT) in the dense case. Our results seem promising in this way!"}, {"review_id": "rFHSvECdUr", "reviewer": "Reviewer_MwwQ", "summary": "This paper introduces a implicit Differentiable Out-of-Distribution (OOD) detection layer. This layer addresses outlier detection by solving for fixed points of a differentiable function and using the last iterate of fixed point solver to backpropagate.", "questions": "refer to the weaknesses part.", "limitations": "Has been discussed in the main paper, and I agree with that.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This is a well-organized and written paper. For example, the language of the introduction section, starting from the statement of the advantage of JFB method to the problem of external knowledge-based multimodal methods, the storyline is smooth, natural, and clear. \n\nThe paper is self-contained. Most contributions claimed in the introduction section have the corresponding evidence in the experiment section. Ablation study, analysis, and limitations, all of them be included, organized, and discussed to support the points of this paper.\n\nFinally, such a JFB idea is pretty interesting.", "weaknesses": "1. As my understanding, VK-OOD is a method that would not incur many additional parameters. Why is the number of parameters in Table 4 between the baselines and the proposed method have such a big gap (e.g BLIP vs VK-OOD(BLIP), 346M vs 412M)? Does it a fair comparison?\n\n2. This paper did a simple idea, making the iterative algorithm (e.g k-means) end-to-end with networks but making it faster and more efficient by JFB method. I don't know if it is a real problem in that community. Does it really matter? Like the results shown in table 1, basically, the improvement is minor.\n\nThis paper is definitely a good paper, I could realize the professionalism of the authors. My concerns are mainly about the contribution. If the authors could offer convincing reasons, I would consider raising my score.  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read and review our paper. We are glad that you found our paper to be well-written and the proposed JFB for outlier detection idea to be interesting. Please check our general responses for the clarification on the \nmodel parameters and performance firstly. Then, we will clarify your concerns about the number of parameters and our contributions, please see the responses below:\n\n**Number of parameters - W1:** In VK-OOD, we update $\\mu$ and $\\sigma$ during the training, which will bring additional parameters. Please refer to the clarification for the number of parameters in the general response 1 and check the updated Table 4 in the rebuttal PDF. As results shown in updated Table 4, the **scalar** VK-OOD also outperforms other baselines with **marginal** increase in the number of parameters, such as VK-OOD-s (BLIP) vs BLIP: 346.4M vs 346M.\n\n**Model performance - W2:** Indeed, Table 1 shows the proof of principles w.r.t time and memory costs of using JFB based OOD feature detection in multimodal pipelines. For the major improvements, please refer to Table 2 and Table 4 to see more details. The results indicate that VK-OOD achieves significant improvements (upto $\\approx$ 5 -10 \\% increased accuracy)  compared to what baselines achieve in the same settings. \n\n**Our contributions:**\nAs noted by other reviewers, we tackle the complex challenging problem of integrating explicit and implicit knowledge in end-to-end multimodal pipelines, aiming to enhance performance while also reducing computational resources. Fortunately, our proposed method is differentiable, has efficiency advantages and can be applied to different datasets and multimodal backbone networks or upstream feature extractors. Furthermore, we provide comprehensive and well-conducted experiments, which illustrate promising understanding and generalization performance improvements on multiple downstream tasks. In short, our main novelty and contribution is the application of JFB method for outlier detection with slightly more parameters in practical multimodal settings."}, {"review_id": "NaKxNYvLhw", "reviewer": "Reviewer_NYTC", "summary": "The paper presents a method for incorporating outlier detection into an end-to-end learnable framework. Essentially the paper shows how it is possible to differentiate efficiently through a per-instance Gaussian mixture model using either unrolling, implicit differentiation or Jacobian-free back-propagation as discussed in recent literature. The paper is well written and experiments combining vision and language for visual-question answering (VQA) are promising.", "questions": "See questions raised in Weaknesses above. I am willing to increase my score depending on the answers to these questions, especially relating to results reported in Table 1.", "limitations": "Limitations were adequately addressed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well written and the proposed method for a differentiable Gaussian mixture model is sound. I am mostly positive about the paper but have a few questions below. I am willing to raise my score depending on the answers provided by the authors.", "weaknesses": "As mentioned above, the paper is mostly well written. However, there are a few minor places where the paper could be improved. Some prior related work on differentiable optimization is missing (e.g., Gould et al., TPAMI 2022). These and cited prior related works already discuss the reduced memory costs during back-propagation via implicated differentiation limiting the novelty of the paper (certainly Line 73). Some of the mathematical expressions can be tidied, e.g.,\n\na. Size of outer parentheses in Eqn (3)\nb. Lower limit in denominator summation in Eqn (4) should be \"u=1\"\n\nAnd some consistency in referencing tables and figures. E.g., Section 4.1 refers to figures in four different ways \"Figure 3a\", \"Figure3a\" (no space), \"Fig 3b\" (no period), and \"Fig. 2\" (period, but breaking space). Also tables are referenced with lowercase \"table\" and figures with uppercase \"Figure\". I recommend being consistent and using non-breaking spaces between label and reference number.\n\nAlso, wrapping tables in the text can result to confusing line breaks, e.g., Line 265.\n\nThe above are all minor presentation issues. Two more serious concerns are:\n\n1. The results from implicit differentiation are only valid when the forward pass algorithm actually results in a fixed point, i.e., is run to convergence. Since the method presented in the paper only iterates for a pre-determined number of iterations, convergence cannot be guaranteed. What happens when the resulting GMM parameters have not converged? Can the authors comment on the validity of the gradients obtained?\n\n2. Table 1 compares three different methods for back-propagation. I understand that the time and memory of these methods will vary but I don't understand why the number of parameters in the models should be different (i.e., 152.6M, 125.2M and 124.8M for Vanilla-EM, JB-EM and JFB-EM, respectively). Can the authors please comment. Also, does the time per epoch include execution of the forward pass? I would expect this to dominate the backward pass. Can the authors please provide forward pass timings separate from backward pass timings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our paper. We appreciate your constructive suggestions to illustrate forward pass and backward pass execution time separately in Table 1. We will fix all the typos including in math expressions and keep the references of tables and figures consistent in the revised version. We will address your concerns in the following responses:\n\n>The results from implicit differentiation are only valid when the forward pass algorithm actually results in a fixed point, i.e., is run to convergence. Since the method presented in the paper only iterates for a pre-determined number of iterations, convergence cannot be guaranteed. What happens when the resulting GMM parameters have not converged? Can the authors comment on the validity of the gradients obtained?\n\n**Concern 1:**\nPlease refer to the general response 3 to see if it is answered your question. As long as the optimization algorithm (JFB-EM) converges to clusters points $\\tilde{\\mu},\\tilde{\\sigma}$ such that the gradients calculated is positively correlated to the true gradients, we can decrease the loss function. Thus, it is not necessary that EM iterations solve the clustering problem exactly. In  practice, we run for few iterations to get a reasonable $\\mu$ and $\\sigma$ and use the energy of GEM score to filter OOD samples in the overall pipeline. Thus, the only use of $\\mu^*$ and $\\sigma^*$ is to calculate OOD score $s$ in equation (4), so the approximate $s$ is sufficient enough for optimization purposes.\n\n>Table 1 compares three different methods for back-propagation. I understand that the time and memory of these methods will vary but I don't understand why the number of parameters in the models should be different (i.e., 152.6M, 125.2M and 124.8M for Vanilla-EM, JB-EM and JFB-EM, respectively). Can the authors please comment. Also, does the time per epoch include execution of the forward pass? I would expect this to dominate the backward pass. Can the authors please provide forward pass timings separate from backward pass timings.\n\n**Concern 2:**\n\n(1) We need to store the intermediate gradients and $\\mu$ and $\\sigma$ of each iteration for Vanilla-EM and full Jacobian matrix for JB-EM during training, so we consider to add them to the number of parameters. \n\n(2) Yes, the time reported in Table 1 includes the execution time of the forward pass. \n\n(3) We agree with the reviewer that forward pass dominates backward pass since we can reuse the weights available during forward pass.\n\n(4) We update Table 1 with separate forward pass and backward pass timings in the rebuttal PDF and will update it in the revised version. "}]

Author rebuttal:
Thank you for taking the time to read and review our paper. We appreciate the reviewers unanimously agree that our submission is sound, well-presented with contributions clearly written. We are committed to fix all the typos and incorporate all other suggested modifications into the revised manuscript. In this response,, we will address all the technical concerns and questions to further strengthen support for our submission.

## 1. Why are the number of parameters higher than the baselines in Table 4? (Reviewer uxWt, h1Px, MwwQ)
To clarify, the parameters ($\mu$ and $\sigma$) of our proposed OOD detection layer can be initialized in two ways: as a **scalar** $\sigma$ or the **dense** matrix where $\sigma$ is a $d \times d$ matrix with d representing the dimension of input embeddings. The number of parameters of OOD detection layer in the scalar case is $d \times k \times \text{batch size} + k \times \text{batch size}$, and the dense case is $d \times k \times \text{batch size} + d \times d \times k \times \text{batch size}$. Note that if we opt for $\sigma$ as a diagonal matrix, each component of the mixture model will require $2d$ parameters, as opposed to just $d$. In our implementation, $d$ is 768, $k$ is 5 and batch size varies depending on different backbones, such as 32 or 64. We have provided an **updated Table 4** with two cases in the rebuttal PDF. Importantly, in the updated Table 4, the number of parameters in scalar VK-OOD are approximately similar to the baselines. Specifically, please consider to compare to other baseline models, while our *scalar* VK-OOD increases the \#parameters slightly -- ***$\approx 0.4$*** million more parameters, it **significantly** *improves* the performance in many of the downstream tasks that we considered in our submission. 

## 2. What improvements does VK-OOD demonstrate in Tables 1 and 4? (Reviewer uxWt, h1Px, MwwQ)
In **updated** Table 1 in the rebuttal pdf, we highlight the empirical advantages (e.g., computational efficiency) of the Jacobian Free Backpropagation (JFB) method in multimodal settings compared to alternative gradient calculation methods. The results illustrate that the proposed  JFB based implicit layer is beneficial to reduce gradient computation costs in term of time and memory usage. 

Reviewers will be able to see that **updated**  Table 4 in rebuttal pdf clearly shows model performance improvements in term of appropriate evaluation metrics on specific downstream tasks as we explain as follows:
- For the visual understanding tasks (such as VQA and NLVR), both our scalar and dense VK-OOD seem beneficial. For example, in NLVR2, our dense VK-OOD achieves the best result in terms of accuracy with **10\% and 1.7\% increase** than ViLT and BLIP respectively. The scalar VK-OOD also outperforms all baselines with similar number of parameters in the understanding tasks shown in the updated Table 4. 
- For the retrieval tasks, the benefits of using VK-OOD is not so significant, and but shows the robustness of the proposed implicit layer for filtering features. We will explain why this empirical result is natural. Please recall that in the experiments, the goal in retrieval is to retrieve images from text queries or to retrieve texts from images. Now, since we augment captions with external knowledge which are known to induce ***noisy textual features***, the slight improvement in image/text retrieval tasks in COCO and Flickr30k dataset is expected. The numbers reported by the baseline have no such external knowledge, indicating that our implementation works as intended! Finally, even in the retrieval tasks, it may be possible to improve performance by adjusting the noise levels (number of external knowledge triplets) during the fine-tuning process. 

## 3. Is it necessary to calculate the fixed point exactly for training in multimodal pipeline? Is it ok to use good EM with few iteration for clustering multimodal features? (Reviewer oiir, NYTC)

No, for training purpose, an approximate $\mu^*$ and $\sigma^*$ is sufficient. In VK-OOD training pipeline, we only require that the gradients provided by the proposed OOD detection layer to be ***a descent direction** with respect to the loss as a function of network parameters*. Thus, as long as the gradient backpropagated is correlated (or has a nonnegative inner product) with the true gradient of upstream layers, the updated weights will still *decrease* the loss function, thereby making optimization progress. Formally, by nonnegative correlation, we need the gradient calculated to satisfy: $\left(g_{\mu^*,\sigma^*}\right)^{\top} g_{\tilde{\mu},\tilde{\sigma}} \geq 0$, where $\tilde{\mu}, \tilde{\sigma}$ is an approximation returned by finite iterations of EM algorithm of the true clusters $\mu^*,\sigma^*$.

Meta-review: The paper focuses on the setting of multi-modal learning, where external expert sources are combined with clean data. The idea is that the expert sources might not always be clean and outliers must be detected and discarded. Outlier detection is then described as an Out-of-Distribution layer. Outliers then are discovered by solving a fixed-point problem, and employing differentiable solvers for backpropagation.

Reviewers are on the borderline but basically positive. They all appreciate the novelty and added value, and negative criticisms are with respect to whether outliers are indeed discovered. However, the authors provide additional experiments to prove so. All in all, this is a submission where reviewers are favorable and I recommend acceptance.
Decision: Accept (poster)