SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Implicit Differentiable Outlier Detection Enables Robust Deep Multimodal Analysis
Zhu Wang Sourav Medya Sathya N. Ravi
Department of Computer Science, University of Illinois at Chicago
{zwang260,medya,sathya}@uic.edu
Deep network models are often purely inductive during both training and inference on unseen data. When these models are used for prediction, but they may fail to capture important semantic information and implicit dependencies within datasets. Recent advancements have shown that combining multiple modalities in large-scale vision and language settings can improve understanding and generalization performance. However, as the model size increases, fine-tuning and deployment become computationally expensive, even for a small number of downstream tasks. Moreover, it is still unclear how domain or prior modal knowledge can be specified in a backpropagation friendly manner, especially in large-scale and noisy settings. To address these challenges, we propose a simplified alternative of combining features from pretrained deep networks and freely available semantic explicit knowledge. In order to remove irrelevant explicit knowledge that does not correspond well to the images, we introduce an implicit Differentiable Out-of-Distribution (OOD) detection layer. This layer addresses outlier detection by solving for fixed points of a differentiable function and using the last iterate of fixed point solver to backpropagate. In practice, we apply our model on several vision and language downstream tasks including visual question answering, visual reasoning, and image-text retrieval on different datasets. Our experiments show that it is possible to design models that perform similarly to state-of-the-art results but with significantly fewer samples and less training time. Our models and code are available here: https://github.com/ellenzhuwang/implicit_vkood
Numerous neural network models are constructed via the stacking of explicit layers that transform inputs into outputs through a sequence of operations associated with the designated layers. Although these explicit layers are expressive, they may be unnecessary in many large-scale applications where only the function value and its gradients are required. For instance, it is possible to employ implicit layers for diverse tasks, including hyperparameter optimization [7], meta learning [50], and solving inverse problems in image processing [21]. Indeed, if a desired input-output correspondence within a network can be expressed as an optimization problem, gradients can be computed efficiently [30].
Recently, JFB [17] proposed an efficient method to backpropagate through implicit layers by implementing them as a Network Operator  $\mathcal{N}(\cdot)$ . Here,  $\mathcal{N}$  is defined as a sequential application of a map  $F(\cdot)$ , which is guaranteed to converge in  $T < \infty$  iterations. Formally,  $\mathcal{N}(x)$  can be represented as,
$$
\mathcal {N} (x; \Theta) = \underbrace {F \circ F \circ \cdots \circ F} _ {T \text {t i m e s}} (x; \Theta), \tag {1}
$$
where  $\Theta$  corresponds to parameters of a potential multimodal model. The convenience of  $\mathcal{N}$  in Equation (1) lies in its ability to backpropagate through  $\mathcal{N}$  without requiring its full sequence (or
Figure 1: Our Proposed Architecture. The inputs are images and captions/questions. Image patches are processing by a visual encoder, while the caption, integrated with a knowledge graph, is passed to a language encoder. The extracted features are then fed into the OOD detection layer. This implicit layer identifies noise concepts before forwarding the features to the multimodal encoder. The ITM and MLM are training objectives, and VQA, NLVR, and image-text retrieval are downstream tasks.
trajectory) or solving an inverse problem with its Jacobian matrix for gradients. In Vision tasks, implicit maps like clustering with fixed parameters – for instance,  $\mathcal{N}$  could be  $k$ -means with fixed means – have been shown to enhance performance [57]. To enable small-scale custom applications or to fine-tune large-scale models, fixed parameters are insufficient, so we consider the problem of evaluating gradients with respect to the means efficiently.
Introducing "Benign" noise in Multimodal Pipelines. Recent advancements [65, 27] have demonstrated that the combinations of multiple modalities improve understanding and boost generalization performance. However, these models, trained by a simple relation (matched or unmatched) between the image and text pair, often falter in capturing important semantic information and implicit dependencies within datasets. For example, as depicted in Figure 1, these models may recognize the green light but fail to infer the legality of a person crossing at the green light. Thus, the incorporation of "noisy" external knowledge presents a potential solution. In this work, we explore the design of an implicit Network Operator  $\mathcal{N}$  to manage feature-level noise during training. Specifically,  $\mathcal{N}$  is employed to integrate external knowledge in multimodal learning, which is a deep network with parameters  $\Theta$  developed to align multiple modalities [49, 46].
Using External Knowledge for Reasoning Purposes. A knowledge graph collects information, organizes it in a specific structure, i.e., ontology, and is utilized to deduce novel insights and knowledge from the integrated information [16]. Often, external knowledge may be beneficial for reasoning tasks [22]. Most recent knowledge-based multimodal models [22, 36, 53] focus on entity retrieval methods from external knowledge resources such as structural knowledge graphs and large language models. Unfortunately, these methods become impractical when combined with features derived from external knowledge resources that are prone to incompleteness and noise. Furthermore, vision-language models like those in [18, 66] often struggle to filter noise pairs during training, resulting in slow convergence. For instance, in Figure 1, given an object "street" and relation "locatedAt", noisy objects like "fire hydrant" are returned with high confidence by ConceptNet [55]. However, the "fire hydrant" is absent in the input image, which is undesirable for training objectives, such as image-text matching.
We propose an implicit out-of-distribution (OOD) detection layer to handle outliers from external knowledge as discussed above. We demonstrate how such a layer can be expressed as a network operator  $\mathcal{N}$  in Equation (1) for memory saving and efficient training. Specifically, in the ongoing example in Figure 1, our proposed approach approximates the density of in-distribution features,
which are extracted from the triplets in the caption (question) corpus, by a fixed point iterative layer. Thus,  $F$  is trained with implicit differentiation at the fixed point according to OOD scores [44] for the retrieved knowledge triplets. Overall, we provide an end-to-end training framework with knowledge graphs and multimodal models in an implicitly differentiable manner, as illustrated in Figure 1 showcase the effectiveness of the proposed implicit layer based on  $\mathcal{N}$  in large-scale multimodal pipelines. We conducted several experiments on multiple vision and language tasks, such as visual question answering and image text retrieval. We have included extensive evaluations and ablation studies to demonstrate the improvements in memory usage and training time compared to vanilla backpropagation.
We summarize our three primary contributions as follows: (1) Our proposed implicit layer can perform outlier detection tasks efficiently with less memory costs during backpropagation in practical settings. This layer can scale to different datasets and multimodal backbones. (2) We demonstrated that integrating explicit knowledge enhances multimodal fusion models. Our model learned from visual and textual modalities while using external knowledge seamlessly that may available in certain situations. (3) Resulting model after training and/or fine-tuning, outperforms baseline models on various downstream tasks. Moreover, we provided various ablation studies and an interactive simulation based user study, which shows that users can sustain their desired in-distribution inputs or features while being able to update model parameters simultaneously.
Deep implicit layers. Implicit differentiation has been an emerging and competitive alternative to explicit unrolling for backpropagation. From the practical point of view, it is often memory-efficient and numerically stable compared to vanilla unrolling [7, 25]. Recent works apply implicit functions and layers in various problems with deep neural network, such as optimization layers [2, 6], convolutional sparse modeling [33] and object representations [9]. Moreover, implicit layers can be used to produce representations that are robust to perturbations, as well as better empirical performance in downstream predictions. For example, [3, 5] used path independence as a technique to improve the OOD generalization. The main argument is that path independence can be guaranteed while training using implicit layers instead of unrolling which is crucial for our applications.
Out-of-distribution (OOD) detection. Out-of-distribution detection is usually studied from the statistical point of view, mainly under Robust Statistics [26]. We will summarize recent developments focusing on our use cases. For OOD detection, Maximum Mahalanobis Distance [29] and energy score [37] are statistical methods for detecting features that may not be obtained using training data. Recently, GEM [44] has introduced a provable and competitive OOD detection method to estimate outliers for deep networks. GEM score can be used to recognize irrelevant distributions of concepts during training. The primary focus is on the detection performance with simulated outliers, whereas our focus in this paper is on utilizing them in real-world noisy data for training purposes efficiently.
Vision-and-language transformers and knowledge-based VQA. Most recent vision-and-language models have shown improved performance by optimizing self-supervision based losses. For example, [10, 27, 32, 31, 54, 1] introduce multimodal architectures wherein cross-modal features are used to learn combined representations of visual and textual contents that can be used to improve predictions. In Vision and Language Processing, knowledge representations in the form of extracted consensus or semantic concepts are aligned with visual concepts to construct concepts vocabulary [24, 19, 60]. In knowledge-based VQA, different explicit knowledge bases can be integrated for answering purposes [40, 63, 23, 22, 53, 36, 11]. In this work, we explore the effectiveness of utilizing external knowledge in deep multimodal models during training.
3 Implicit OOD Detection Layer for Multimodal Analysis
Basic Notations. We denote input features as  $x_{i} \in \mathbb{R}^{d}, i = 1, \dots, N$  including features of extracted knowledge triplets  $l_{j} \in \mathbb{R}^{d}, j = 1, \dots, M$ . The goal of OOD layer is to compute a score  $s(l_{j}) \in \mathbb{R}_{+}$  for  $l_{j}$  to estimate density of in-distribution (ID) data. In this section, we explain how to estimate the distribution of the ID features as a (normalized) linear combination of simpler distributions using an implicit layer to detect outliers and backpropagate for gradients, efficiently.
3.1 Finding Fixed Points for Forward Pass
Finite mixtures with  $k$  components are conceptually simple, and computationally attractive. Mathematically, they can produce accurate approximations to most density functions [43]. So, we approximated the density of ID features using a Gaussian Mixture Model (GMM) and used  $\mu_k^* \in \mathbb{R}^d$ ,  $\sigma_k^* \in \mathbb{R}^{d \times d}$ ,  $k = 1, \ldots, K$  to denote the optimal means and covariance matrices of  $K$  components.
To obtain an optimal GMM, we solved  $\mu^{*}$  and  $\sigma^{*}$  by using Expectation Maximization (EM) algorithm [45]. Our main observation is that the update rule used in an EM-based algorithm of  $\mu_{k}$  on the current iterate  $t\in T$  can be written as a fixed point iteration as follows:
$$
\mu_ {k} ^ {t + 1} \leftarrow \frac {\sum_ {i = 1} ^ {N} \exp \left(- w \left(\mu_ {k} ^ {t}\right)\right) x _ {i}}{\sum_ {i = 1} ^ {N} \exp \left(- w \left(\mu_ {k} ^ {t}\right)\right)} \tag {2}
$$
where the weight of current iterate  $\mu_k^t$  on  $x_i$  denoted by  $w(\mu_k^t) \coloneqq \sum_{k=1}^{K} \|\sigma_k^{-0.5}(x_i - \mu_k^t)\|_2^2$ . We updated  $\sigma_k$  similarly using  $(x_i - \mu_k^t)(x_i - \mu_k^t)^T$  in the numerator of Equation (2). Therefore, we obtained an approximation of in-distribution density that can be used for further outlier detection.
3.2 GEM score for Outlier Detection
After computing the optimal means  $\mu^{*}$  and covariances  $\sigma^{*}$  using the above described fixed point function, the final ingredient we need is a samplewise and memory efficient forward pass OOD detection method. After obtaining the ID parameters, we can compute a score for each  $l_{j}$ , the features derived from external knowledge triplet. In deep network context, the recently introduced GEM score [44] has already been tested on a few classification applications. Specifically, we used GEM score to filter anomalous triplets. GEM score is also beneficial to obtain memory efficient gradients, details will be explained in Section 3.4. Given a derived feature from a knowledge triple  $j\in M$ , its GEM score  $s(l_j)$  is defined using an energy function as,
$$
s \left(l _ {j}\right) = \log \sum_ {k = 1} ^ {K} \exp \left(- \frac {1}{2} \left(l _ {j} - \mu_ {k} ^ {*}\right) ^ {T} \sigma_ {k} ^ {- 1} \left(l _ {j} - \mu_ {k} ^ {*}\right)\right) \tag {3}
$$
where  $l_{j}$  is the textual feature of retrieved external triplets,  $\mu_k^*$  is the optimal means and  $\sigma_k^*$  is the optimal covariance matrix of ID features.
Do we require fixed point iterations in Equation (2) to be exact? In our training pipeline, we only require that the gradients provided by the proposed OOD detection layer to be a descent direction with respect to the loss as a function of network parameters. Thus, the only use of  $\mu^{*}$  and  $\sigma^{*}$  is to calculate OOD score  $s$  in Equation (3), so the approximate  $\mu^{*},\sigma^{*}$  may be sufficient enough for the training purpose. To see this, we considered a simplified setup in which language features  $l$  in Equation (3) are first processed by ReLU based upstream layer parameterized by  $W_{\mathrm{up}}$ , followed by energy calculation using the output means  $\mu_k^*$  of EM algorithm.
Formally, we considered the loss function given by  $E(W_{\mathrm{up}}; \mu^{*}) \coloneqq -\log(\exp(0.5 \cdot \| \mathrm{relu}(W_{\mathrm{up}} \cdot l) - \mu^{*} \|_2^2))$  where we assumed number of components to be one i.e.,  $K = 1$  for simplicity. We now look at the gradients computed by using the approximate output of EM algorithm. Now, the gradients of the energy score (Equation (3)) itself is given by Chain rule as follows:
$$
\begin{array}{l} \nabla_ {W _ {\mathrm {u p}}} E (W _ {\mathrm {u p}}; \mu^ {*}) = \frac {\partial (- \log (\exp (0 . 5 \cdot \| \operatorname {r e l u} (W _ {\mathrm {u p}} \cdot l) - \mu^ {*} \| _ {2} ^ {2})))}{\partial W _ {\mathrm {u p}}} \\ = - \left(\operatorname {r e l u} \left(t _ {0}\right) - \mu^ {*}\right) \odot \operatorname {r e l u} \left(\operatorname {s i g n} \left(t _ {0}\right)\right) \cdot l ^ {T} \tag {4} \\ \end{array}
$$
where  $t_0 \coloneqq W_{\mathrm{up}} \cdot l$ , and  $\odot$  denotes the Hadamard or Elementwise product. Importantly,  $\nabla_{W_{\mathrm{up}}} E(W_{\mathrm{up}}; \mu^{*})$  is linear in  $\mu^{*}$ . By definition of a descent direction, it is possible to reduce the loss using an approximate  $\tilde{\mu}$  computed with finite iterations as long as  $\operatorname{tr} \left( \nabla_{W_{\mathrm{up}}} E(W_{\mathrm{up}}; \mu^{*})^{\top} \nabla_{W_{\mathrm{up}}} E(W_{\mathrm{up}}; \tilde{\mu}) \right) < 0$ . Our calculation above considered only one upstream layer which is rarely the case in practice. We leave extensions to more upstream layers as future work.
Algorithm 1 Fixed Point Network Operator based OOD Detection Layer for Language Features  $l_{j}$
Input  $x_{i}\forall i\in [N],l_{j}\forall j\in [M],\mu_{k}^{0}\forall k\in [K],\sigma_{k}^{0}\forall k\in [K],T$    
Output  $s(l_j),\nabla_{li}s(l_j)(l_j,\mu_i^*,\sigma_i^*)$ $\triangleright$  Feature-wise OOD scores and Gradients   
- Begin Forward Pass -   
while  $t\leq T$  do Update  $\mu_k^{t + 1}$  using Equation (2)   
end while   
Set  $\mu_k^*$  to be the last iterate of  $\mu_k^T$    
OOD Detection. Compute  $s(l_j)$  in Equation (3)   
- Begin Backward Pass -   
Jacobian Free Backpropagation. Output gradient  $\nabla_{lj}s(l_j)(l_j,\mu_i^*,\sigma_i^*)$  by computing derivative of composition of log-sum-exp and ReLU functions in Equations (3) using Chain rule.
3.3 Implementing Differentiable OOD layer in Multimodal Pipeline
To implement our implicit OOD detection layer discussed in previous sections in multimodal pipelines, we propose a novel architecture called VK-OD by fusing Vision and external Knowledge features. In essence, we used an OOD detector as the network operator  $\mathcal{N}$  and detected the retrieved knowledge triplets which can potentially lead to slow convergence of upstream and/or downstream layers during training.
Architecture. Given an image with a caption (question), our pipeline consists of the following steps: (1) Transform the image to visual features using the vision encoder, such as ViT-based [15], (2) Retrieve knowledge triplets using external knowledge bases, and transform to textual features with the language encoder, such as BERT-based [14], more details of retrieval module are introduced in Appendix A, (3) Approximate density of in-distribution features, and filter outliers in the image-triplet pairs with the proposed implicit OOD detection layer, (4) Finally, we learn vision and textual representations by a multimodal encoder with multiple training objectives.
Image-text Matching Loss with GEM scores. We aimed to incorporate the calculated GEM scores to the widely used image-text matching (ITM) loss. Specifically, we consider the model is encouraged to not only match images and texts correctly, but also to map OOD pairs farther away from ID pairs in the feature space. Given image-text pairs  $D = \{(v_{j},l_{j})\}_{j = 1}^{M}$ , the similarity of image-text pair based on  $s(\cdot)$  is derived as:
$$
p \left(v _ {q}, l _ {j}\right) = \frac {\exp \left(m \left(v _ {q} , l _ {j}\right) / \tau + \beta \cdot \operatorname {s i g n} \left(m \left(v _ {q} , l _ {j}\right)\right) \cdot s \left(l _ {j}\right)\right)}{\sum_ {u = 1} ^ {M} \exp \left(m \left(v _ {q} , l _ {u}\right) \tau + \beta \cdot \operatorname {s i g n} \left(m \left(v _ {q} , l _ {j}\right)\right) \cdot s \left(l _ {u}\right)\right)} \tag {5}
$$
where  $m(\cdot)$  is the cosine similarity of image-text pair,  $\beta$  is a learnable weight parameter,  $\mathrm{sign}(\cdot)$  is a sign function. Then, the modified ITM loss is written as:
$$
\mathcal {L} _ {\mathrm {i t m}} = \mathbb {E} _ {(v, l) \sim D} \mathcal {H} \left(y _ {\text {i t m}}, p (v, l)\right) \tag {6}
$$
where  $\mathcal{H}$  denotes the cross-entropy,  $y_{itm}$  corresponds to the image-text matching label. Note that,  $s(\cdot)$  of the same triplets may vary on random masks and different tasks. Hence, we also modified other training objectives losses based on  $s(\cdot)$ , which are similar to  $\mathcal{L}_{\mathrm{itm}}$ . See more details in B.
3.4 Efficient Backpropagation for OOD Detection Layer
Having found  $\mu_k^*$ ,  $\sigma_k^*$  in the forward pass, we can view the initial part of our OOD network operator  $\mathcal{N}$  as the EM algorithm. It outputs the optimal parameters of ID feature density given by  $\mu$ ,  $\sigma$ . However, EM viewed as an operator from  $\mathbb{R}^d \to \mathbb{R}^d$  (mapping  $l_j \in \mathbb{R}^d$  to  $\mu_k^* \in \mathbb{R}^d$ ) makes backpropagation tricky since the Jacobian of such a map will be a  $\mathbb{R}^{d \times d}$  matrix, practically infeasible for training purposes even when  $d \approx 100$  is not very large. Hence, in this section, we describe how we adopted Jacoian-Free backpropagation for the implicit OOD detection layer.
Applying Chain Rule with Limited Unrolling for Gradients. Since each iteration in GMM as in Equation (2) is differentiable, we can easily backpropagate through few iterations of EM algorithm
to compute gradients. First we used Chain rule to illustrate the necessary components to obtain gradients. Note that from Equation (3), we know that  $\nabla_s\mathcal{L}\in \mathbb{R}$  is a scalar, and moreover for a fixed  $\mu$ ,  $s$  itself is a smooth scalar function  $s_\mu :\mathbb{R}^d\to \mathbb{R}$ . Thus, we consider the following composition of maps to compute  $\nabla_x\mathcal{L}\in \mathbb{R}^d$  gradient with respect to  $x_{i}$  (denoted generically as  $x$ ),
$$
x \longmapsto \mu^ {*} \quad \longmapsto s \mapsto \mathcal {L}
$$
$$
\mathbb {R} ^ {d} \xrightarrow {J _ {x , \mu^ {*}}} \mathbb {R} ^ {d} \longrightarrow \mathbb {R} \rightarrow \mathbb {R}, \tag {7}
$$
where  $\mu^{*}$  is the output of our OOD layer from forward pass. The key difficulty is in approximating the Jacobian  $J_{x,\mu^{*}}\in \mathbb{R}^{d}$  and/or its inverse for backpropagation since it maybe dense. We now describe three standard ways that can be used to backpropagate through implicit layers. First, in the vanilla backpropagation often called as "unrolling" is used where each iterate is stored, and a "path" gradient approximation is calculated. That is, since the parameters of each unrolled layer are fixed, we can simply use them along with the closed form gradients available for Equation (2) for backpropagation. Second, Jacobian based methods form an approximation to the Jacobian inverse or in other words, solve a linear system formed using the fixed point condition in Equation (2) itself. Often, this approach yields better gradients. Third, Jacobian-Free Backpropagation (JFB) combines both these approaches, that is, in which we simply use or unroll the last few iterates of the EM algorithm for gradients. As we can see, JFB enables to train with fixed memory costs  $(\mathcal{O}(1))$  and efficient backpropagation without computing gradients at each iterations. The time complexity is  $\mathcal{O}(n)$  for forward and  $\mathcal{O}(1)$  for backprop. We provide empirical results of the above three methods in Section 4.1.
Benefits of EM algorithm. EM updates are provably convergent in various settings, and it takes few iterations when it is guaranteed to converge [13]. This also implies that we can simply initialize  $\mu_{k}$  randomly and perform few more iterations to find a fixed point. However, since the denominator in Equation (2) contains terms that are also unknown parameters, such update schemes may be numerically unstable. In such cases, we can simply use recently proposed gradient based EM algorithms as the network operator, for fine-tuning deep networks in large scale settings, see [51] for convergence analysis.
Interaction via Outlier Detection. The applications which require a high number of image patches (or concepts), the likelihood that one of the patch features or text features to be an outlier also increases dramatically. In high dimensional settings, this increases the training time taken by first order methods significantly, especially when mini-batches are used to compute gradients [20]. Alternatively, when features  $l_{j}$  are computationally easy to extract, say using CLIP [49], it is reasonable to expect that a certain fraction of the  $l_{j}$  are outliers, and should not be used for backpropagation purposes. In a more optimistic scenario, we consider to customize our predictions, and handle "on-the-fly" integration of explicit knowledge. In our framework, this corresponds to treating  $\mu_{k}$  in Equation (3) as trainable parameters. We can update the initialization  $\mu_{k}$  without storing the trajectory, or forming the full Jacobian which can be expensive, as in our Algorithm 1.