SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Talia Konkle
Department of Psychology, Kempner Institute
Center for Brain Sciences
Cambridge, MA 02138
talia_konkle@harvard.edu
Vision Sciences Laboratory
alvarez@wjh.harvard.edu
Given the rich visual information available in each glance, humans can internally direct their visual attention to enhance goal-relevant information—a capacity often absent in standard vision models. Here we introduce cognitively and biologically-inspired long-range modulatory pathways to enable 'cognitive steering' in vision models. First, we show that models equipped with these feedback pathways naturally show improved image recognition, adversarial robustness, and increased brain alignment, relative to baseline models. Further, these feedback projections from the final layer of the vision backbone provide a meaningful steering interface, where goals can be specified as vectors in the output space. We show that there are effective ways to steer the model that dramatically improve recognition of categories in composite images of multiple categories, succeeding where baseline feed-forward models without flexible steering fail. And, our multiplicative modulatory motif prevents rampant hallucination of the top-down goal category, dissociating what the model is looking for, from what it is looking at. Thus, these long-range modulatory pathways enable new behavioral capacities for goal-directed visual encoding, offering a flexible communication interface between cognitive and visual systems.
In any given view there can be multiple kinds of objects present, each with a variety of features and properties. Humans have the capacity to direct their encoding of the visual world based on their internal goals, e.g. when looking for keys, flexible top-down attention mechanisms select and amplify the relevant key-like image statistics [1, 2, 3, 4, 5]. However, while standard deep neural network vision models can accurately classify objects, a key limitation of these visual encoding models is their inability to more deeply integrate with cognitive systems—that is, to encode the visual world in a goal-directed manner [6, 7]. Drawing on insights about top-down attention in both neuroscience and visual cognition, here we design and implement long-range modulatory (LRM) feedback connections that can be added to feed-forward deep neural network models, to enable goal-directed 'cognitive steering' of visual processing.
The idea of adding feedback connections and recurrent processing in deep neural network models is certainly not new (see [8] for review). However, this research typically focuses on characterizing the efficiency benefits of reusing features (e.g. [9, 10]), and does not explore the potential for goal-directed encoding. Other prior research aims more directly at adding goal-directed encoding into deep neural networks, whether using long-range feedback or alternative mechanisms (e.g. [7, 11, 12, 13, 14, 6]). However, these models typically operate over simpler datasets, require customized rather than flexible goal-directed encoding, and/or suffer from strong hallucinations of the goal-directed category
arising from strong additive attention signals. Thus, this present work uniquely leverages long-range modulatory feedback to enable goal-directed steering of visual processing.
Here we draw on research in the biological and cognitive sciences, which provides several constraints on how steerability might be implemented in deep neural network models. Anatomically, top-down attention relies on long-range feedback connections, which are extensively found in the neural system [15, 16, 17, 18]. For example, frontal (cognitive) areas project to object-responsive IT cortex, a key endpoint of the ventral visual encoding stream [19]. And within the ventral visual stream, there are extensive feedback connections, with prominently studied pathways from IT cortex to V4, and from V4 to V1. Indeed, recent evidence using optogenetic suppression of feedback in non-human primates demonstrated that goal-directed attention is causally dependent on top-down feedback [20]. Further, cognitive research has shown that feature-based attention obligatorily operates over the full-field (e.g. if you want to attend to faces on the left side of the image, faces on the right side of the image will also be amplified [21, 22, 23, 24, 25]). We leverage these known characteristics of the biological system to guide our implementation of long-range modulatory feedback projections.
We first introduce long-range modulatory feedback models (LRM models), and describe the details of their design, focusing on their distinctive multiplicative modulatory motif (Fig. 1). Next, we probe the default dynamical behavior of these models. Finally, we show that these learned feedback pathways can be used to enable cognitive steering, and we explore which kinds of steering vectors are most effective at recognizing objects in multi-class composite images.
The main contributions of this paper are as follows. (1) Models with long-range modulatory feedback pathways (LRM models) naturally learn more accurate and adversarially-robust representations, relative to a matched baseline model. (2) Feedback projections from the source layer provide a meaningful steering interface, and there are effective ways to steer the model which dramatically improve recognition of categories in composite images. (3) The multiplicative modulatory motif prevents rampant hallucinations/false alarms when the goal category is not present. (4) These LRM models learn feature tuning that is naturally more aligned with neurophysiological data; and with active steering, these models qualitatively reproduce effects of top-down category-based attention observed in human brain responses. Broadly, these results demonstrate how long-range modulatory feedback pathways allow for different goal states to make flexible use of fixed visual circuitry, supporting dynamic goal-based routing of incoming visual information.
2 Models with Long-Range Modulatory (LRM) Feedback Projections
We developed long-range modulatory (LRM) projections, implemented in PyTorch as a wrapper that can augment any PyTorch model with the ability for any source layer to modulate the activity of any other destination layer<sup>1</sup>. We then outfitted standard Alexnet architectures with a variety of different feedback pathways (Sec 2.1). We detail how these networks operate in the next three sections (Fig. 1). Sec 2.2 describes the macro-scale flow of visual information through an LRM model, clarifying when incoming feedback signals influence the next processing step of any given layer. Sec 2.3 digs into the details of how this feedback signal is computed, and what is learned. Sec 2.4 covers how these models were trained with a two-step loss function.
2.1 Architectures
Which layer-to-layer feedback pathways should be included? There is a large space of possible architectures, even in a simple 8-layer Alexnet. If we were to draw on insight from the neuroanatomy of the ventral stream, all layers should project back on all earlier layers (e.g. [17]). Here, we started more simply, implementing three progressively rich long-range modulatory models-LRM1, LRM2, and LRM3-with an increasing number of feedback pathways. The LRM1 model includes modulatory pathways  $(Output\rightarrow Conv4)$  and  $(Conv4\rightarrow Conv1)$ , loosely inspired by the known prominent connections from IT to V4 and from V4 to V1. The LRM2 model adds  $(Output\rightarrow Conv5)$  and  $(Conv5\rightarrow Conv2)$ . The LRM3 model further adds  $(FC6\rightarrow Conv3)$ . In all cases, the source of feedback was a relu stage (output of a layer block), and the destination was a conv stage (pre-relu). Critically, the output layer is the source of some of these feedback projections, which will later be important for cognitive steering.
$^{1}$ Code available here: https://github.com/harvard-visionlab/lrm-steering
Figure 1: Schematic of long-range modulatory (LRM) network architecture. The image is processed in the initial feed-forward pass, and then source layers modulate destination layers in the modulated pass. Learnable parameters of the modulatory motif in bold.
2.2 Long-range modulatory pathway dynamics
In the standard feed-forward operation of the base vision network, the flow of the information proceeds sequentially from the input layer to output layer. In a model with LRM projections, the flow of information through the network can be best understood as consecutive feed-forward passes.
In the initial feed-forward pass (pass  $= 0$ ), the output of layer  $L$  is:
$$
x _ {p} ^ {\text {o u t}} = L \left(x _ {p} ^ {\text {i n}}\right) \tag {1}
$$
After this pass, layers that are the source of feedback pathways store their output activations  $x_{p}^{out}$ , to influence destination layers on the next forward pass. During the next feed-forward pass of the image—also referred to here as the modulated pass—each layer first computes its typical feedforward output. Then, the layer incorporates feedback influences coming from any sources in the prior pass  $(F_{p})$ , with the following multiplicative effect:
$$
x _ {p + 1} ^ {\text {o u t}} = L \left(x _ {p + 1} ^ {\text {i n}}\right) + L \left(x _ {p + 1} ^ {\text {i n}}\right) \cdot F _ {p} \tag {2}
$$
With this motif, the source layers provide a multiplicative, modulatory influence on the destination layer's feed-forward signal. In the absence of feedback drive, the feed-forward signal is unchanged. And, in the absence of input, the feedback pathways do not drive activations through the network.
2.3 Modulatory motif
How are the source layer activations transformed to modulate the destination layers on the next pass  $(F_p)$ ? The modulatory motif we implemented is motivated by the hypothesis that there are meaningful relationships between channels in the source layer and the destination layer (e.g. [26]). The motif is schematized in Fig. 1 and detailed mathematically in Appendix A.1, and summarized below.
First, source layer activations undergo a series of normalizing and scaling transformations to control and align their modulatory influence with the destination layer. Briefly: (1) A channel normalization operation serves to emphasize some features, and de-emphasize others, over a given image, with learnable normalization parameters. (2) To keep the multiplicative modulation values in a controlled range, normalized activations are next passed through a tanh function. We include two learnable scalars  $\sigma+$  and  $\sigma-$ , to enable the model to learn how to best scale the positive and negative modulatory
influences, respectively. (3) Next, the size of the source activation map is aligned to the size of the destination map. (If the source layer is fully-connected and the destination layer is a convolutional layer, the channel activations are spatially broadcast across the full activation map. If both the source and destination layers are convolutional, the source activation map is spatially up-sampled to align with the size of the destination activation map.)
Next, the key transformations carried out in these LRM pathways are the channel-to-channel influences that are learned between the source features and the destination features. This stage was implemented through a 1x1 convolutional layer, and outputs a feedback map  $F$  with the same size as the destination layer activation map. To give an intuition, these channel-to-channel weighting parameters enable long-range projections to learn that some feature channels in the source layer should amplify particular feature channels in the earlier stage, and suppress other feature channels, to help correctly classify the image on the next pass through the network.
2.4 Loss Function and Training
All LRM models were trained on the ImageNet dataset [27, 28] on the task of image classification. The total loss was calculated as the average of the cross-entropy  $CE$  loss on the predictions of the initial feed-forward pass  $y_{p=0}^{\prime}$  and one modulated pass  $y_{p=1}^{\prime}$ , given ground truth labels  $y$ :
$$
L = \frac {1}{2} \left(C E \left(y, y _ {p = 0} ^ {\prime}\right) + C E \left(y, y _ {p = 1} ^ {\prime}\right)\right) \tag {3}
$$
The motivation for this two-term loss function is that we wanted the base encoder to learn an effective set of visual features for pure feed-forward operation (as is true of the biological system). And, we wanted to ensure that learned feedback modulation routing could further reinforce the feed-forward operation for successful object classification. Note that in this work, only two passes were ever trained (one initial feed-forward pass, and one modulated pass).
Models were trained using the Fast-Forward Computer Vision (FFCV) library [29]. We used a one-cycle policy [30] over 100 epochs, with  $1\mathrm{e} - 4^{*}$  peak-learning-rate as the initial rate, 0.10 as a the peak at epoch 15, approaching 0 as the final rate. We used a batch size of 2048, a weight decay of 5e-5, and momentum of 0.9, with the SGD optimizer. Models were trained on an internal computer cluster with 48 cores, 500GB of system RAM, and 4 A100 GPUs with 40GB RAM each. LRM-Alexnet models train in less than 8 hours on this hardware.
3 Model Performance During Default Operation
First, we characterize the performance of these LRM models during their default modulatory operation. Note there is no goal-based cognitive steering yet—the model simply is carrying out its default modulatory feedback processes at inference time.
3.1 LRM models have increased accuracy and robustness
How well do the LRM models classify images on their feed-forward pass and modulated passes, and how does that compare to a baseline model without LRM pathways? Figure 2A shows top-1 classification accuracy, plotted for all three LRM models, as a function of modulatory passes tested at inference. Note that all LRM models were trained on only the initial pass (pass 0) and first modulatory pass (pass 1), but we allowed further modulatory passes at inference to explore these effects.
All LRM models achieved higher accuracy than a comparison Alexnet backbone architecture, trained with the same recipe, but without any LRM pathways (dashed gray line). In addition, in all models, the modulated passes had higher accuracy than the initial feed-forward pass, with accuracy plateauing after the second modulatory pass. An implication of this result is that some images are initially incorrectly classified, but with default feedback modulation on the next pass, the image maximally activates an alternate, correct class. We show an example of such a case, using the AblationCam to observe the shifting evidence between the incorrect and correct classification, in Fig. 3).
Figure 2: LRM model default behavior. (A) Imagenet Accuracy plotted for the three LRM models, as a function of initial and increasing modulatory passes. (B) Adversarial robustness is plotted for the LRM3 model, as a function of epsilon. (C) Percent improvement in adversarial robustness, relative to the standard Alexnet backbone, is plotted for all three LRM models. Dashed gray lines in (A), and (B) indicate performance of a matched Alexnet baseline, with no LRM pathways.
Figure 3: Example of modulatory-pass improvement. Image is initially incorrectly classified as a "fig", and then correctly classified as "ringlet" (moth). AblationCam [31] shows the evidence with respect to these two output target units. Initially the depicted leaves dominate the classification, with some evidence for "ringlet" focused on the depicted moth. During the modulated pass, however, the "fig" output unit no longer depends on the depicted moth in the image, resulting in the "ringlet" becoming the maximally activated output unit.
Note that LRM models have more trainable parameters, which could underlie this difference in accuracy. However, even the 0th (feed-forward) pass of the LRM models was slightly but consistently higher than the baseline Alexnet, indicating that the LRM models actually have learned better feed-forward features than the matched architecture model without feedback projections.
In addition, we also found that LRMs have increased robustness to adversarial attacks, using Fast Gradient Sign Attack [FGSM 32]. Fig. 2.B shows the LRM3 model robustness, plotted for the feedforward pass and the first two modulatory passes. Fig. 2.C summarizes the robustness effect for all models (summed AUC over the epsilon range), plotted as a percentage increase in robustness, relative to the control Alexnet model. These results demonstrate that the addition of long-range modulatory pathways naturally confer adversarial robustness in their default modulatory operation.
Why is the modulatory pass so effective? We hypothesize that learned feedback connections are reinforcing the implicit routes that images take through the computational hierarchy (e.g. [33, 26, 34]). If images are ambiguous between multiple classes at the later source stage, normalization-based competition may shift the evidence about which class is present. Learned feedback signals then amplify relevant features and suppresses irrelevant features in the earlier stages, to steer the input along different routes in the modulated pass. An alternative possibility, or framing, is that top-down connections provide a holistic view of the scene, across both spatial and channel dimensions, and
serve to edit early activations to be most consistent with the current, holistic high-level representation (see [35] for related ideas on hierarchical Bayesian inference in visual cortex).
3.2 LRM models have higher Brain-Scores
To examine whether these brain-inspired feedback projections also yield better representational alignment between the model and the brain data, the LRM3 model was submitted to the Brain-Score benchmark [36], and results are reported in Table 1. We find that the LRM3 model shows much improved predictivity of the later stage of the ventral visual cortex (IT), dramatically better than PyTorch default Alexnet (e.g. model rank  $145 \rightarrow 35$  across submitted models, at the time of this analysis). More generally, the LRM3 model with default feedback operation shows a better fit to three of the four visual brain regions, both in its feedforward pass and its modulated pass, compared to the PyTorch default Alexnet model.
Brain Area	Baseline Alexnet	LRM3 (pass 0)	LRM3 (pass1)	Δ (from baseline)	rank change
IT	r = 0.358	r = 0.393	r = 0.400	+0.042	#145 → #35
V4	r = 0.443	r = 0.454	r = 0.467	+0.024	#153 → #97
V2	r = 0.353	r = 0.341	r = 0.333	-0.020	#13 → #48
V1	r = 0.507	r = 0.492	r = 0.531	+0.024	#68 → #32
Table 1: Brain-Score results for the Baseline Alexnet model and LRM3 model. The r-value indicates the average single-unit neuron predictivity scores, reported for different visual areas along the ventral visual stream hierarchy.
4 Model Performance with Cognitive Steering
We next show that these learned feedback pathways can be co-opted for goal-directed cognitive steering. The underlying idea is intuitive: the output layer is the source of feedback projections that have already learned how to effectively modulate feed-forward visual processing. And, the output latent space is interpretable, and thus can be thought of as cognitively-accessible steering interface. In this 1000-d space, different vectors  $v$  can be meaningfully interpreted as steering towards (or away) from different categories (Fig. 4). During goal-directed cognitive steering, we simply intercept the default feedback sent from the output source projections, and instead, inject a different output activation pattern to drive the modulatory feedback pathways.
Note that a premise of cognitive steering is that you have a goal or target in mind. As an example, the ongoing cognitive task may next require the agent to look for a key. In this case, the cognitive system may retrieve the "prototypical" output vector for the "key" class from memory, which could be used as a top-down steering vector, and be injected into the feedback projections. We predict that cognitive steering should allow the visual encoder to amplify any responses for key-like features that are present, enabling better detection. And, because the feedback is purely modulatory, this motif should prevent the model from hallucinating keys everywhere.
4.1 Composite Image Recognition Challenge
To test these ideas systematically, we used a composite image recognition task, inspired by cognitive science research testing effects of top-down feature based visual attention [37, 5]). Specifically, composite images were created, consisting of two images of different classes either presented side-by-side, or directly overlaid on top of each other and averaged. The logic here is that these composite images will be hard for models to classify in typical feed-forward operation, but either class will be accurately recognized with corresponding cognitive steering. And, conversely, if we cognitively steer the model toward a category that is not present in the composite images, the model should not hallucinate that the goal-category is present (i.e. false alarm).
First, we created a controlled composite image test, using the 'Imagenette' subset of 10 categories to create a set of image triplets (3,910 image triplets). Each triplet was created by randomly drawing 3 images from 3 distinct categories (without replacement). The first two of the images were used to form a composite image with two "target-present" categories, by either placing them side-by-side
Figure 4: Cognitive Steering. Feedback pathways from the output layers are used for goal-directed influence over the modulated pass. The output latent space serves as a cognitive-steering interface, where any directional vector  $(v)$  serves as the guidance signal. The gain of the modulatory signal can be controlled with  $\alpha$ -scaling.
or overlaid (Fig. 5.A). The third image from the triplet was kept aside to be used a "target-absent" category to test for steering-induced hallucinations, i.e. whether steering towards a non-present category leads the model to report a category that was not present in the image.
The baseline Alexnet model is very poor at identifying either of the two classes of the composite images correctly: side-by-side composite recognition  $= 23.6\%$ ; overlay composite recognition  $= 1.0\%$  (top-1 accuracy). It can, however, very accurately classify these images when presented individually: top-1 accuracy  $= 77.0\%$ . Thus, these composite images provide a good challenge for testing whether cognitive steering can improve classification, and to what degree.
4.2 Cognitive Steering Variations
How should a cognitive system effectively steer the models—that is, what vector should be used as the source of the feedback to the vision backbone, to enable the most accurate classification of a category in these composite images? We designed and compared a variety of steering signals, reporting only a subset here for clarity (see also Appendix A.5). Here we focus on several forms of target-based steering, which aim to steer the model towards a representation of the target of interest, with different possible levels of granularity:
- One-hot steering, using a 1000-d vector with a 1 on the target category unit.
- Instance-steering, using the 1000-d activation profile of the output layer when the network processes that exact target image, in isolation.
- Category-prototype steering, using a 1000-d vector of the average output activation, over all exemplars of the target category.
- Language-based category steering, leveraging a diffusion-prior model [38] to map between CLIP-text outputs and the LRM visual embedding space (see Appendix A.3), providing a text-based prediction of the output activation expected for the target category from the vision model.
Each of these steering signals operationalizes a different high-level query for how the cognitive system can interact with the visual system to guide visual encoding (e.g. "look for this specific image" or "look for anything from category X").