Cognitive Steering in Deep Neural Networks via Long-Range Modulatory Feedback Connections
Authors: Talia Konkle
George A. Alvarez
Venue: NeurIPS 2023 poster (2023)
Keywords: convolutional neural networks
steerability
computer vision
TL;DR: Introducing long-range modulatory feedback connections to deep neural networks enables cognitive-guided visual encoding

Abstract: Given the rich visual information available in each glance, humans can internally direct their visual attention to enhance goal-relevant information---a capacity often absent in standard vision models.  Here we introduce cognitively and biologically-inspired long-range modulatory pathways to enable `cognitive steering’ in vision models.  First, we show that models equipped with these feedback pathways naturally show improved image recognition, adversarial robustness, and increased brain alignment, relative to baseline models. Further,  these feedback projections from the final layer of the vision backbone provide a meaningful steering interface, where goals can be specified as vectors in the output space.  We show that there are effective ways to steer the model that dramatically improve recognition of categories in composite images of multiple categories, succeeding where baseline feed-forward models without flexible steering fail. And, our multiplicative modulatory motif prevents rampant hallucination of the top-down goal category, dissociating what the model is looking for, from what it is looking at. Thus, these long-range modulatory pathways enable new behavioral capacities for goal-directed visual encoding, offering a flexible communication interface between cognitive and visual systems.

Cognitive Steering in Deep Neural Networks via Long-Range Modulatory Feedback Connections
Talia Konkle
George Alvarez
Department of Psychology, Kempner Institute Center for Brain Sciences
Department of Psychology. Kempner Institute
Vision Sciences Laboratory
Harvard University
Harvard University
Cambridge, MA 02138
Cambridge, MA 02138
edu
al alvareeuwjh http://www.hh..arddc.aaao www.aaraaad....... edu
Abstract
Given the rich visual information available in each glance, humans internally direct their visual attention to enhance goal-relevant information a capacity often absent in standard vision models. Here introduce cognitively and biologically- inspired long-range modulatory pathways to enable 'cognitive steering" in vision models. First, show that models equipped with these feedback pathways natu- rally show improved image recognition, adversarial robustness, and increased brain alignment, relative baseline models. Further, these feedback projections from the final layer of the vision backbone provide meaningful steering interface, where goals can be specified as vectors the output space. We show that there are effec- tive ways to steer the model that dramatically improve recognition of categories in composite images of multiple categories, succeeding where baseline feed-forward models without flexible steering fail. And, our multiplicative modulatory motif prevents rampant hallucination of the top-down goal category, dissociating what the model is looking for, from what it is looking at. Thus, these long-range modula- tory pathways enable new behavioral capacities for goal-directed visual encoding, offering flexible communication interface between cognitive and visual systems.
Introduction
In any given view there can be multiple kinds of objects present, each with a variety of features and properties. Humans have the capacity to direct their encoding of the visual world based on their internal goals, e.g. when looking for keys, flexible top-down attention mechanisms select and amplify
the relevant key-like image statistics i 4 5. However, while standard deep neural network vision models can accurately classify objects, a key limitation of these visual encoding models is their inability to more deeply integrate with cognitive systems-that is, to encode the visual world in a goal-directed manner [6] 7 Drawing insights about top-down attention in both neuroscience and visual cognition, here we design and implement long-range modulatory (LRM) feedback connections that be added fordadorward deep neural network models, enable goal-directed 'cognitive steering' of visual processing.
The idea of adding feedback connections and recurrent processing in deep neural network models is certainly not new (see [8] for review). However, this research typically focuses on characterizing the efficiency benefits of reusing features (e.g. I0), and does not explore the potential for goal-directed encoding. Other prior research aims more directly at adding goal-directed encoding into deep neural networks, whether using long- range feedback or alternative mechanisms (c.g. 12 13 [4] 6)). However, these models typically operate over simpler datasets, require customized rather than flexible goal-directed encoding, and/or suffer from strong hallucinations of the goal-directed category
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

arising from strong additive attention signals. Thus, this present work uniquely leverages long-range modulatory feedback to enable goal- steering of visual processing.
Here we draw on research in the biological and cognitive sciences, which provides several constraints on how steerability might be implemented in deep neural network models. Anatomically, top-down attention relies on long-range feedback connections, which are extensively found in the neural system [5 I6 [8]. For example, frontal (cognitive) areas project to object-resoonsive IT cortex, key endpoint of the ventral visual encoding stream [9]. And within the ventral visual stream, there are extensive feedback connections, with prominently studied pathways from IT cortex to V4, and from V4 to V1. Indeed, recent evidence using optogenetic suppression of feedback in non-human primates demonstrated that goal goal-directed attention is causally dependent on top-down feedback [20]. Further, cognitive research has shown that reature-based attention obligatorily operates over the full-field (e.g. you want to attend to faces on the left side of the image, faces on the right side of the image will also be amplified [21] 22 23 24, 25.). We leverage these known characteristics of the biological system to guide our implementation of long-range modulatory feedback projections.
We first introduce long-range modulatory feedback models (LRM models), and describe the details of their design, focusing on their distinctive multiplicative modulatory motif (Fig. D. Next, we probe the default dynamical behavior of these models. Finally, we show that these learned feedback pathways can be used to enable cognitive steering, and we explore which kinds of steering vectors are most effective at recognizing objects in multi-class composite images.
The main contributions of this paper are as follows. (1) Models with long-range modulatory feedback pathways (LRM models) naturally learn more accurate and adversarially-robun representations, relative matched baseline model. (2) Feedback projections from the source layer provide meaningful steering interface, and there are effective ways to steer the model which dramatically improve recognition of categories in composite images. (3) The multiplicative modulatory motif prevents rampant hallucinations/falee alarms when the goal category is not present. (4) These LRM models learn feature tuning that is naturally more aligned with neurophysiological data; and with active steering, these models qualitatively reproduce effects of top-down category- baased attention observed in human brain responses. Broadly, these results demonstrate how long- rraggraae modulatory feedback pathways allow for different goal states to make flexible use of fixed visual circuity, supporting dynamic goal-based routing of incoming visual information.
2 Models with Long-Range Modulatory (LRM) Feedback Projections
We developed long-range modulatory (LRM) projections, implemented in PyTorch as a wrapper that augment any PyTorch model with the ability for any source layer to modulate the activity of any other destination layer" We then outfitted standard Alexnet architectures with a variety of different feedback pathways (Sec 2.1). We detail how these networks operate in the next three sections (Fig.
D. Sec [2.2 describes the macro-scale flow of visual information through an LRM model, clarifying when incoming feedback signals influence the next processing step of any given layer. Sec 2.3 digs
into the details of how this feedback signal is computed, and what is learned. See 2.4 covers how these models were trained with a two-step loss function.
2.1 Architectures
Which -lyyer feedback pathways should be included? There is large space of possible architectures, even in a simple 8-layer Alexnet. If we were to draw on insight from the neuroanatomy of the ventral stream, all layers should project back on all carlier layers (c.g. (17). Here, we started more simply, implementing three progressively rich long-range modulatory models-LRM1, LRM2, and -Rt3-with increasing number of feedback pathways. The LRMI model includes modulatory pathways (Output - Conu4) and (Conv4 -> 1)n loosely inspired by the known prominent connections from IT V4 and from V4 to VI. The LRM2 model adds (Oulpul + Conw5) and (Comc5 - Conw2). The LRM3 model further adds (FC6 - Cono3). In all cases, the source of feedback was relu stage (output of a layer block), and the destination was conv stage (pre-relu). Critically, the output layer is the source of some of these feedback projections, which will later be important for cognitive steering.
'Code available here: http::///ttbbbbooom/maaaaaariilt

---

Long-range modulatory networks
feed forward pass
modulated pass
modulatory motif
Destination layer
Xp Xput Fp
Source layer
norm tanh scale resize
Ft
Conv
modulatory signal
channel-to-channel influences
control
parameters
Figure 1:
Schematic long-range modulatory (LRM) network architecture. The image is processed in the initial feed-forward pass, and then source layers modulate destination layers in the modulated pass. Learnable parameters of the modulatory motif in bold.
2.2 Long-range modulatory pathway dynamics
In the standard feed-forward operation of the base vision network, the flow of the information proceeds sequentially from the input layer output layer. In model with LRM projections, the flow of information through the network can be best understood as consecutive feed passes.
In the initial feed-forward pass (pass 0), the output of layer L is:
out
P = L(xp")
After this pass, layers that are the source of feedback pathways store their output activations xr to influence destination layers on the next forward pass. During the next feed-forward pass of the image-also referred to here as the modulated pass-cach layer first computes its typical feed- forward output. Then, the layer incorporates feedback influences coming from any sources in the prior pass (Fp), with the following multiplicative effect:
x
++1
L(
(2)
With this motif, the source layers provide multiplicative, modulatory influence on the destination layer's feed-forward signal. In the absence of feedback drive, the feed-forward signal unchanged. And, in the absence of input, the feedback pathways do not drive activations through the network.
2.3 Modulatatory motif
How are the source layer activations transformed to modulate the destination layers on the next pass (F,)? The modulatory motif implemented is motivated by the hypothesis that there meaningful relationships between channels in the source layer and the destination layer (e.g. (26)). ). The motif is schematized in Fig. and detailed mathematically in Appendix A.I and summarized below.
First, source layer activations undergo series of normalizing and scaling transformations control and align their modulatory influence with the destination layer. Briefly: (1) A channel normalization operation serves to emphasize some features, and de-emphasize others, over a given image, with learnable normalization parameters. (2) To keep the multiplicative modulation values in a controlled range, normalized activations are next passed through a lanh function. We include two learnable scalars o+ and to enable the model to learn how to best scale the positive and negative modulatory

---

influences, respectively. (3) Next, the size of the source activation map is aligned to the size of the destination map. (If the source layer fully-connected and the destination layer a convolutional layer, the channel activations are spatially broadcast across the full activation map. If both the source and destination layers are convolutional, the source activation map is spatially up-sampled to align with the size of the destination activation map.)
Next, the key transformations carried out in these LRM pathways are the channel-to-chanee influences that are learned between the source features and the destination features. This stage was implemented through a Ixl convolutional layer, and outputs feedback map with the same size as the destination layer activation map. To give an intuition, these shannel-to-chame parameters enable long-range projections should amplify particular feature channels in the earlier stage, and suppress other feature channels, to help correctly classify the image on the next pass through the network.
weighting learn that some feature channels in the source layer
2.4 Loss Function and Training
the task of image classification. The total loss was calculated as the average of the cross-entropy CF loss on the predictions of the initial feed-forward pass V/= and one modulated pass 1 given ground truth labels
All LRM models were trained on the ImageNet dataset 27 28
1 L 2 (CCE(V. + CCU CE(U.
The motivation for this two-term loss function is that we wanted the base encoder to learn an effective of visual features for pure feed-forward operation (as is true of the biological system). And, we wanted to ensure that learned feedback modulation routing could further reinforce the feed-forward operation for successful object classification. Note that in this work, only two passes were ever trained (one initial feed forward pass, and one modulated pass).
Models were trained using the Fast-Forward Computer Vision (FFCV) library 22 We used one-cycle policy (30] over 100 epochs, with le-4 * peak-learning-rate as the initial rate, 0.10 as the peak epoch 15, approaching O as the final rate. We used batch size of 2048, weight decay of 5e-5, and momentum of 0.9, with the SGD optimizer. Models were trained on an internal computer cluster with 48 cores, 500GB of system RAM, and 4 A100 GPUs with 40GB RAM each. LRM-Alexnet models train in less than 8 hours on this hardware.
3 Model Performance During Default Operation
First, we characterize the performance of these LRM models during their default modulatiory operation. Note there no goal- goal-based cognitive steering yet-the model simply carrying out its default modulatory feedback processes at inference time.
3.1 LRM models have increased accuracy and robustness
How well do the LRM models classify images on their feed-forward pass and modulated passes, and how does that compare to a baseline model without LRM pathways? Figure shows top-1 classification accuracy, plotted for all three LRM models, as a function of modulatory passes tested at inference. Note that all LRM models were trained only the initial pass (pass 0) and first modulatory pass (pass 1), but we allowed further modulatory passes at inference to explore these effects.
All LRM models achieved higher accuracy than comparison Alexnet backbone architecture, trained with the same recipe, but without any LRM pathways (dashed gray line). In addition, all models, the modulated passes had higher accuracy than the initial feed-forward pass, with accuracy plateauing after the second modulatory pass. An implication this result is that some images are initially incorrectly classified, but with default feedback modulation on the next pass, the image maximally activates an alternate, correct class. We show an example of such a case, using the AblationCam to observe the shifting evidence between the incorrect and correct classification, in Fig. 3).

---

A
LAM3
1.0
C
LAM2 LRM1
Modulatory Pass
imagene' accuracy
imagenet accuracy
0.8
AUC % increase relative to baseline
30
(top5 %
0.6
alexnet baselne
0.4
20
0.2-
with modulatory feedback
50
0.0
modulatory pass number
0.00
0.02
epsilon
0.04
0.06
LRM1
LRM2
LRM3
Figure 2: LRM model default behavior. (A) Imagenet Accuracy plotted for the three LRM models, as a function of initial and increasing modulatory passes. (B) Adversarial robustness is plotted for the LRM3 model, as function of epsilon. (C) Percent improvement in adversarial robustness, relative to the standard Alexnet backbone, is plotted for all three LRM models. Dashed gray lines (A), and (B) indicate performance of matched Alexnet baseline, with no LRM pathways.
"Fig" (arror) "ringlet" (correct)
Pass D: target="fig "fig"
Pass target="fig"
Pass "rrget=""inglatt
Pass 1: target= "ringlet"
Figure 3: Example of modulatory pass improvement. Image is initially incorrectly classified as "fig", and then correctly classified as "ringlet" (moth). AblationCam [31 shows the evidence with respect to these two output target units. Initially the depicted leaves dominate the classification, with some evidence for "ringlet" focused on the depicted moth. During the modulated pass, however, the "fig" output unit no longer depends on the depicted moth in the image, resulting in the "ringlet" becoming the maximally activated output unit.
Note that LRM models have more trainable parameters, which could underlie this difference in accuracy. However, even the Oth ffeed-forward pass of the LRM models was slightly but consistently higher than the baseline Alexnet, indicating that the LRM models actually have learned better feed-forward features than the matched architecture model without feedback projections.
In addition, also found that LRMs have increased robustness to adversarial attacks, using Fast Gradient Sign Attack [FGSM 32). Fig. B shows the LRM3 model robustness, plotted for the feedforward pass and the first two modulatory passes. Fig. C summarizes the robustness effect for all models (summed AUC over the epsilon range), plotted as a percentage increase in robustness, relative the control Alexnet model. These results demonstrate that the addition of long-range modulatory pathways naturally confer adversarial robustness in their default modulatory operation.
Why is the modulatory pass so effective? We hypothesize that learned feedback connections are reinforcing the implicit routes that images take through the computational hierarchy (e.g. [33 26 34)). If images are ambiguous between multiple classes at the later source stage, normalization based competition may shift the evidence about which class is present. Learned feedback signals then amplify relevant features and suppresses irrelevant features in the earlier stages, steer the input along different routes in the modulated pass. alternative possibility, or framing, that top-down connections provide holistic view of the scene, across both spatial and channel dimensions, and
5

---

serve to edit early activations to be most consistent with the current, holistic high-level representation (see [35 for related ideas on hierarchical Bayesian inference visual cortex).
3.2 LRM models have higher Brain-Scores
To examine whether these brain-inspired feedback projections also yield better representational alignment between the model and the brain data, the LRM3 model was submitted the Brain-Score benchmark [36], and results are reported in Table [ We find that the LRM3 model shows much improved predictivity of the later stage of the ventral visual cortex (IT), dramatically better than PyTorch default Alexnet (e.g. model rank 145 + 35 across submitted models, at the time of this analysis). More generally, the LRM3 model with default feedback operation shows a better fit to three of the four visual brain regions, both in its feedforward pass and its modulated pass, compared the PyTorch default Alexnet model.
Brain Arca
Baseline Alexnet
LRM3 (pass 0)
LRM3 (passl)
A (from baseline)
rank change
IT
0.358
r= 0.393
0,400
+0.042
#145 #35
V4
r= 0.443
r= 0,454
r= 0.467
+0.024
#153 - #97
V2 VI
r= 0.353
r= 0.341
r= 0.333
-0.020
#13 + #-48
r= 0.507 r= 0.492
r= 0.531
+0.024
#68 + #32
Table Brain- results for the Baseline Alexnet model and LRM3 model. The r-value indicates the average single-unit neuron predictivity scores, reported for different visual areas along the ventral visual stream hierarchy.
4 Model Performance with Cognitive Steering
We next show that these learned feedback pathways can be co-opted for goal-directed cognitive steering. The underlying idea is intuitive: the output layer is the source of feedback projections that have already learned how to effectively modulate feed-forward visual processing. And, the output latent space interpretable, and thus can be thought of as cognitively-accessibbl steering interface. In this 1000-d space, different vectors U can be meaningfully interpreted steering towards (or away) from different categories (Fig. 4 During goal-directed cognitive steering, we simply intercept the default feedback sent from the output source projections, and instead, inject different output activation pattern to drive the modulatory feedback pathways.
Note that a premise of cognitive steering is that you have goal or target in mind. As example, the ongoing cognitive task may next require the agent to look for a key. In this case, the cognitive system may retrieve the "prototypical" output vector for the "key" class from memory, which could be used as a top-down steering vector, and be injected into the feedback projections. We predict that cognitive steering should allow the visual encoder to amplify any responses for key-like features that are present, enabling better detection. And, because the feedback is purely modulatory, this motif should prevent the model from hallucinating keys everywhere.
4.1 Composite Image Recognition Challenge
To test these ideas systematically, we used a composite image recognition task, inspired by cognitive science research testing effects of top-down feature based visual attention [37 5)). Specifically, composite images were created, consisting of two images of different classes either presented side- by-side, or directly overlaid on top of each other and averaged. The logic here is that these composite images will be hard for models to classify in typical feed-forward operation, but either class will be accurately recognized with corresponding cognitive steering. And, conversely, if we cognitively steer the model toward a category that is not present in the composite images, the model should not hallucinate that the goal-category is present (i.e. false alarm).
First, we created a controlled composite image test, using the 'Imagenette' subset of 10 categories to create set of image triplets (3,910 image triplets). Each triplet was created by randomly drawing 3 images from distinct categories (without replacement). The first two of the images were used to form composite image with two "target-present" categories, by either placing them side-by-side

---

Cognitive Steering
steering interface
layer
output space
feed forward pass
goal-modulated pass
dov
gain control
Figure 4: Cognitive Steering. Feedback pathways from the output layers are used for goal-directed influence over the modulated pass. The output latent space serves as a cognitive-steering interface, where any directional vector (v) serves as the guidance signal. The gain of the modulatory signal can be controlled with a-scaling.
or overlaid (Fig. A). The third image from the triplet was kept aside to used "target-absent" steering-induced
category to test for steering induced hallucinations, i.c. whether steering towards non-present category leads the model to report a category that was not present in the image.
The baseline Alexnet model is very poor at identifying either of the two classes of the composite images correctly: side-by-side composite recognition = 23.6% overlay composite recognition 1.0% (top-1 accuracy). It can, however, very accurately classify these images when presented individually: top-1 accuracy 77.0%. Thus, these composite images provide good challenge for testing whether cognitive steering can improve classification, and to what degree.
4.2 Cognitive Steering Variations
How should cognitive system effectively steer the models-that is, what vector should be used the source of the feedback to the vision backbone, to enable the most accurate classification of a category in these composite images? We designed and compared a variety of steering signals, reporting only a subset here for clarity (see also Appendix A.5). Here we focus on several forms of target-based steering, which aim steer the model towards a representation of the target of interest, with different possible levels of granularity:
One-hot steering, using a 1000-d vector with a on the target category unit.
Instance-ste steenncestterinn, using the 1000-d activation profile of the output layer when the network processes that exact target image, in isolation.
Category-protoryy steering, using a 1000-d vector of the average output activation, over all exemplars of the target category.
Language- -aaee category steering, leveraging diffusion-pri -prior model 38 to map between CLiP-text outputs and the LRM visual embedding space (see Appendix A.3), providing a text-based prediction of the output activation expected for the target category from the vision model.
Each of these steering signals operationalizes a different high-level query for how the cognitive system can interact with the visual system to guide visual encoding (e.g. "look for this specific image" or "look for anything from category X").
We also explored whether we could vary steering strength. LRM models have learned o and scaling parameters that control the magnitude of modulation. We probed whether scaling these values by a has systematic effects on the strength of the cognitive steering influence (a a+, a a for 2, 3, 4, Fig. 5 B). In human vision, this factor is loosely equivalent to generalized cognitive effort, and our capacity to marshall increasing resources to amplify top-down effects [39].
"The side- images have 2x the horizontal pixel resolution. Our model uses PyTorch's 's standard, built-in Adaptive Average Pooling operation at the final convolution stage to adjust the size of the representation to match the size expected by the cullecco layers.
7

---

ar scaling
Pass a
c²
Pass 1, a
Side-by-Side
Overlay
Target- Absent
Pass a
Example Image Triplet
Gain Modulation
Pass 1, a
1-Hot Steering
mnstance-Steering
Category-Steerin
CLIP-Steering
Accuracy
Target
Target Abseett
Target present
Target Absent
Target present
Target Absient
Target present
Tarpet Absent
present
8
6-
-6-
Accuracy
-2
.2
Target
Target
Target
Target
Target
Target
Target
Target
present
Absent
present
present
Aboent
present
Figure 5: Cognitive Steering Tests. (A) Example triplet from the controlled composite tests. (B). Schematic of gain modulation mechanism. (C). Results of LRM architecture are shown, for both side-by-side and overlay composites tests. Subplots show different cognitive steering signal variations. Target- present conditions indicate percent of times the target category was predicted, when steering towards that target. Target-absent condition indicate the percent of times that the target-absent category was predicted, when steering towards the target absent category (false alarm). Error bars indicate the 95% bootstrapped confidence interval across composite images. Horizontal, dashed gray lines indicate the accuracy of classifying the individual target images alone, computed for the LRM3 model for its default modulated pass.
4.3 Cognitive Steering enables flexible recognition in composite images
The results of the LRM3 model with various forms of cognitive steering are shown in Fig. 5| We see that without any steering (in the feed-forward pass O, black bars), the model is unable to accurately identify either category from these composite images: side-by- aidebb--idd=2245%% overlay=5.1 We found that cognitive steering was generally able to improve recognition of either category, in some cases, quite dramatically. For example, category- category-prototype steering (with a-scaling=3) showed side-by-side recognition 68.9% and overlay recognition = 51.8%. Language- based steering (also with a-scaling==) yielded side-by-side recognition 70.4%. overlay recognition 50.5%. To contextualize these effects, the level of accuracy obtained by this model using its default modulated processing pass over the individual images is 82.2% (dashed gray lines Fig. C). Finally, gain with increasing
control was also particularly effective for category-prototyyn prototype and CLIP-stecring a-scaling following u-shape curve (for > 4, accuracy continues to decrease). Comparable results were observed for the LRM2 model, and much weaker results for the LRMI model (see Table (A.4). Appendix Fig. provides additional plot highlighting the effect cognitive steering on the logits.
Broadly, the pattern of results show that category prototype steering and CLIP steering were the most effective. This pattern is perhaps surprising, revealing that a more graded, probabilistic steering target is more effective than steering based on the objective of model or through the specific representation of the exact image. Taken together, these experiments demonstrate that these LRM feedback pathways enable effective, parameterizable cognitive steering, yielding dramatic improvements in recognizing the target categories present in composite images.

---

A
Modulation of human brain regions with category-based attention
B
Modulation of model units with category-based steering
Steering
Attend-Fase Attand-House
Identify Face-selective neurons
2.4
3.0
stanal char nangel
1.8
Activation
2.0
1.0
0.8
Identify house- selective neurone
Scane
Model Faca-Units
Model House-Units
Rogion
Region
Figure 6: (A) Modulation in face- and scene-selective visual brain regions, during top-down attention of face-scene composite images 41. (B) Parallel results in LRM networks, where face-tuned and scene-tuned units from a late stage layer of the model show different levels of activation to a composite image, when steering by face- and pouse-proto.spe.
4.4 Multiplicative modulation prevents rampant hallucinations
We next tested whether cognitive steering towards a non-depicted category would lead to increased false recognition. For example, given composite of English Setter and Church, what if the model is steered to "look for" Parachute? For these tests, we used the representation of the target-absent image of each triplet for cognitive steering. The results show that the likelihood of classifying the composite image towards the absent, non-target category was negligible, for all steering types (Fig. (5)
In follow-up analyses, we directly tested the claim that the multiplicative feedback is critical for preventing excessive false alarms to the steered category. First, we implemented matched LRM models with additive feedback projections. Our (standard) L.M-muliiplicative motif is L(xp+1) L(xp+1) Fp: we trained additional LRM-additive models that influence destination layers additively: L(Tx+1) 1 + F' (Appendix A.7)). Second, we also considered CORnet model, brain-inspired recurrent neural network model with additive feedback of a layer, onto itself, over time 44], and invented a different way to inject steering signal into its recurrent processes (Appendix A.8 We found that steering LRM-additive models and CORnet models did improve detection of the goal category, but also lead to increased false alarms. These comparisons help clarify the benefits of multiplicative over additive feedback modulation: while both motifs can boost target-aligned features, only the multiplicative motif maintains low false alarms, even when increasing attentional gain.
4.5 Cognitive Steering recapitulates neural signatures of category-based attention
In the biological vision sciences, extensive research has shown that goal-based attentional states modulate activity along the entire visual processing hierarchy (e.g. evidenced across methods, including single-unit electrophysiology [5] 42), functional magnetic resonance imaging [41 and magnetoenceephalggaphy (2T) 44]; for review see [45] 46 47 48).. For example, seminal empirical work showed that when participants viewed composite face-house images, their face-selective brain regions were
active with goal-directed attention faces, while scene-selective regions were more active with goal-directed attention attention to scenes, even though the actual composite visual input always the same (Fig. A; 41).
Our LRM models recapitulate this well-known signature of top-down category- based attention, with systematic modulation of face-selectiv, and scene-selectin units based on the steering target (Fig. B;; see Appendix A.6). our knowledge, this the first mechanistic model, operating over rich naturalistic images, to account for these empirical neural signatures (for the most related earlier work see IT). Further, to our knowledge, current feed-forw foddfarrrr//rerrreentt vision models do not (yet) have mechanisms to support top down-goal based modulation of carly visual processing stages.
5 Limitations
The three LRM models tested here reflect only a small part of the space of possible long-range modulatory architectures. Our results show increasing benefits with more range modulatory

---

pathways, leaving open the search for the optimal macro-scale architecture design. Additionally, it is an open question whether similar improvements in accuracy and adversarial robustness would emerge if LRM pathways were added to standard ResNets [49 or state-of-the art Convnexts [50] or Vision Transformers [51]. We note, however that even if improved accuracy and robustness are not observed in larger models during default operation, adding LRM feedback pathways would still enable new
cognitive steering capacities, and is thus an important avenue for future work.
In this work, we have not done an analysis of the structure in the learned channel-to-chane weights, nor characterized how these modulations are changing the representational geometry of each layer, nor provided mechanistic clarity on why these feedback connections are effective. And, we have not explored the impact of the learnable normalization components relative to the learnable channel- to-channel weights on the effects reported here, nor considered alternate motifss of more spatially localized (rather than globally broadcast) modulatory influences. Given the behavioral improvements found in these LRM models, deeper exploration into these inter-workings and variations would be valuable next steps.
Finally, while we used category-supervised training here, we think an important next step where the capacities of these LRM models would particularly shine is in self-supervised setting (e.g. [52] 53 (4]).. In particular, both the category prototype steering signals and the language-based signals are completely agnostic the content of the output space, and can operate equally well in self-supervised domains. Thus, this work sets the stage for cognitive queries to help guide self- supervised visual encoding of the world, enabling new featural distinctions be emphasized between the feed-forwan and modulated pass, related to the goals of the cognitive system.
6 Related Research
Recurrent feedback connections have been integrated in deep neural networks in prior research, often drawing inspiration from the extensive back-projectiont present in the visual system [c.g. [53 [56 10 convolutional networks, with loss function for each time step (as we did here), focusing on the benefits of iterative emergence of categorical information over many successive steps. (7] explored external cognitive bias implemented through recurrent feedback, guide spatial selection mechanisms to recognize overlapping digits. [|] used an extensive procedure to search through a dillion-dimensional parameter space of feedback connection motifs, yielding models that show similar capacity to correct mis-classification errors with recurrent steps [see also 9].
57 58
g 59 [2 60, 6 61]. For example, [62] introduced feedback connections to
Our model introduces relatively simple multiplicative feedback modulation, contrasting with modeling approaches in which feedback activation signals additively influence carlier layer activation [e.g. 10 177, or more structured approaches which process top-down signals through separate parallel architectural branches [e.g. 63 58, 64). We also designed our model to operate with one modulatory pass, enabling rapid goal-directed steering with separate gain modulation- departs from traditional recurrent neural networks which typically focus on the benefits of recurrance over longer roll-outs [e.g. 62. [56). Our work focuses on how feedback connections can be used for top-down goal-directed visual encoding-this has parallels to work focusing on top-down regulation, to aid in recognition despite partial occlusion clutter, in object localization [e.g. [59 55 63
12 65 (66. Top-down
feedback networks have also been used to connect with theories of predictive coding, typically in support of more temporally extended processing of bottom-up visual information through time [e.g. 67 68, 600 69 (88, whereas here we focus on their use for external guidance signals to influence feed-forward visual processing.
7 Conclusions
Here we introduce cognitively- and biologically-inppiee long-range modulatory pathways. We show that these learned modulatory feedback routes create fixed circuitry that can be leveraged for more flexible, goal-based modulation of visual input processing, enabling "cognitive steering" of visual processing. Broadly, we suggest that these architectural pathways provide novel targets for the introduction of top-down steering, offering new possibilities for integrative systems (e.g. mudti-modal vision-language alignment; RL agents with goal goal-directed visual encoding) to enable more flexible communication between visual and cognitive components of the models.
10

---

References
[1] Jeremy M Wolfe. Guided search 2.0 revised model of visual search. Psychonomic bulletin & review, 1:202- 1:202-238, 1994.
[2] Diane M Beck and Sabine Kastner. Top-down and bottom-up mechanisms in biasing competition in the human brain. Vision research, 49(10):1154-116; 2009.
[3] Charles Connor, Howard E Egeth, and Steven Yantis. Visual attention: bottom-up versus top-down. Current biology, (111)))850-R852 2004.
[4] Georgia Gregoriou, Stephen Gotts, Huihui Zhou, and Robert Desimone. Long-range rangerrage neural coupling through synchronization with attention. Progress in brain research, 176:35 45, 2009.
[5] Robert Desimone and John Duncan. Neural mechanisms of selective visual attention. Annual review of neuroscience, 18(1):193-222, 1995.
[6] Xiaoliang Luo, Brett D Roads, and Bradley C Love. The costs and benefits of goal-directed attention in deep convolutional neural networks. Computational Brain & Behavior, 4(2):213- 230, 2021.
[7] Qian Wang, Jiaxing Zhang, Sen Song, and Zheng Zhang. Attentional neural network: Feature selection using cognitive feedback. Advances in neural information processing systems, 27, 2014.
[8] Gabriel Kreiman and Thomas Serre. Beyond the feedforward sweep: feedback computations in the visual cortex. Annals of the New York Academy of Sciences, 1464(1):222-2411, 2020.
[9] Aran Nayebi, Daniel Bear, Jonas Kubilius, Kohitij Kar, Surya Ganguli, David Sussillo, James J DiCarlo, and Daniel Yamins. Task-driven convolutional recurrent models the visual system. Advances in neural information processing systems, 31, 2018.
[10] Courtney Spoerer, Patrick McClure, and Nikolaus Kriegeskorte. Recurrent convolutional neural networks: better model of biological object recognition. Frontiers in psychology, 8:1551, 2017.
[11) Marijn F Stollenga, Jonathan Masci, Faustino Gomez, and Jürgen Schmidhuber. Deep networks with internal selective attention through feedback connections. Advances neural information processing systems, 27, 2014.
[12] Chunshui Cao, Yongzhen Huang, Yi Yang, Liang Wang, Zilei Wang, and Tieniu Tan. Feedback convolutional neural network for visual localization and segmentation. IEEE transactions on pattern analysis and machine intelligence, 41(7):1227-1640, 2018
[13] Weiwei Zhang and Steven Luck. Feature-based attention modulates feedforward visual processing, Nature neuroscience, 12(1):24-25, 2009.
[14] Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018
[15] Daniel Felleman and David C Van Essen. Distributed hierarchical processing in the primate cerebral cortex. Cerebral cortex (New York, NY: 1991), 1(1):1-47, 1991.
[16] Sabine Kastner and Mark A Pinsk. Visual attention multilevel selection process. Cognitive, Affective, & Behavioral Neuroscience, 4(4):483-500, 2004.
[17] Charles D Gilbert and Wu Li. Top-down influences visual processing. Nature Reviews Neuroscience, 14(5):350-33) 353,3, 2013.
[18] Frederick Federer, Seminare Ta' afuu,, Sam Merlin, Mahlega S Hassanpour, and Alessandra Angelucci. Stream-specifie feedback inputs to the primate primary visual cortex. Nature Communications, 12(1):228, 2021.

---

[19] Simon Kornblith and Doris Y Tsao. How thoughts arise from sights: inferotemporal and prefrontal contributions to vision. Current Opinion in Neurobiology, 46:208-218 2017.
[20]
Samantha R Debes and Valentin Dragoi. Suppressing feedback signals to visual cortex abolishes attentional modulation. Science, 379(6631):468 473, 2023.
[21] Viola S Störmer, Michael A Cohen, and George A Alvarez. Tuning attention object cate- gories: Spatially global effects of attention to faces in visual processing. Journal of cognitive neuroscience, 31(7):937-947,
2019. 2019.
[22] Alex White and Marisa Carrasco. Feature-based attention involuntarily and simultaneously improves visual performance across locations. Journal of vision, 11(6):15-15, 2011.
[23] Melissa Saenz, Giedrius T Buracas, and Geoffrey M Boynton. Global effects of ceature-based attention in human visual cortex. Nature neuroscience, 5(7):631-632, 2002.
[24] Melissa Saenz, Giedrius T Buračas, and Geoffrey M Boynton. Global feature-baser attention for motion and color. Vision research, 43(():622 637, 2003.
[25] Taosheng Liu, Feature-based attention: effects and control. Current opinion in psychology, 29:187-192, 2019.
[26] Chris Hamblin, Talia Konkle, and George Alvarez. Pruning for interpretable, feature-preserving circuits in cnns. arXiv preprint arXiv:2206.01627 2022.
[27] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-252, 2015.
[28] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 245-255. leee, 2009.
[29] Guillaume Leclere, Andrew Ilyas, Logan Engstrom, Sung Min Park, Hadi Salman, and Aleksander Madry. FFCV: Accelcrating training by removing data bottlenecks. https /ggithub.com//iit wttm////www////e//////////oc 2022. commit XXXXXXX.
[30] Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domant operations applications, volume 1 1006, pages 369-386. SPIE, 2019.
[31] Harish Guruprasad Ramaswamy et al. Ablation-cam: Visual explanations for deep convolutional network via gradient-free localization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 983-991, 2020.
[32] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint 1rXiv:14/2.665 2014.
[33] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5(3):00024-0011, 2020.
[34] Ziming Liu, Eric Gan, and Max Tegmark. Seeing is believing: Brain-inspired modular training for mechanistic interpretability, 2023.
[35]
Tai Sing Lee and David Mumford. Hierarchical bayesian inference in the visual cortex. JOSA A, 20(7):1434-1448, 2003.
[36] Jonas Kubilius, Martin Schrimpf, Kohitij Kar, Rishi Rajalingham, Ha Hong, Najib Majaj, Elias Issa, Pouya Bashivan, Jonathan Prescott-Roy, Kailyn Schmidt, al. Brain-like object recognition with processing systems, 32, 2019.
shallow recurrent anns. Advances in neural information
[37] Kathleen O' Oraven, Paul E Downing, and Nancy Kanwisher. fmri evidence for objects as the units of attentional selection. Nature, 401(6753):584-557, 1999.
12

---

[38] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2004.06125, 2022.
[39] Daniel Kahneman. Atlention and effort, volume 1063. Citeseer, 1973.
[40] J Kubilius, M Schrimpf, A Nayebi, D Bear, DLK Yamins, and JJ DiCarlo. Cornet: Modeling the neural mechanisms of core object recognition. biorxiv, 408385, 2018.
[41] Kathleen M O'Craven, Paul E Downing, and Nancy Kanwisher. fimri evidence for objects as the units of attentional selection. Nature, 40166753):544-577 1999.
[42] Julio C Martinez- Trujilo and Stefan Treue. Feature-based attention increases the selectivity of population responses primate visual cortex. Current biology, 14(9):744 751, 2004.
[43] Elias H Cohen and Frank Tong. Neural mechanisms of object-based attention. Cerebral Cortex, 25(4))1080-109) 2015.
[44] Daniel Baldauf and Robert Desimone. Neural mechanisms of object-based attention. Science, 344(6182):424 427, 2014.
[45] Robert Desimone. Visual attention mediated by biased competition in extrastriate visual cortex. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 353(13333:1225--255 1998.
[46] John HR Maunsell and Stefan Treue. Feature-based attention in visual cortex. Trends in neurosciences, (9)))3117-22,, 2006.
[47] Behrad Noudoost, Mindy H Chang, Nicholas A Steinmetz, and Tirin Moore. Top-down control of visual attention. Current opinion in neurobiology. 10,,);183-190, 2010.
[48] Marius V Peelen and Sabine Kastner. Attention in the real world: toward understanding its neural basis. Trends in cognitive sciences, 18(5):242-250, 2014.
[49] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
[50] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xic. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976-11986, 2022.
[51] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11923 2020.
[52] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PMLR, 2020.
[53] Talia Konkle and George A Alvarez. A self-supervised domain-generaa geniral learning framework for human ventral stream representation. Nature communications, 13(1))491, 2022.
[54] Jure Zhontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In International Conference Machine Learn- ing, pages 12310-12320. PMLR, 2021.
[55] Ming Liang and Xiaolin Hu. Recurrent convolutional neural network for object recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3367-3375, 2015.
[56] Qianli Liao and Tomaso Poggio. Bridging the gaps between residual learning, recurrent neural networks and visual cortex, 2020.
13

---

[57] Siming Yan, Xuyang Fang, Bowen Xiao, Harold Rockwell, Yimeng Zhang, and Tai Sing Lee. Recurrent feedback improves feedforward representations in deep neural networks. arXiv preprini MnXiv:1912 10489, 2019.
[58] Kuan Han, Haiguang Wen, Yizhen Zhang, Di Fu, Eugenio Culurciello, and Zhongming Liu. Deep predictive coding network with local recurrent processing for object recognition. Advances in neural information processing systems, 31, 2018.
[59] Mingqing Xiao, Adam Kortylewski, Ruihai Wu, Siyuan Qiao, Wei Shen, and Alan Yuille. Tdapnet: Prototype network with recurrent top-down attention for robust object classification under partial occlusion. arXiv preprint urXiv: N0Giv:/909.03300 1909.03879, 2019.
[60] Yujia Huang, James Gornet, Sihui Dai, Zhiding Yu, Tan Nguyen, Doris Tsao, and Anima Anandkumar. Neural networks with recurrent generative feedback. Advances in Neural Information Processing Systems, 33:535-545, 2020.
[61] Grace W Lindsay and Kenneth D Miller. How biological attention mechanisms improve task performance in large-scale visual system model. ELife, 7e38105, 2018.
[62] Amir R Zamir, Te-Lin Wu, Lin Sun, William B Shen, Bertram E Shi, Jitendra Malik, and Silvio Savarese. Feedback networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1308-1317, 2017.
[63] Abhinav Shrivastava, Rahul Sukthankar, Jitendra Malik, and Abhinav Gupta. Beyond skip connections: Top-down modulation for object detection. arXiv preprint arXiv:1612.06833 2016.
[64] Md Amirul Islam, Mrigank Rochan, Shujon Naha, Neil DB Bruce, and Yang Wang. Gated feedback refinement network for coarse-to-fine dense semantic image labeling. arXiv preprint 1806 1 1266, 2018.
[65] Shantanu Jaiswal, Basura Fernando, and Cheston Tan. Tdam: Top-down attention module for contextually guided feature selection in cnns. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXV, pages 259-276. Springer, 2022.
[66] Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. Top-down neural attention by excitation backprop. International Journal of Computer Vision, 126(10):1084-1000 102, 2018.
[67] Andrea Alamia, Milad Mozafari, Bhavin Choksi, and Rufin VanRullen. On the role of feedback in image recognition under noise and adversarial attacks: A predictive coding perspective. Neural Networks, 157:280-282 2023.
[68] Yujia Huang, Sihui Dai, Tan Nguyen, Pinglei Bao, Doris Y Tsao, Richard G Baraniuk, and Anima Anandkumar. Brain-inspired robust vision using convolutional neural networks with feedback. In Real Neurons { Hidden Units: Future directions at the intersection of neuroscience and artificial intelligence@ NeurlPS 2019, 2019.
[69] William Lotter, Gabriel Kreiman, and David Cox. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint mrxiiv:600.. 000000 2016
[70] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages $748-8763. PMLR, 2021.
[71] Nancy Kanwisher, Josh McDermott, and Marvin M Chun. The fusiform face area: module in human extrastriate cortex specialized for face perception. Journal of neuroscience, 17(1 (7(13223302- 4311, 1997.
[72] Russell Epstein and Naney Kanwisher. cortical representation of the local visual environment. Nature, 392(6676):598-60.) 1998.
14

---

[73] Ewa Wojciulik, Naney Kanwisher, and Jon Driver. Covert visual attention modulates face- specific activity in the human fusiform gyrus: fmri study. Journal of neurophysiology. 79(3):1574-1578, 1998.
[74] Jacob S Prince, George A Alvarez, and Talia Konkle. A contrastive coding account of category selectivity in the ventral visual stream. bioRxiv, pages 2023-03 2023.
[75] Neil A Macmillan and C Douglas Creelman. Detection theory: A user's guide. Psychology press, 2004.
15

---

A Appendix
A.1 Modulatory Motif Detail
Here is more detailed description of how source activations modulate the destination layer activations. First, channel normalization operation serves emphasize some source features, and de-emphasize others, over a given mmgge We include learnable and oe parameters across all channels in the source layer. For source layer activation channel c, the normalization function W was defined as:
N M(cc Mhe σa) 0g Ue
For convolutional layers that operate jointly channels and spatial locations (x, v). the normaliza- tion function was extended to:
(ac,z.gi Ve,a,y Te.x,y) Oc de.rry
dc2,,9
After normalization, the activations a' passed through the tanh function, with learned scaling parameters. Function S scales the output of the tanh function based on the sign of the normalized activation:
((a';ot,o-)) a+ tanh(a') ifa' 2 o- tanh(a)) ifa 0
where is the Iearnable scalar for positive activation scaling, and is the learnable scalar for negative activation scaling. Next, the size of source activations a" is aligned to the size of the destination map Dw.h- If the source layer is fully-connected and the destination layer convolutional layer, the source channels' activations S. are spatially broadcast to the size of the of the destination map activation map.
Resize(a")) = aa" x DD...
where a" is the activation of each source channel and 1D.., a is matrix of ones with dimensions Dwr If both the source and destination layers are convolutional, the source activation map is upsampled to be the size of the destination activation map:
U(o",..yi Sw.. + Dw...) = ( Upssmmleedd"...p Sw, Dw..))
where d' represents the activation of the source channel e at spatial location (x, y) and the function Upsample up-samples the source activation map from size S to size D Following the normalization and scaling operations, the key transformation in the LRM pathways is the learned toaneel-o--chnnel influences. This is implemented using Ixl convolutional layer:
Se We deay
where Wes the Iearnable weights of the Ixl convolution, ""..gy is the source activation, and ae is the resulting activation that will use to modulate the destination layer's next pass.
16

---

A.2 Impact of steering on logits
Fig. provides a more detailed depiction of the cognitive steering signal effects, focusing the goal- steering, a-scaling=3), The output activation
directed pass of the LRM3 model unit of target target and the target-absent category are shown. When steering toward the category prototype of the either target, there is substantial increased activation along that output unit dimension. Steering towards the absent, non-target category does not have the same consequence (a benefit of the purely multiplicative modulatory interaction).
Category-steering
LRM3, Pass1,
Side-by-Side Composites
Overlay Composites
Steer Target1
activation
6
Steer
Target 2
2
Steer Target
Absent
output unit
output unit target2
output unit target- -eseet--beett
output unit target1
output unit target2
outpuE unit target-absent
target1
Figure 7: Impact of steering on output unit activation levels (1000 units corresponding to each image category). Average activation is plotted for the output unit corresponding the category of each triplet image, including the two that were presented (targetl, target2) and the image that was absent (absent-image), for both overlay and side-by-side composite images. Error bars reflect standard deviation over images. Data reflect activations from the LRM3 model, after the first modulated pass with category-protoge prototype steering and gain modulation of = 3.
A.3 Cognitive Steering with Clip Text Embeddings
One potential application of LRM networks is to enable top-down steering of visual encoding via natural-language queries e.g., steering the visual encoder by asking "Is there English Springer present?" amplifying any visual responses aligned with the query throughout the vi- sual encoder. Ideally this language-based steering would be implemented via integrated vi- sion+language multimodal model, such as CLIP [70.. using an LRM network as the vision encoder. We haven't it yet trained multi-modal LRM network, but tion, we generated steering vectors from category labels using an intermediate model that was trained to map CLIP text outputs and LRMNet vision outputs. Our approach the "diffusion prior" of the Dalle2 architecture (38) using the lucidrains PyTorch implementation lhttps://gii http//gtttu.... com/ luci www.ddr/aaaaaaa.. -yttochcc))t.hh).... modified allow mapping from N-D to M-D feature spaces, since our vision models had a different output dimension than the CLIP text encoders. The goal was to provide category label, e.g., "English springer", a CLIP text encoder, and to then map the CLIP text output to the expected LRM network output for an image from that category, specifically to the mean vision output for that image category. This predicted embedding could then be used as a steering vector to exert top- control over visual processing within the LRM network (see "clip-text prototypes", Sections and
proof of concept for this applica-
modeled after
A.3.1 Image Embeddings
We first extracted the output vector for each image in the ImageNet training set, then averaged across images within each category. yielding 1000x 1000 tensor of "visual category prototypes" (1000 categories 1000 output units). We also computed the prototypes for the ImageNet validation set. These visual category prototypes were computed separately for each LRM network.
A.3.2 Text Embeddings
We used the "ViT-B/16" text encoder from the official open source CLIP models https generate clip-text embeddings for each ImageNet cate- gory, we used the zero-shot imagenet labels and prompts from the original CL.IP paper http:///waannoottoo////////o/oo/ Angineeringgfor_inee ImageNett.ttttt... ipynb Each ImageNet target label was converted to the ImageNet
770. as described htte com/ wwtp://ppaaaa/..I///bbb/oaaaaa
17

---

classname (e.g., label=O, classname "tench") ") and each classname was included within seven zero-shot prompts ("a photo of the large [I.". { in video game.", ", "art of the [}.". "a photo of the small ().", "itap of ().", "a bad photo of the ()-", "a origami ()."). These text prompts were then passed through the text encoder and the outputs were averaged across prompts, yielding 512-D text embedding, and 77x512 text encoding (embedding token, max sequence length 77)
for each ImageNet category. These clip-text representations could then be used to predict the image embeddings of an LRMNetwork.
A.3.3 Text-to-lmage Diffusion Model
Following 38 trained diffusion model generate image vectors conditioned on text embed- dings and encodings. We used the lucidrains DiffusionPrioNNewoork and DiffusionPrior modules: prior httar-DfffussonPooooeetttrrrdna
depth=6, dim head=64, heads=8), and Diffusion- Prior(net=prior network, image_embess eeeeee_meee_eeeeeeeeeeeeee dim=1000, timesteps=100, cond _ropp_prob=220 20, condi- tion_on_ con_text_eeee _exx_eenodings/Tnue image_emhed, seathddscale-True
order align the text representations (512-D) with the image representations (1000-D), we used a linear projection layer (512 input chan- nels, 1000 output channels), and then passed these 1000-D text representations into the Diffusion Prior to condition the diffusion process. The output of the Diffusion Prior is a 1000-D vector, which serves as the predicted visual category prototype for the class specified by the text label. Each batch was composed of the full set of 1000 text-embeddings and 1000 image embeddings, and the models were trained for 20000 iterations using the Adam optimizer (Ir=.0001, betas=(0.9, 0.999), decight_decy=-dlee
Finally, after a DiffusionNetwork was trained. we could "sample" text-conditioned image embeddings from the DiffusionPrior for each of the 1000 ImageNet labels. We aggregated across 100 samples category to generate "clip-text prototypes", which we treat as the final mapping between ImageNet text labels and the visual encoder outputs.
A.3.4 Text-to-Image Embedding Quality
To measure the accuracy of the text-to-vision alignment, computed the cosine similarity between the 1000 clip-text prototypes, and the visual category prototypes for the training set and the validation As shown in Table A1, the clip-text prototypes for all models showed high cosine similarity scores on both the training set (>= -938%) and validation set (>= 88%)). ). For comparison, the cosine similarity between training and validation set vision prototypes was .94 for all LRM models.
LRM1
LRM2
LRM3
Training Set
0.950
0.944
0.938
Validation Set
0.895
0.889
0.884
Table Cosine Similarity between Clip-Text Prototypes and Visual Category Prototypes
for each LRM network
A.4
Composite Test Accuracy, All Models
Composite Test
Model
Pass 0
1-Hot
Instance
Category
Clip
LRMI
0.246
0.143
0.275
0.395
0.400
Side-by-Side
LRM2
0.236
0.109
0.382
0.743
0.718
LRM3
0,245
0,235
0.507
0.689
0.704
LRMI
0,016
0.024
0.139
0.250
0.236
Overlap
LRM2
0.014
0.051
0.234
0.674
0.635
LRM3
0.016
0.075
0.272
0.543
0.527
Table 3: Composite test accuracy vs. steering type for all models with alpha modulation 3.
18

---

A.5 Additional Cognitive Steering Variations
We also tested discriminative steering signals that take into account both the targets, in order to amplify one and suppress the other (Vtarget1 Vtarger2). These steering vectors effectively query the system "look for X's distinct from Y's". To our surprise, we found that discriminative steering did not have dramatic impact on composite recognition accuracy, relative to target-based steering (Fig 8).
One interpretation of this result is that the modulatory circuitry may have already learned to amplify the target category relative distractor category is not effective. Another possibility is that the particular images used in this side-by-side test are by design relatively distinct, and the discriminative steering signals may be more effective if the target and distractor categories are more similar (e.g kit fox vs arctic fox). Future work will be required to clarify the mechanisms and theory around the most effective cognitive steering signals.
other categories, so the additional suppressive steering for the
Finally, we also explored whether extra modulatory passes would amplify any steering effects. However, we found that across models and all levels of a-scaling and all cognitive steering variations, additional passes had negligible impact beyond the first modulatory pass. This feature of our LRM models further dissociates them from traditional recurrent neural networks, which have the assumed characteristic running over multiple timesteps times. contrast, our model has a single parameter (a) that controls the gain on the modulation, operating on a single modulated pass.
Discriminative Cognitive Steering (Target 1 Target 2)
1-Hot Steering
Instance Steeriig
Category-Steenie
CLPP-Steerig
Accuracy
.6
.5
langet present
langer Absent
Target prosent
Tarpet
Targat prosert
Torpet Absent
Target present
Abcent
Acouracy
Yarget
Torget
Target
"arget
Target
Target
Targat
Target
prisent
Azreent
Figure 8: Results of the LRM3 architecture are shown, for both side-by-side and overlay composites tests. Subplots show different discriminative cognitive steering signal variations. Target-present conditions indicate percent of times the target category was predicted, when steering towards that target. Target-absent condition indicate the percent times that the target-absent category was predicted, when steering towards the target absent category (false alarm). Error bars indicate the 95% bootstrapped confidence interval across composite images. Horizontal, dashed gray lines indicate the accuracy of classifying the individual target images alone, computed for the LRM3 model for its default modulated pass.
A.6 Steering modulates face-tuned and scene-tuned units
Classic studies in cognitive neuroscience have shown that category- and object-based attention dramatically alter neural responses in the ventral stream, even within category-selectiv brain regions. For instance, in the human visual cortex, there are face-selective and scene-selective brain regions: e.g., the fusiform-face area (FFA) responds more to faces than scenes any other category of object 711, whereas the parahippocampal place area (PPA) responds more to scenes than to faces or other categories of object [72] When presented with overlapping faces and scenes, the activity of these
19

---

regions varies dramatically based on the attended category [37 73 demonstrating that top-down guidance signals have a strong influence on processing within the ventral stream (Fig. (-A)
We tested whether the LRM3 model would show similar effects of category- attention within analogues of the human-FFA and human-PPA. To do so, we employed method we introduced in prior work [74) to identify both face-selective and scene-selective units within the model using the same stimuli and statistical corrections used in the fMRI literature. Specifically, we used a "category localizer" image dataset composed of 400 images, 80 per category for 5 categories (Faces, Bodies, Objects, Scenes, Words) known to selectively activate different regions of the human visual cortex. We passed each image through the network and stored the activations separately for each layer. To determine whether a unit (channel, x, y) responds selectively to faces, we performed a t-test on activation scores for 80 face images vs. activations to 80 images from another category (separately for each other category one-by-one). The p-values were corrected using a false-discovery-ratt (FDR) of .05 to adjust for multiple comparisons. A unit was considered face-selective if its activation was significantly greater for faces than each of the other categories in their head-to-head t-tests. The same procedure was used to identify scene-sclective units, identifying units that showed higher activation for scenes than all other categories.
For steering signals, we computed the average model output for all face images (face-prototype) and all scene images sxcene-proooyype To determine the effect of steering on face-units and scene-units, we presented overlapping face and scene pairs (each face image was paired with a scene image, and they were blended 50/50: composite = 5 * facel 1 + .5 scenel). We measured the activation of face-units and scene-units when the network was steered towards the sice-prototype, or the scene- prototype, with an alpha of 3.0. As can seen in Fig. 66B, the activation of model face-units and scene-units is strongly modulated by the steering target, with face-units showing stronger activation when steering towards faces, and scene-units showing stronger activation when steering towards scenes. Thus, LRM models with cognitive steering are able to qualitatively capture this well-known signature of category- based attention in the human brain.
A.7 Steering in LRM-additive models
Initial analyses were conducted to explore the impact of LRM pathways with additive modulatory influence on destination layers. We hypothesized that during cognitive steering, models with additive feedback would have more "hallucinations' (or false alarms) to the steered category, even if it was not present in the image.
Table 4 focuses subset of the results, considering prototype steering, with overlaid images. The -uulppuliipieaative model showed peak performance at a=3; the a.dMivddiiivve model showed higher hit rates, but also higher false alarm rates. Further, when steering the LRM-additive models, the false alarm rate climbs with increasing attentional gain (a=5), but that does not happen with the LRM-multiplicative models.
Considering the signal detection measure A' (sensitivity of hits relative to false alarms (75)), the LRM3 2BM33-ultiplicative model has an A' that is high and fairly stable across the alpha-gain range. Meanwhile, the LRM3-additive model has an A with a steeper decline of sensitivity with increasing attention modulation. Qualitatively similar results were obtained with the bide-b composites. Thus, these LRM -addtivee models help clarify the role of the multiplicative feedback mechanisms in reducing hallucinations during high attentional gain states.
LRM RRM-mulipllicative hit rate % false alarm
LRM-addirive % hit % false alarm
LRM3, a=3
53.3%
5.8% 3.2%
62.3% 67.0%
21.2%
LRM3, o=5
35.0%
38.9%
Table 4: Comparison between LRM3 models with either multiplicative or additive feedback modula- tion. These results show the impact of cognitive steering, on the overlaid image composite test.
20

---

A.8 Steering with CORnet
CORnet [40] is recurrent neural network, with similar consecutive forward- -pass dynamics as LRM models. That is, the image is processed serially through the hierchical layers in the first time step. In CORnet, the activations from each layer are carried forward in time, and then add to the same layer's activation on the next time step (as opposed to earlier layers, as in our LRM models). Pre-trained CORnet models perform 5 forward passes by default.
To outfit this model with steering capacity, we replaced the previous time point's activation with steering signal. We used a steering signal at every layer at every time point. For instance-level steering, the steering inputs consisted of the activations of the exact image, at every layer and every time point-i.e. perfect oracle instance-level steering. For category-proootype steering, the steering inputs consisted the average activation map over all exemplar images of that category (in the ImageNet Training Dataset), computed at each time step and layer. We additionally included steering modulation strength factor, multiplying the steering signals by a constant (s = 2, 3, 4).
Note that implementation of CORnet steering visual encoding backbone at every time point, per category. This steering interface differs dramatically from the LRM networks, which leverages single 1000-dimensional vector per category. We note that there are likely many ways to outfit recurrent neural networks to inject steering-we opted for these choices because they seemed the most likely to lead to the largest consequences of cognitive steering.
stored representation every layer of the
The results shown in Figure 9 with varying strengths of both instance steering and prototype steering, assessed on composite overlaid images. It is clear that increasing the strength of steering increases correct target recognition but also introduces false alarms when steering category that not present in the image (target-absent conditions).
We used the signal detection measure A' (A-prime)
75 to quantify this effect. A' is measure
of sensitivity for distinguishing between signal (target present) and noise (target absent), and is a con-parametric version of d' (d-prime). A' values range from 0 to I, with .5 indicating chance, I indicating perfect target detection with no false alarms (values less than .5 are uncommon, e.g., indicates 100% false alarms and 0% hits). The A' effect is plotted for LRM3 and CORnet models on the same plot (though note the way the steering strength implemented differs between these models). With increasingly strong steering, the LRM models maintain their ability to correctly detect target present categories without false alarms, while the same is not true for the CORnet model (as have implemented steering here).
21

---

A Implementing Steering in CORnet
Default CORnet recurrence
Computing steering signals For this instance (Image)
Over-write state influence with
steering signal
decoder
decoder
decoder
decoder
decoder
decoder
decoder
decader
decoder
decoder
decoder
decoder
decoder
decoder
decoder
W
96
feedbook
feedback
feedbock's
Input
Input
Input
Input
Input
Input
Input
Input t=
Input t=1
Input
input
Input
t=0
t=1
t=2
t=3
t=
t=1
t=
t=3
t=2
t=3
1-4
Self-recumne recurreturnnt cornectionss pass a layers output to itself next time step
Record states at layers at all time steps
Inject steering signal during composite image processing
B
CORnet: Sestance-Steering
C
Sensitivity Comparison: Instance instance-Steering
#1hh-
1.0
pharget category is activation login
alphand
Hallucinations
0.9
Sensttvity (A)
LAMS
0.8
0.7
0.
CORnet
0.
3.0
2.0
3.0
4.
08
1.0
20
50
0.5
Target present (Hits)
Target abeent (False Alarms)
Modulation Muttipler
CORnet: Category-Steering
E
Sensitivity Comparison: Category-Steering
10
Halluchaatins
ptarret catagory is
max activation logi)
0.
LRM3
Sensltivit (A)
0.6
0.4
CO
0.8
3.0
2.8
3.0
48
Targel present (Hits)
Target absent (False Alarme)
Modulation Muttiplier
Figure 9: Steering in CORnet. (A) Diagram of the flow of information in CORnet (adapted from Kubilius et al., 2018). Steering signals are calculated for each image (or the average of all images in a category), at each layer and each time point (yellow arrows). During steering, the state is overwritten with these arrows. (B) Instance-stcering in CORnet models, on composite overlay images. Probability that a target is the max logit is plotted for steering when the target category is present absent, for different levels of steering strength. (C) Sensitivity measure, A' (reflecting corrected Hit rate taking into account False Alarms) is plotted for both LRM3 and CORnet RT models with increasing steering strength. (D,E) Same plots but for category-level steering signals.
22


Reviews:
[{"review_id": "qeb0mShVLs", "reviewer": "Reviewer_esAd", "summary": "Authors study ways of incorporating cognitive steering in vision neural network models. They add a top-down feedback mechanism to Alexnet with which they report improved performance. Further, they test other steering mechanisms, including prototypes, language-based signals etc. These tests are over image composite tasks where the approaches show greatly improved performance. ", "questions": "Improvements to text and minor corrections:\n* Please make it clear in the text what \"target absent\" control means. Only place it is explained is the Figure 5 caption. I had to spend a lot of time trying to understand that until I stumbled on Fig 5 caption. \n* Please consider updating Figure 3 to say \"target unit\" or \"target neuron\" or \"target logit\" instead of \"target\" \n* Line 169: where &rarr; were\n\nQuestions:\n* Why does LRM models have higher accuracy than alexnet at 0th modulatory pass? They should be same?\n* In the \"target absent\" control - how is the absent target chosen? Is it average of every other (998 other target classes not present in the composite or some random class?).\n* How many modulatory passes were they trained for and tested for? Is it the same number of passes in training and testing?", "limitations": "NA", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper is interesting, the experiments and the controls are convincing. Cognitive steering in deep CNNs is novel as far as I know, especially with language signals. Some parts of the paper are well written. \n\n", "weaknesses": "* The paper lacks benchmarking. There are several methods of incorporating feedback in deep learning models from previous years that weren't tested. Look at [1] for a survey. Although I am sympathetic about this since cognitive steering in itself is interesting but the paper needs to be clear that the contributions are in studying various steering methods/signals and not introducing feedback itself. \n\n* Side-by-side composition is not a straightforward task setting - putting images side by side reduces the scale of the objects and since CNNs are not scale invariant that poses a challenge. So I am not quite convinced it is a good test for steering.  \n\n\n\n[1] Kreiman, G., & Serre, T. (2020). Beyond the feedforward sweep: feedback computations in the visual cortex. Annals of the New York Academy of Sciences, 1464(1), 222–241. https://doi.org/10.1111/nyas.14320", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Strengths:\n> *The paper is interesting, the experiments and the controls are convincing. Cognitive steering in deep CNNs is novel as far as I know, especially with language signals. Some parts of the paper are well written.*\n\nThanks!\n\n# Weaknesses:\n> *The paper lacks benchmarking. There are several methods of incorporating feedback in deep learning models from previous years that weren't tested. … the paper needs to be clear that the contributions are in studying various steering methods/signals and not introducing feedback itself.*\n\nBased on your and other reviewers’ feedback, we have (1) added a new paragraph in the introduction that directly states this, and (2) added a new analysis of CORNet (as per R2). CORnet shows high false-alarms when steering to a category that is not present in the image, unlike LRM models. Details in general rebuttal. \n\n> *Side-by-side composition is not a straightforward task setting - putting images side by side reduces the scale of the objects and since CNNs are not scale invariant that poses a challenge. So I am not quite convinced it is a good test for steering.*\n\nOur models are able to handle side-by-side images without reducing the scale of the objects presented to the convolutional backbone, by using PyTorch’s built-in adaptive average pooling at the interface between the convolutional and fully-connected layers.\n\nOne level of explanation deeper: objects in the side-by-side images (256x512 pixels) are presented at the same physical size (in pixels) to convolutional kernels as when a single image is presented (256x256) — we’ve basically “expanded the model’s view of the world at a single scale.” The adaptive average pooling operation resizes the final convolutional layer to align with the first fully-connected layer (reshaping from Cx6x12 to Cx6x6). This operation does introduce a slight aspect ratio distortion at the conv5-fc6 interface, but the steering signal is still able to effectively operate over this image to recognize either category, supporting our empirical claims. We do agree that other models may have rigid constraints in input image size, making this a potentially less general benchmark to employ on other models. To this end, the composite overlay task is a complementary task, as it does not have the same resolution issue. We added a note in section 3.2. \n\n# Questions:\n> *In the \"target absent\" control - how is the absent target chosen?*\n\nWe created a set of 3,910 image triplets, randomly drawing 3 images from 3 distinct imagenette categories (without replacement).  Two of the images were used to form a composite image and the third image was used as a “target-absent” or “distractor” category.  This “target-absent” category is critical to test for steering-induced hallucinations, i.e. whether steering towards a non-present category leads the model to report a category that was not present in the input. We have added this text to Section 3.2.\n\n> *How many modulatory passes were they trained for and tested for? Is it the same number of passes in training and testing?*\n\nOnly one modulatory pass was ever trained. When assessing default performance (e.g. recognition and robustness, without steering), we ran the model for one or more modulatory passes at test time (to see if additional passes helped). All steering analyses were done with only one modulatory pass. We clarified these method details in section 3.1.\n\n# Improvements to text and minor corrections:\n\n>  *Please make it clear in the text what \"target absent\" control means. Only place it is explained is the Figure 5 caption. I had to spend a lot of time trying to understand that until I stumbled on Fig 5 caption.*\n\nWe have added clarifying text (as in response to Q1 above). Sorry for the confusion, and thanks for sleuthing it out.\n\n> *Please consider updating Figure 3 to say \"target unit\" or \"target neuron\" or \"target logit\" instead of \"target\"*\n\nWe updated it to say “target unit.”\n\n> *Line 169: where → were*\n\nNice catch (though this text is now changed, as in Q1)\n"}, {"review_id": "PxH7HOkr2L", "reviewer": "Reviewer_sqng", "summary": "This work introduces a recurrent modulation module that can be added to visual models so that the top layer can project to and influence the earlier layers. The authors find that the model with the feedback projection layers outperforms the baseline feedforward model in both the categorization performance and adversarial robustness. The model is further analyzed to test whether the feedback modulation can be controlled to meaningfully steer the representations. The authors find that the top-layer steering yields significant performance increase when mixed targets are presenting in the same image in a side-to-side or overlaying fashion.", "questions": "See weakness.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written and easy to read. The significant improvement of the feedback-augmented model compared to the baseline model is also interesting. The steering analysis is done on both side-to-side and overlaying settings.", "weaknesses": "The idea of adding feedback modulation from top layers to earlier layers is not new. The authors should more clearly discuss the difference between their work and other models.\n\nThe performance and robustness evaluation also needs to be more careful. The feedback connection adds more computation and trainable parameters. But the performance is still compared to the baseline model with less computation and trainable parameters. The authors should compare their model to a larger or deeper architecture.\n\nAlthough the steering analysis shows that the model can reach higher performance, this analysis is kind of circular as the target signal is explicitly used to generate the modulation signal, which makes the performance improvement unsurprising. This steering property of the new model has its potential, as the proposed visual model is now available to be tested in attention experiments just as how humans can be asked to attend to different parts or features in their input. But more tests and experiments to compare models to humans are needed to illustrate this potential.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Strengths:\n> *The paper is well-written and easy to read. The significant improvement of the feedback-augmented model compared to the baseline model is also interesting. The steering analysis is done on both side-to-side and overlaying settings.*\n\nThanks!\n\n# Weaknesses:\n> *The idea of adding feedback modulation from top layers to earlier layers is not new. For example, the CORnet [1] model has this type of modulation architecture and also finds that adding feedback modulation helps performance and also makes the models more brain-like. The authors should more clearly discuss the difference between their work and this model.*\n\nIn fact, CORnet does **not** actually have long-range feedback connections, nor does it have any layer-to-layer interactions over time(!). CORnet recurrence is entirely \"local\" – it operates only within a layer, with the outputs of a layer serving as subsequent inputs to the same layer. There is no long-range, top-down feedback. In contrast, our long-range-modulatory networks connect deep layers to earlier layers via a cascade of long-range feedback connections (e.g., the output layer to FC6, FC6 to Conv4, Conv4 to Conv1). That being said, we agree that adding feedback modulation is a theme in the literature which we now discuss directly in the introduction. (See also general rebuttal).\n\n>*The performance and robustness evaluation also needs to be more careful. The feedback connection adds more computation and trainable parameters. But the performance is still compared to the baseline model with less computation and trainable parameters. The authors should compare their model to a larger or deeper architecture.*\n\nWe agree there are more parameters in the models with long-range feedback connections, and now note this caveat in section 3.1. And, we emphasize that models with more parameters (say, e.g. an Alexnet with more channels per layer) would not support cognitive steering, which is the primary aim of this work. \n\n> *Although the steering analysis shows that the model can reach higher performance, this analysis is kind of circular as the target signal is explicitly used to generate the modulation signal, which makes the performance improvement unsurprising.* \n\nWe somewhat agree, but with important clarifications. First, not all steering signals were equally effective–e.g. we thought the 1-hot steering would be the strongest (given that was the objective of the model during training!). However in fact, this was the weakest form of top-down steering, leading to negligible if any benefit (Figure 5.c. left). Thus, injecting category-relevant steering is not guaranteed to lead to improved recognition, as might be expected in a (statistically) circular analysis.  Second, we now more clearly emphasize the key result that the LRM models do not hallucinate a category that is not present in the image. If the steering results were purely circular, such that you always get as output what you inject in as a steering signal, then we would expect to see high false recognition when steering towards a category that is absent. This doesn't occur with our LRM models, but does occur with our CORnet steering tests, which use additive modulation and show dramatic increases in false alarms/hallucinations. In this way, the modulatory motif is also critical. We have clarified these points.\n\n# Summary:\nIn following the recommendations of your review, we ourselves learned more details about other recurrent models like CORnet, which we think also might be news for you as well.  Further, based on your comments, we have added a new paragraph and a new analysis attempting to implement steering in a CORnet model, which helps to clarify the benefits of multiplicative modulation, and situate our work more clearly in the literature.  We hope this additional work will entice you to increase your score, even if only a little ; ). Thank you!\n"}, {"review_id": "bNfXtw6rTH", "reviewer": "Reviewer_Bsgy", "summary": "The paper presents a novel, brain-inspired modulatory feedback circuitry (long-range modulation, LRM) for regular feedforward DNNs. The multiplicative modulatory pathways can be conditioned on a) higher-level activations (computed in the initial forward pass or subsequent modulatory passes, Sec 2.1, 2.2) to improve the network’s ImageNet accuracy and adversarial robustness (Fig 2) with brain-like dynamics (Fig 3), or on b) steering signals (derived from labels, instance/class activations, CLIP embeddings, etc., Sec 3.3, 3.5) to perform the composite image recognition task (Fig 5, 6). The evaluation is based on AlexNet, with the small-scale composite task created using Imagenette images (side-by-side or overlay, Sec 3.2) following experimental neuroscience protocols.", "questions": "1) Why are the 0th-pass results in Fig 2A and 2C better than the AlexNet baseline? Or, results in L216 better than L176? What does the 0th-pass model have in addition to the baseline?\n2) How’s the model’s training & inference speed compared to the baseline? How does the model’s accuracy compare to stronger baselines (either AlexNet with more parameters, or newer networks) running at a similar speed?", "limitations": "The authors have sufficiently addressed the paper’s limitations in Sec 4.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "+ [Originality] The paper is sufficiently novel in my opinion, with key architectural features well motivated by experimental psychology & neuroscience evidence and reasonably different from existing RNN & predictive coding based architectures (Sec 5).", "weaknesses": "- [Clarity] Although the overall writing is reasonably clear and easy to follow, the ambiguity in technical details renders accurate understanding of the paper impossible without digging into the source code. Examples are as follows.\n1) How exactly are the modulatory pathways (Sec 2.2) and subsequent forward passes executed? E.g., in LRM1, is (Conv4 -> Conv1) executed after or concurrently with (Output -> Conv4)? Is the modulatory signal $f$ applied to e.g. $x$ from the initial pass or $x’$ from the first modulatory pass (i.e. $x’ + x*f$, or $x’(1+f)$)?\n2) How exactly does the model (likely trained with 224x224 ImageNet images, Sec 2.3) handle both the overlay setting (same image size as training) and the side-by-side setting (2x image size) at the same time? \n3) AblationCam (Fig 3), output activation unit (Fig 6), $\\sigma\\pm$ (Sec 2.1), etc. are undefined.\n\n- [Quality] Empirical evaluation (soundness) is the main issue of the paper. While the proposed composite task and dataset are likely acceptable in psychology & neuroscience papers, it’s unfortunately not really sufficient for conferences like NeurIPS in my opinion. I strongly suggest the authors include additional experiments on more standard (commonly seen) CV datasets, such as [72, 73] or ones from [48-69], and comparisons against (some) existing approaches [48-71] whether they’re mechanistically similar to this work or not.\n\n- [Significance] Although the paper is sufficiently novel, given its non-negligible weaknesses in clarity and quality (soundness), it’s unfortunately hard to conclude that this work is significant (i.e. sufficiently promising). Brain-Score [74] could be a different direction to showcase the paper’s significance.\n\n[70] mixup: Beyond Empirical Risk Minimization, ICLR, 2018.\\\n[71] CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features, ICCV, 2019.\\\n[72] https://paperswithcode.com/dataset/clevr \\\n[73] https://paperswithcode.com/dataset/multi-dsprites \\\n[74] https://www.brain-score.org/", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Strengths:\n> *[Originality] The paper is sufficiently novel in my opinion, with key architectural features well motivated by experimental psychology & neuroscience evidence and reasonably different from existing RNN & predictive coding based architectures.*\n\nThanks!\n\n# Weaknesses:\n> *How exactly are the modulatory pathways (Sec 2.2) and subsequent forward passes executed?*\n\nThe general flow of information through the network can be best understood as consecutive feed-forward passes. In the initial feed-forward pass ( time $t$), the image input is processed sequentially through the deep neural network as usual. Layers that are the source of feedback to earlier layers will store their output activations, to influence destination layers on the next pass. During the next feed-forward pass ($t+1$)---also referred to in the paper as the \"modulated pass\"---each stage first computes its typical output, and then checks for any feedback influences ($f_{t}$) coming from the prior pass. The final output of this layer is calculated as $x_{t+1} + x_{t+1} \\times f_{t}$.  We have created a new section 2.1 (“Long-range feedback dynamics”), clarifying these important details, and made corresponding changes to Figure 1. \n\n> *How exactly does the model (likely trained with 224x224 ImageNet images, Sec 2.3) handle both the overlay setting (same image size as training) and the side-by-side setting (2x image size) at the same time?*\n\nOur models are able to handle side-by-side images without reducing the scale of the objects presented to the convolutional backbone, by using PyTorch’s built-in adaptive average pooling at the interface between the convolutional and fully-connected layers.  One level more detailed: objects in the side-by-side images (256x512 pixels) are presented at the same physical size (in pixels) to convolutional kernels as when a single image is presented (256x256). The adaptive average pooling operation resizes the output of the final convolutional layer to align with the first fully-connected layer (reshaping from Cx6x12 to Cx6x6). This operation does introduce a slight aspect ratio distortion at the conv5-fc6 interface, but the steering signal is still able to effectively operate over this image to recognize either category.\n\n>*AblationCam (Fig 3), output activation unit (Fig 6), $\\sigma\\$± (Sec 2.1), etc. are undefined.*\n\nReference to “AblationCam” added.  Fig 6 caption updated. Attempted further clarification on $\\sigma$± definition. Thanks for the careful read. \n\n>*[Quality] Empirical evaluation (soundness) is the main issue of the paper. While the proposed composite task and dataset are likely acceptable in psychology & neuroscience papers, it’s unfortunately not really sufficient for conferences like NeurIPS in my opinion. I strongly suggest the authors include additional experiments on more standard (commonly seen) CV datasets, such as [72, 73] or ones from [48-69], and comparisons against (some) existing approaches [48-71] whether they’re mechanistically similar to this work or not.*\n\nBased on your recommendation, we looked into many models, but as noted in the general rebuttal, most were either not available or trained in outdated coding frameworks (Caffe, LUA-Torch, MATLAB) and/or over simpler datasets (MNIST, Cifar10).  We further worked on implementing steering on a CORnet, even though it was not mechanistically similar to ours–based on your comment! Doing so also helped us clarify the unique contribution of our LRM models.  \n\nWe see the potential applicability to visual question answering tasks and multi-object array queries as standalone projects, fruitful directions for future work. Our present goal was to get the general infrastructure in place that allows a new approach to these sorts of applications. Which is why we strove for an implementation that is actually agnostic to the task (e.g. any model can have these connections added to it, trained on any objective).  In terms of connecting with CV more, you mentioned Mixup as an augmentation strategy–our overlay composites are essentially mixup with 50/50 blending. We also looked into CutMix, but found many images where the background and inset are no longer identifiable on their own, thus using CutMix this would require further curation. In general, we recognize this is not the most satisfying answer for you, but we think in part the challenge of comparisons here is that goal-directed visual encoding modulation is not (yet?!) an active area of study. \n\n> *[Significance] … Brain-Score [74] could be a different direction to showcase the paper’s significance.*\n\nIndeed, we plan to relate these models to brain data in future work. Until then, we offer that the design and demonstration of networks which can effectively cognitively steer their visual encoding of the input (with minimal hallucination), reflects a novel contribution.\n\n# Questions:\n> *How’s the model’s training & inference speed compared to the baseline? How does the model’s accuracy compare to stronger baselines (either AlexNet with more parameters, or newer networks) running at a similar speed?*\n\nThe model takes ~2x as long to run both at training and inference compared to the baseline model. We did not compare to an Alexnet with more parameters, as our goal was not to increase accuracy in the default operation, but to introduce a meaningful interface for top-down cognitive steering.\n\n# Summary:\nBased on your comments, we added much needed technical detail about the multiplicative modulatory operations. While we acknowledge that we have not added brain-score or other human-data measurements, we have attempted to clarify the novel computational contributions (e.g. with the comparison to the CORnet model), and hope that you might even be tempted to slightly increase your score. Thank you. \n"}, {"review_id": "w8WS6Sm8Rc", "reviewer": "Reviewer_kmpt", "summary": "This paper explores adding long-range modulatory feedback connections to deep CNNs (specifically AlexNet, evaluated on ImageNet). It explores 2 ways of incorporating the feedback: a default mode and a cognitive steering mode. The results show improvements on ImageNet accuracy, adversarial robustness, and a composite image recognition task.", "questions": "- What is the precise definition of \"modulatory\" used here? It seems like one could argue that any feedback connections are ultimately modulatory. What's the exact definition you're using such that other types of feedback connections *aren't* considered modulatory?\n- This isn't essential, but I'm curious (and I suspect some readers might be) -- is there something more biologically plausible about this version of feedback connections than some of the prior work?\n- How important is the fact that the steering is global vs. local? It might be worth discussing this more.\n- In Figure 3, what do you mean by \"full branch\"?\n- In Section 3.2, what label is used?\n- At the end of Section 3.1, could you further spell out why relevant features are amplified and irrelevant features are suppressed? It might be helpful to connect this more precisely to the mathematics/implementation, if added to Section 2.1 (as discussed in Weaknesses).\n- nit: typo in the second paragraph of Section 3.2 (where --> were)\n- In Section 3.3, you're using \"target\" as the label whereas it's previously used differently in the \"source/target\" description, right? If so, this dual use might not be optimal. Could you find a way to use 2 different words?\n- Is there possibly a better phrase than \"target absent category\"? It took me a little while to parse this. Do you have any ideas to help clear this up?", "limitations": "The paper includes a nice description of the limitations of this work, including limited exploration of different architectures and a lack of mechanistic analysis into why the long-range connections help.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The paper is high-quality: it's well-written, clear, the visualizations seem to have been very thoughtfully prepared, etc. The motivation for the modulatory connections is well-explained, and the empirical results (ImageNet accuracy, robustness, and cognitive steering effects) are compelling.\n- The authors anticipated most of my questions and responded to them in the body of the paper. For example, Section 2.2 has a great description of why certain decisions were made and which options were explored. \n- The experiments are explained clearly, and the visualizations really enhance the presentation.\n- I think this is a very significant question (how to incorporate long-range feedback into deep neural networks), which has been studied for many years but hasn't quite become mainstream yet. I applaud the authors for thoughtfully probing this question and taking a step toward bringing long-range feedback connections into modern neural networks, which I expect will be a quite impactful addition to NNs when it finally lands.", "weaknesses": "- In Section 2.1, I would have liked to see either mathematical equations or pseudocode to remove any ambiguity regarding the implementation of the feedback connections. The descriptions are decent, but I'm having to guess what the exact computations are. The implementation is the essence of the paper, in some sense. It would be nice to make this very precise in the body of the paper.\n- It would be good to have a thorough discussion of the costs associated with incorporating these connections (time, memory, etc.). Right now, the paper kind of reads like there's no reason *not* to use them, which probably isn't entirely fair. What am I losing if I incorporate these connections?\n- Although the Related Research section is nice, I would like to better understand which 1-3 works are most closely related to this one, with a more detailed description of how this implementation differs from these 1-3 most closely related prior works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Strengths:\n>*The paper is high-quality: it's well-written, clear, the visualizations seem to have been very thoughtfully prepared, etc. … I think this is a very significant question (how to incorporate long-range feedback into deep neural networks), which has been studied for many years but hasn't quite become mainstream yet.*\n\nThank you for your positive comments and evaluation of this work’s potential!\n\n# Weaknesses:\n> *In Section 2.1, I would have liked to see either mathematical equations or pseudocode to remove any ambiguity regarding the implementation of the feedback connections.* \n\nWe agree this was very unclear (see also R3). We have added a new Section 2.1 clarifying the “Long-range feedback dynamics.” We will also include a mathematical treatment of the modulatory motif in an appendix section, so people do not have to dig into the source code. \n\n> *It would be good to have a thorough discussion of the costs associated with incorporating these connections (time, memory, etc.).* \n\nGenerally the main cost of using these models is time: training is slower (~2x), as both the feed-forward and modulatory pass require different computations. Will add this. \n\n> *Although the Related Research section is nice, I would like to better understand which 1-3 works are most closely related to this one, with a more detailed description of how this implementation differs from these 1-3 most closely related prior works.\n\nThe two most relevant recent works are (1) Luo, et al. 2021 and (2) Nayebi et al., 2021\n\n(1) Luo, et al. have the same goal: to add category-based attentional steering to models.  However, their implementation does not leverage feedback connections.  Instead, they introduce an attentional weighting layer operation after conv4, where the weights must be learned for each category and each modulation strength (effectively 1000 * 5 different models). Their model also induces extreme false alarms with increasing attentional strength (hence their title about the “costs and benefits of goal-directed attention.”). \n\n(2) Nayebi et al., 2021 is complementary: they actually implemented a huge model search that included architectures with long-range feedback connections from later layers to earlier layers. However, as far as we can tell, the variation in modulatory motifs only included concatenation influences, and they were only focused on the goal of object recognition (not steering). \n\n\n# Questions:\n> *What is the precise definition of \"modulatory\" used here? It seems like one could argue that any feedback connections are ultimately modulatory.*\n\nTrue!  We intended to mean the activation of feedback does not induce activation in the absence of feedforward drive (e.g. $x + x*f = 0$ if $x=0$; this is not true with additive recurrent feedback for example, $x+f$). We have clarified this in a new Section 2.1.\n\n> This isn't essential, but I'm curious (and I suspect some readers might be) -- is there something more biologically plausible about this version of feedback connections than some of the prior work?\n\nWe’d like to think so, but in truth the detailed connectivity circuits of long-range feedback connections in biological visual system are not known (to our knowledge!). Recent insight into the causal effect of feedback from V4 onto V1, using a very clever optogentics multi-site recording protocol, was recently published in Science this year by Debes & Dragoi, 2023–and we designed our motifs to capture these phenomena. More work to do to compare to biological neural data under different attentional states. \n\n> *How important is the fact that the steering is global vs. local? It might be worth discussing this more.*\n\nGood question! We explicitly designed our feedback globally (channel-to-channel mixing) based on the visual cognition literature, and did not compare to alternative channel x space remixing models; we have added this as a limitation and direction for future work. \n\n> *In Figure 3, what do you mean by \"full branch\"?*\n\nWe meant the depicted branch of the tree that the Ringlet moth is sitting on. We changed the text to be more clear. \n\n> *In Section 3.2, what label is used?*\n\nWe are not quite sure what your question is exactly, but we did try to add clarity to this section following comments of Reviewer 1.\n\n> *At the end of Section 3.1, could you further spell out why relevant features are amplified and irrelevant features are suppressed?*\n\nWe can only speculate why, and further tests (or theoretical analyses) are required to understand this modulatory “edit” of the feed-forward features. We hypothesize that certain lower-level features are partially informative for some higher-level categories (and not others); thus later evidence of those higher-order categories could be projected back to amplify the earlier stage features (and similarly, suppress the less relevant features).  One possible way to test this would be to analyze the channel-to-channel remixing weights, to explore whether channels linked with positive weights are both more causally involved in recognizing the same categories. We are planning to work on these in future work. \n\n> *In Section 3.3, you're using \"target\" as the label whereas it's previously used differently in the \"source/target\" description, right? Could you find a way to use 2 different words? \n\nThanks for catching this! Feedback pathways now use “source/destination” terminology in Section 2.  And, “targets” now refer exclusively to category labels in Section 3. \n\n—\n(1) Luo, et al (2021). The costs and benefits of goal-directed attention in deep convolutional neural networks. Computational Brain & Behavior.\n\n(2) Nayebi, et al (2022). Recurrent connections in the primate ventral visual stream mediate a trade-off between task performance and network size during core object recognition. Neural Computation."}]

Author rebuttal:
# Summary of Strengths:

There was near consensus about the **novelty** of introducing cognitive steering capacities into deep neural network models through long-range modulatory feedback connections (R1,R3,R4; though see R2), with experimental results that are  **convincing**, **interesting**, and **compelling** (R1,R2,R4; though see R3), presented in a way that was generally **well-written** (R1,R2,R3,R4), with model design choices that were **well-motivated by experimental psychology and neuroscience** (R3, R4). R4 further notes the quality of visualizations and highlights the future potential of this work, stating that they “expect that bringing long-range feedback connections into modern neural networks… will be a quite impactful addition to NNs.” (Thanks!)

# Summary of Weaknesses:

*Most reviewers noted that adding feedback connections to deep neural networks is not itself new, and our experiments could more clearly relate to prior work (e.g. R1: “[clarify that] the contributions are in studying various steering methods/signals”; R2: “compare to CORnet”, R3: “include … comparison against other approaches”, R4: “include a more detailed comparison to most closely works.”)*

* **We agree.**   We attempted to collect available models with code and weights that use recurrence/feedback in object recognition tasks for direct comparisons. However, most papers we found did not publish code (e.g. 50, 51, 54, 70, 71, 72), had code without pretrained weights (e.g., 52, 56), had unofficial code with changes to the architecture and/or no pretrained weights (e.g., 48, 70), were designed and tested on simpler datasets (e.g. MNIST variations 70, 50, 72], Fashion-MNIST [56], Cifar10[73, 56]), or were implemented in outdated/less widely-used frameworks (e.g. Caffe[75], Lua/Torch[59], Matlab[49], MXNext[63]).  The work in [74] actually implemented long-range feedback connections (with a concatenation motif, no tests of steering); we contacted them and are now working to port their TensorFlow1 implementation into PyTorch. Thus, **we found there are actually relatively few (if any) available models that we could directly test for cognitive steering capacity.**


* Instead, we focused on **CORnet**, which had an open source implementation in PyTorch with pretrained weights.  CORnet does not actually have long-range feedback connections–instead it has within-layer additive recurrence. But, we tried to implement a form of steering anyway by directly injecting a steering signal into the state representation of the recurrent operation (at every layer, at every time step). This steering was able to boost recognition in overlaid composite tests, but also boosted false recognition when steering towards a category that was not present in the image.  This hallucination behavior is due to the *additive* way the recurrent state information is combined with the input in CORnet, which further helps to highlight a key advantage of our *multiplicative* modulatory feedback motif, which has minimal false alarms. **We report this as a new analysis (Section 3.5, Appendix A.4), and the figure is uploaded along with this response.** 


* Finally, we added a new paragraph in the introduction, to clarify earlier in the narrative that recurrence and feedback connections have been integrated into deep neural networks in much prior research, and discuss different feedforward-feedback motifs that have been used (e.g. additive, concatenation). The paragraph drives to a clearer statement of how “our work departs from these prior approaches, introducing long-range modulatory feedback pathways in order to support flexible "cognitive steering" of visual encoding.”
 
*Two reviewers requested *increased clarity for the details of the modulatory motif* and other technical details (R3, R4).* 

* **We agree**  and have added a new Section 2.1 clarifying the long-range feedback dynamics, as well as additional notation and edits to Section 2.2 (modulatory motif). Responses to R3 and R4 have further details. 

*Two reviewers asked why LRM models have *higher accuracy on the 0th pass* (R1 & R3).*
 
* **This is in fact a surprising and systematic result.** In these models, the objective of having both a good feedforward and a good modulated pass leads to *better* feed-forward features than a pure feed-forward objective. We highlight this point (Section 3.1), and speculate this arises due to better tuning alignment between features across hierarchical layers.

*The remaining weaknesses and questions were more idiosyncratic to individual reviewers and we address those more extensively below.  Thanks to the reviewers for their time and effort.*  

# Summary
Our work introduces cognitive steering into deep neural networks via long-range modulatory feedback, a novelty acknowledged by reviewers.  While adding feedback/recurrence is not new, our unique approach emphasizes "cognitive steering" of visual encoding (e.g., via memory representations or language instructions). We have benchmarked against a modified "Steerable CORnet", highlighting our model's advantages. We believe our contributions offer original advances for the field and hope for the committee's favorable consideration.


References:

[70] Mnih et al. Recurrent models of visual attention. NeurIPS, 2014.

[71] Alom et al. Inception recurrent convolutional neural network for object recognition. Machine Vision and Applications, 2021.

[72] Wang et al. Attentional neural network: Feature selection using cognitive feedback. Neurips, 2014. 

[73] Wang et al. Residual attention network for image classification. CVPR, 2017.

[74] Nayebi et al. Recurrent connections in the primate ventral visual stream mediate a trade-off between task performance and network size during core object recognition. Neural Computation, 2022.

[75] Fu et al. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. CVPR, 2017.


Meta-review: There was unanimous agreement among reviewers that understanding how to incorporate long-range feedback into deep neural networks is significant. There was initial concerns raised by the reviewers regarding experimental limitations but the rebuttal appears to address most of them. One reviewer maintained a relatively negative assessment because they felt that the paper did not make a sufficiently compelling case to justify how this new architecture allows for modeling behavioral data beyond previous models. However, the AC feels that there is enough support from the other reviewers (with one reviewer describing the paper as a "key stepping stone for the community") to warrant acceptance of the paper. Thus the AC recommends the paper to be accepted.
Decision: Accept (poster)