Towards flexible perception with visual memory
Authors: Robert Geirhos
Priyank Jaini
Austin Stone
Sourabh Medapati
Xi Yi
George Toderici
Abhijit Ogale
Jonathon Shlens
Venue: ICML 2025 poster (2025)
Keywords: retrieval
memory
interpretability
unlearning
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: We build a simple visual memory for classification that scales to the billion-scale regime, enabling a number of capabilities like unlearning and attributing model decisions to datapoints.

Abstract: Training a neural network is a monolithic endeavor, akin to carving knowledge into stone: once the process is completed, editing the knowledge in a network is hard, since all information is distributed across the network's weights. We here explore a simple, compelling alternative by marrying the representational power of deep neural networks with the flexibility of a database. Decomposing the task of image classification into image similarity (from a pre-trained embedding) and search (via fast nearest neighbor retrieval from a knowledge database), we build on well-established components to construct a simple and flexible visual memory that has the following key capabilities:
(1.) The ability to flexibly add data across scales: from individual samples all the way to entire classes and billion-scale data;
(2.) The ability to remove data through unlearning and memory pruning;
(3.) An interpretable decision-mechanism on which we can intervene to control its behavior.
Taken together, these capabilities comprehensively demonstrate the benefits of an explicit visual memory. We hope that it might contribute to a conversation on how knowledge should be represented in deep vision models---beyond carving it in "stone" weights.

Towards flexible perception with visual memory
Robert Geirhos"! Priyank Jaini" Yi
Austin Stone"
Sourabh Medapati Jonathon Shlens
George Toderici!
Abhijit Ogale'
Abstract
The real world constantly keeps changing, and do data requirements. New data and datasets become available, and existing ones become deprecated for a variety of reasons, including concerns around fairness, biases or unsafe content. Knowledge changes, and concepts drift (Tsymbal, 2004; Lu et al., 2018): Phones and cars look different today than they did few years ago, and different from how they will look in the future. When it comes data, the only constant is change (Cao & Yang, 2015; Bourtoule et al., 2021; Nguyen et al., 2022; Zhang et al., 2023). Consequently, from a modeling perspective, in order to keep up with this change one would ideally want to constantly re-train or fine-tune models, which is not feasible. In short, as anyone who has ever deployed a model has experienced firsthand, one con- stantly battling the symptoms of a single underlying cause: the fact that deep learning models have static knowledge representation entangled in millions or billions of model pa- rameters. We, among many others working on memory e.g. Weston et al. (2014); Chen et al. (2018); Wu et al. (2021); 1); Iscen et al. (2022); Nakata et al. (2022); Iscen al. (2023); Prabhu et al. (2023); Gui et al. (2024); Shao et al. (2024); Silva et al. (2024), believe that this is not a great way to represent visual knowledge for deep learning. Instead, we argue that we should build models that cleanly separate rep- resentation (how things are represented, e.g. through feature embeddings) from visual memory (what is known). In short, deep learning models need flcxible visual memory: a way to explicitly utilize and edit knowledge.
Training a neural network is a monolithic en- deavor, akin to carving knowledge into stone: once the process is completed, editing the knowl- edge in network is hard, since all information is distributed across the network's weights. We here explore a simple, compelling alternative by mar- rying the representational power of deep neural networks with the flexibility of a database. De- composing the task of image classification into image similarity (from a pre-trained embedding) and search (via fast nearest neighbor retrieval from a knowledge database), established components to construct simple and fiexible visual memory that has the following key capabilities: (1.) The ability to flexibly add data across scales: from individual samples all the way to entire classes and billion-scale data; (2.) The ability remove data through unlearning and memory pruning; (3.) An interpretable decision- mechanism on which we can intervene to control its behavior. Taken together, these capabilities comprehensively demonstrate the benefits of an explicit visual memory. We hope that it might contribute conversation on how knowledge should be represented in deep vision models- beyond carving in "stone" weights.
build on well-
In this work, we build simple visual memory for classi- fication and show that it has seven desirable capabilities, including the ability to flexibly add data across scales (from individual samples to classes and even billion-scale data), the ability to remove data from our model's classification process through machine unlearning and memory pruning, and a simple, interpretable
1. Introduction
In the pretty diagrams on "Intro to Machine Learning" slides, an ideal MI. workflow looks like this: Data collection, pre- processing, choosing a model, training, evaluation, deploy- ment. Happy ending the model is deployed, the users love it, and one can finally go on that well-deserved vacation and catch up on the latest AGI memes.
on which intervene to control its behavior. Our main goal is to provide a compelling idea of how beneficial a flexible visual memory for deep learning can be from variety of perspectives and capabilities. From technical standpoint, we aim for simplicity: retrieving i nearest neighbors (in an embedding feature space) along with their labels to classify query image. This approach allows us investigate where simple visual memory mechanism helps, where its limi-
Until, of course, the enemy of any ideal world sets in: reality.
Gooole DeepMind. Correspondence to: Robert Geirhos <last- name@ gomm Priyank Jaini <pjaini upjaini@googll.coom googlc.com>>
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Towards flexible perception with visual memory
made) is problematic. As an alternative, built working proof-of-coneept: By building on the long history of nearest neighbor methods, and "marrying" them with a powerful deep learning representation (such as SSL features from D2 and a billion-scale visual memory.
80
Neighbor accuracy (%)
75-
70
65
Related work. The concept of a visual memory has a long history in ML, neuroscience and psychology. In psychol- ogy, exemplar theory posits that humans recognize objects by comparing them to existing examples in visual memory (Medin & Schaffer, 1978; Nosofsky, 1986; Dopkins & Glea- son, 1997: Jäkel et al., 2008: Nosofsky, 201 I), like the AL- COVE model (Kruschke, 2020). In ML, prior to deep learn- ing, instance- based learning (also known as memory-based learning) was popular alternative to model-based learning (Aha et al., 1991; Quinlan, 1993). For instance, Turk & Pentland (1991) used nearest neighbor methods to classify faces, and Sivic & Zisserman (2003) build a visual memory inspired by text retrieval for object retrieval from videos. In recent years, hybrid approaches have started to combine the benefits of both approaches. Deep neural network variants (model-based since they learn generalized abstractions of data) of k-nearest neighbor algorithms (instance-based since they compare new data existing exemplars in memory) have been proposed with various motivations, including few- shot learning (Wang et al., 2019b; Yang et al., 2020; Bari et al., 2021), improving adversarial robustness (Sitawarin & Wagner, 2019; Papernot & McDanicl, 2018; Rajani et al., 2020), medical image classification (Zhuang et al., 2020), confidence calibration (Papernot & McDaniel, 2018), in- terpretability (Papernot & McDaniel, 2018; Wallace et al., 2018; Lee al., 2020; Rajani et al., 2020), image denoising (Plötz & Roth, 2018), retrieval-augmented learning (Khan- delwal et al., 2019; Drozdov et al., 2022), anomaly and out-of-distribution detection (Bergman et al., 2020; Sun et al., 2022). Recently, Nakata et al. (2022) tested a kNN- based visual memory up to ImageNet-scale (1.28M images), and Khandelwal et al. (2019); Wu et al. (2021) applied kNN- based approaches to neural language models. In contrast, we scale visual memory to the billion scale, improve ranking, and show systematic benefits across different tasks.
55
DinoV2 VIT-J14
DinoV2 VIT-3/14
50-
Dinov2 VIT-5/14
CLIP VIT-L/14
CLIP VIT-8/16
20
40
60
80 100
Neighbor index
Figure 1: Reliability of retrieved memory samples. This plot visualizes the ImageNet top-1 validation accuracy of a single retrieved neighbor depending on the index of the neighbor (index nearest neighbor). The decrease in ac- curacy with increasing neighbor index follows smooth tra- jectories and can be approximated by two-parameter loga- rithmic fit (black lines). More detailed plot in Appendix H.
tations may be, and where there might be opportunities for improvement through a more complex system. We hope that by demonstrating clear benefits from simple visual mem- ory, this article might contribute to conversation on how knowledge ought to be represented in deep vision models. Our contributions are as follows:
1. Our main contribution to demonstrate that a visual memory based system enables flexible capabilities, making progress towards some of deep learning's grand challenges: In specific settings, unlearning with perfect guarantees which is currently consid- ered a major open problem in deep learning, an im- proved understanding of how decisions relate to data- merision-mechaissn
points through an interpretable enabling data atrrbution, and controlling sample in- fluence through memory pruning. These capabilities highlight the promise of separating knowledge from representation through a visual memory.
2.
A simple visual memory performs well at scale: On the methodological side, we aim for simplicity. In- stead of re-inventing the wheel, we build on estab- lished building blocks from the literature like SSL features, KNN classification and fast, scalable similar- ity search. We contribute technical improvements like Rank Voting and VLM re-ranking, achieving 88.5% top-1 ImageNet validation accuracy which improves over both D2 VIT-L14 kNN and linear probing.
2. Building a retrieval-based visual memory for classification
Given a dataset Drest := (i.1.11,, ,,,,,,.),,,,....... }r we want to classify each image ã; E Drest- Our classification ap- proach consists of two steps: (i) building a visual memory, and (ii) using it for fast nearest neighbor based inference.
2.1. Building a visual memory
We argue that the way current deep learning models repre- sent knowledge (static knowledge representation, hard to update, hard to unlearn, hard to understand how decision is
Our visual memory retrieves (image, label) pairs from an image dataset when a query is made by directly retrieving those images that are considered similar to a test image
2

---

Towards flexible perception with visual memory
84
PluralityVoting DistanceVoting SoftmaxVoting
82
83.5
80
Tep-1 accuracy (%6
accuracy
78
82.5
76
82.0
74
DinoV2 VT--114
DinoV2 VTT8/14
DinoV2 V/T-
CLIP VIT-
CLIP ViT-8/16
80.5
0
40 Number of neighbors (k)
60
100
40
60
80
100
Number of neighbors (k)
(a) Existing aggregation methods, D2 VIT-L14
(b) Rank across models
Figure 2: Aggregating information across retrieved memory samples. (left) Existing aggregation methods are over- confident in distant neighbors, resulting in the paradox of decaying ImageNet-Ik accuracy with more information. The same pattern is also seen for other models and datasets in the Appendix (Figures and 8). (right) This not the case for Rank a power-function based method which reaches higher and stable performance across models and choices of k.
according to model. The model is a fixed pre-trained image encoder, meaning that no training takes place when adding information to visual memory. No copies of the dataset are stored in the visual memory. Instead, feature maps are extracted from the model based on a set of images related the downstream classification task at hand, such visual
the highest aggregate score. We implemented retrieval based classification using one of the following two approaches:
1. Fast inference using matrix multiplication on GPUs/TPUs: For smaller datasets like ImageNet, we saved VisualMemory as matrix of size num images num dims. During inference, for an encoded query image of size num.dims, we computed the dot product of this coded image with every entry in VisualMemory getting matrix of size mum_images x 1. We then computed the k nearest neighbors using the arg max operation.
standard training set. For experiments, memory comprises of features extracted from a dataset like the ImageNet-IK IK (Russakovsky et al., 2015) training set or JFT (Zhai et al., 2022) using different encoders like Dino (Oquab et al., 2023) and CLIP (Radford et al., 2021). Thus, given a pretrained image encoder, t and a dataset of (image, label) pairs Drrain = (x1, 91), (I2, 32)..... (Ø yN), obtain features 2: = ((x:;,, Væs, E Dirain- Subse- quently, the feature maps and corresponding label pairs are put in database thereby creating VisualMemory == {(z1,M)) (z2, ........ ..... (ZN, } } for classification. For both V2 and CLIP, we use the last image embedding layer as a feature space.
2. Fast and scalable nearest neighbor search: We used ScaNN (Guo al., 2020) for accelerating nearest neighbor search scale. Specifically, we saved the VisualMemory database and used ScaNN for fast lookup of nearest neighbors during inference. Storing features requires only about 1-3% of the space of storing the dataset itself and search latency is 500-600 QPS at perfect recall for 1M features, thus approximately 2 milliseconds per query due to parallelization which can be done on CPUs. See Appendix for details on latency and storage. This method scales easily to billion-scale memory (cf. Section 3.3).
2.2. Retrieval-based classification using visual memory
Given a query image t E Diest= we extract its fea- ture map, & = (()). We then query VisualMemory to extract k feature vectors, Neighbors(E)
We mentioned earlier that we retrieve a set of neighbors, Neighbors and aggregate information across them to make a classification decision. In order to understand how reliable (i.e., accurate) retrieved memory samples are from the first to the 100th neighbor, we systematically analyze neighbor reliability in Figure 1. As expected, reliability decreases as the neighbor index A increases, but even at large l the neighbors contain above-chance information about the ground truth class. This suggests that aggregating information across different neighbors may be beneficial to decision-making leading to the question: What the best
:= [(11) M/jj), (z (6. (2())) vWa)). that are closest to the query features & using the cosine distance, which is the default retrieval similarity measure for SSL models like D2. Nēi)h are ordered by distance i.c.
dist(ž, [1]) < dist(2, =b)), Vi S.
We then assign a weight, w;, to each neighbour (2 Maa) and aggregate the scores for each neighbour with the same label. Finally, we assign that label the query image with
3

---

Towards flexible perception with visual memory
aggregation strategy? We empirically study this by testing different weighting strategies for aggregation:
on aggregating information from different neighbors? We perform controlled experiment using the Gemini 1.5 Flash model (Reid et al., 2024) to test this: We add the 50 nearest neighbors from D2 VIT-L14 for query image along with their labels into Gemini's 's context. We then query Gem- ini to predict the query image's 's label. This achieves 88.5% ImageNet validation accuracy, substantial improvement over both Dino ViT-L14 KNN (83.5%) and linear probing (86.3% performance. Interestingly. Gemini's 's performance is mainly driven by the neighbor information through in- context learning since it only achieves 69.6% accuracy with- out neighbors (when just the query image is provided to the model). The performance improvement highlights the po- tential of using vision- langugge models as a visual memory re-ranker. Given that our main goal is to explore a simple visual memory system, we mostly focus on non-Gemini ranking methods throughout our analysis.
Plurality voting: Each neighbour in (ēighbors(:)) is as- signed an equal weight of 1.0. This is the classic, most simple voting method and used e.g. by Nakata et al. (2022).
as- signed a weight based on its Cosine distance to the query image ă i.e. uy exp (- dist(2, zg))). This approach has been used by Khandelwal et al. (2019) for nearest neighbor language models.
Distance voting: Each neighbour in Neighbors(@))
Each neighbour is assigned a weight based on the softmax function i.e. W, = (estmax((ist({= 2 r) where is the temperature. This voting method con- sidered ofate-of-the-art; for example nearest neighbor ac- curacies of self-supprvised models are reported using this method. A temperature of T 0.07 frequently appears in literature (Wu et al., 2018; Caron et al., 2021; Oquab et al., 2023) and is reported as a parameter "which we do not tune" in the Dino paper (Caron et al., 2021. p. 18). We observe that performance is sensitive to this parameter; other tem- peratures perform worse. We therefore follow the literature in using 0.07.
Softmax voting:
3. Capabilities of visual memory
Our primary goal is to motivate the concept of a machine visual memory from a variety of different perspectives. To this end, we investigate how such a memory can benefit the following capabilities: 3.1 Flexible lifelong learning: adding novel OOD classes; 3.2 Flexibly trading off compute and memory; 3.3 Flexibly adding billion-scale data without training: 3.4 Flexible removal of data: machine unlearning; 3.5 Flexibly increasing dataset granularity; 3.6 Flexible data selection: memory pruning; 3.7 Interpretable & attributable decision-makee making.
We propose using a simple aggregation ap- proach wherein each neighbour is assigned power-function weight based on its rank in the ordered set Neighbors( (z) ie. u. = 11((a= + rank,) where rank, is and a is an offset to avoid division by zero that is set to 2.0. This is sim- ilar, though not identical to, Gou et al. (2011) who used power-law weighting in different context.
Rank voting:
3.1. Flexible lifelong learning: adding novel 0OD classes (data and labels)
In Figure 2a, we compare the top-1 ImageNet validation accuracy of different ranking methods as function of num- ber of neighbours, with the ImageNet- 1K training set as the visual memory using the D2VoT2/Vi--L14 model as the featurizer. Paradoxically, existing aggregation methods like plurality voting, distance-based voting, and softmax voting show decaying performance as the provided information (number of nearest neighbors) increases. This suggests that the methods are overconfident in distant neighbors, assign- ing them too much weight. Our simple, parameter-fre rank based voting method, however, leads to an increase in per- formance with more neighbors until a certain & after which the performance plateaus, which is the ideal scenario (Fig- ure 2b). Furthermore, rank- based voting also outperforms baselines in absolute terms; quantitative comparisons can be found in the Appendix (Tables 4 to 8) where we also study the influence of hyperparameters (Figure 9). This indicates based method can reliably
Standard classifiers, whether trained end-to-end (supervised models) with linear classifier (self-supervised models), are not able to handle new information without re-training. For instance, adding new classes or changing labels in an existing model usually involves either re-training or fine- tuning parts of the model. A retrieval- visual memory, in contrast, is able to process such information in a natural and flexible way, aligning with the requirements of lifelong learning (Parisi et al., 2019). We tested this by adding data for 64 new classes, along with their new labels, to the visual memory of a pre-trained Dino ViT-L14 model (in addi- tion to the ImageNet train set, which is in-distribution for the model). We took the new classes from the NINCO dataset (Bitterwolf et al., 2023), a dedicated OOD dataset that is designed to have no overlap with existing ImageNet labels and samples. This requires the model to transfer what has learned to new, unseen concepts. The new task is therefore harder, as the model has to retrieve images from both in- distribution and OOD classes. The resulting visual memory has 1064 classes (IK from ImageNet and 64 from NINCO).
that a simple, integrate information across retrieved memory samples.
Gemini re-ranking. Our results above demonstrate that different aggregation strategies have large impact down- stream performance. How far can we push the upper limit
4

---

Towards flexible perception with visual memory
DineV2 VIT-L/14
DinoV2 VIT-L/14
60
DinoV2 VIT-B/14
DinoVz VIT-S/14
60
DinoV2 VIT-
Top-1 (%6)
50
CLI VIT-L/14
VIT-B/16
Top-1 rate
40-
30
30
20
1K
10K
100K
1.28M
1K 10K
100K
1M
10M
100M
18
Number of images memory
Number images memory
(a) ImageNet (million-scale)
(b) JFT (billion-scale)
Figure 3: Memory scaling: flexibly trading off' compute and memory. ImageNet top-1 validation error decreases systematically as the memory size is increased (i.c., recognition accuracy increases with scale). (left) Million-scale memory consisting of ImageNet- trainn labels. (right) Billion-scale memory bank consisting of machine- geeeinteee pscudo labels on the JFT dataset (Zhai et al., 2022). Accuracy continues decrease even with billion-scale data in memory. The roughly constant offset between models of different sizes suggests the possibility of flexible trade-off: The same error rate can be achieved with small model and large memory, or a large model and small memory.
Table 1 shows that with visual memory it is possible to add new classes such that the in-distribution accuracy is main- tained without catastrophic forgetting (the new classes only change ImageNet validation performance by 0.02-0.049 depending on the aggregation method), while the same time reaching very high accuracy on the new OOD classes (approx. 87% top-1) without any training. Figure 12 in the appendix confirms that the samples are indeed OOD for the model, as demonstrated by larger distances to nearest neigh- bors. This highlights that a visual memory is capable of flexibly adding new information-an important capability since the world is not static. Furthermore, the memory is highly robust towards label corruption for up to 60% random labels, as shown in Appendix D.
3.2. Flexibly trading off compute and memory
Next, we turn our attention to studying the scaling behaviour of visual memory with increasing memory model size. We specifically test whether smaller models larger memory can match the performance larger models with smaller mem- ory. This is because, all else being equal, bigger model might require fewer examples in memory represent dif- ferent concepts. We empirically study the scaling behaviour of visual memory based retrieval systems in Figure 3a using models of different sizes like Dino ViT models of sizes S/14 (21M params), B/14 (86M params), and L/14 (300M params), as well as CLIP VIT models of sizes B/16 and L/14. We plot the top-1 error rate as function number of images in visual memory. The plot demonstrates that for each model, the error rate consistently decreases increase the visual memory size. Notably, already with single exemplar per class in memory, ImageNet validation performance is well above chance (41 % top-1 error for Di- no ViT-L14).. It also suggests the possibility of flexible trade-off between model size and memory size: e.g. for the different DinoV2 models, the S/14, B/14, and L/14 variant achieve similar performance 1.28M, ~ 150K, and 70KK memory capacity respectively. This indicates that smaller model with large memory can match the performance of larger model with smaller memory.
Table I: Flexible lifelong learning: adding OOD classes. A visual memory of D2 ViT-L14 with ImageNet-train (IN-train) memory database is able to handle simple "insert into memory" operation for 64 out-of-distribution classes (data and labels) from the NINCO dataset (Bitter- wolf et al., 2023), leading to high performance on new classes without harming existing ones.
memory +
IN-train IN---An-and--IINCO
query + IN-val
IN-val
NINCO
aggregation
81.1
81.1
86.4
3.3. Flexibly adding billion-scan data without training
Plurality Voting
83.2
83.2
86.9
Distance Voting
83.3
83.3
87.1
Billion-scale dataset with pseudo labels. As demonstrated in Section 3.2, performance systematically improves with increased memory size across both small and large models.
Softmax Voting
83.6
83.5
87.5
Rank Voting
83.6
83.6
87.4
5

---

Towards flexible perception with visual memory
We here test how far this trend holds beyond relatively small- scale, well-curated settings like ImageNet-11 KK by scaling visual memory to the billion-scale unlabeled data regime. Billion-scale search is known to be fast (Johnson et al., 2019; Guo et al., 2020; Chen et al., 2021; Khan et al., 2024), but it is an open question how well a billion-scale memory with a modern featurizer like V2 performs for classification. In order to test predictive performance at scale, we obtain a billion-scale dataset from the union of the ImageNet- IK train set and a subset of the JFT-3B dataset (Zhai et al., 2022). To this end, we treat JFT as an unlabeled dataset by ignoring its original labels and instead obtaining psendo labels via 1T1-22B-224pp (Dehghani et al., 2023), a highly performant classifier. We excluded images whose labels do not have a correspondence with ImageNet labels.
influence of specific training data from model's decision- making process after it has been trained (Cao & Yang. 2015;
Bourtoule et al., 2021; Nguyen et al., 2022; Zhang et al., 2023). A range of intricate methods are being developed to remove or reduce the influence of certain training sam- ples (Gupta et al., 2021; Sekhari et al., 2021; Ullah et al., 2021; Kurmanji et al., 2024; Sepahvand et al., 2024)-a challenging endeavour if knowledge is embedded in mil- lions billions of model weights. In contrast, for models with an explicit visual memory and in the context of classifi- cation, machine unlearning becomes as simple as removing the dataset sample from the visual memory. For instance, after adding the NINCO dataset (Bitterwolf et al., 2023) into visual memory, we can remove any NINCO sample with outstanding performance on all three key unlearning metries reported by Liu (2024): Efficiency: How fast is the algorithm compared to re- training? (Very fast: less than 20 milliseconds for deleting a feature/sample from disk.) Model utility: Do we harm performance on the retain data or orthogonal tasks? (By design not at all.) Forgetting quality: How much and how well are the 'forget data' ac- tually unlearned? (Completely and entirely by design.) In comparison, the winner of the NeurIPS 2023 machine un- learning challenge (Triantafillou et al., 2024) still suffers from an accuracy gap of 4%, low forgetting quality, and being relatively inefficient requiring 8 epochs of training.
Scaling, In Figure 3b, we show the downstream ImageNet validation performance of two D2-oV2-ViT's as function of memory size. The plot demonstrates that even in the billion-scale data regime, validation error decreases when increasing memory size without any training. In log-log space, a logarithmic function fits the empirical scaling trend well. In the literature, simple scaling trends such as the one we observe are powerful predictors of scaling behaviour for different model and dataset sizes (Hestness et al., 2017: Kaplan et al., 2020; Hoffmann et al., 2022). This experi- ment confirms that a memory-based system is performant across seven orders of magnitude, improving from both better features and more data.
Can machine unlearning therefore be solved with a visual memory? If the embedding model is trained on data that needs to be unlearned, machine unlearning remains chal- lenging. If, however, the embedding model is trained on safe, generalist dataset (c.g.. publicly available image dataset) and data that may need to be considered for un- learning later is put into the visual memory, then machine unlearning for classification indeed becomes as simple as deleting a datapoint from the visual memory. This can be particularly helpful for tasks that may require private or confidential data: a model can be trained on publicly avail- able datasets to learn general and information features and the private data can be added to a visual memory on local devices for downstream tasks to preserve privacy.
dustr--iistribution performance. In order to understand whether the benefits of increased memory size transfer to out-of-distribution (OOD) data, we compare D2 ViT- L14 once with ImageNet- traain in memory and once with JFT pseudo-labels in memory. The models are evaluated on the ImageNet-A (Hendrycks et al., 2021), ImageNet-R (Hendryeks et al., 2020), ImageNet-Skeree 2019a), ImageNet- V2 (Shankar et al., 2020), and ImageNet- Real. (Beyer et al., 2020) datasets. As additional well- performing yet "inflexible" baseline, we report linear prob- ing accuracies from the D2noV2 paper (Oquab et al., 2023). Table 2 shows that visual memory scaled with JFT data improves OOD performance across all datasets compared to an ImageNet-based visual memory. Gemini re-ranking again leads to performance gains. Overall, the finding that memory scale transfers to OOD improvements is important in the context of continual learning. where flexible visual memory can easily incorporate newly available data that the model was not trained on, improving performance both in- and out-of distribbbu.....
(Wang et al.,
3.5. Flexibly increasing dataset granularity on iNaturalist
In contrast to static classification, where a model is trained once without updates, a visual memory model should be able to flexibly refine its visual understanding as more in- formation becomes available. We test this using DinoV2 VIT-L14 embeddings on the iNaturalist21 dataset (iNatural- istTeam, 2021), large-scale imbalanced dataset of animal and plant images containing 10,000 species spanning seven taxonomic levels, from coarse (kingdom) to fine-grainer -rained (species). In a leave-one-out fashion, we simulate the dis- covery of a new species by putting 50 exemplars for each of
3.4. Flexible removal of data: machine unlearning
The world is not static. Thus, in addition to the need to flexibly add novel data, it is often desirable to remove the
6

---

Towards flexible perception with visual memory
Table 2: OOD evaluation. Out-of-distribution performance improves with larger visual memory size. Across all datasets, a visual memory with JFT memory outperforms ImageNet memory demonstrating advantages of scaling visual memory for OOD performance. Probe details: Appendix J.
Model
Method
IN-A
IN-R
IN-Sketch
IN-V2
IN-Real.
D2 ViT-L14
linear probe
71.3
74.4
59.3
78.0
89.5
D2 ViT-L14
ImageNet memory
58.8
62.8
61.5
75.6
87.1
+ Gemini re-ranking
68.4
72.3
72.5
81.7
89.9
V2 ViT-L14
JFT memory
61.1
73.7
68.0
77.6
88.2
Gemini re-ranking
69.6
81.4
75.0
82.3
90.5
the 9,999 species into memory and then iteratively adding more data for the remaining "newly discovered" species-
100
starting from zero exemplars all the way to 50 exemplars.
80
Classification is performed using =1 nearest neighbors
Top-1 accuracy (%)
samples in memory
(no aggregation).
60
In Figure 4 we observe the following: (1.) Already before a single example of the new species is added, it can already be placed in the right part of the taxonomic tree well beyond chance (35.2% accuracy the genus level compared to ~0% chance). (2.) Accuracy at the species level improves substantially by adding just handful of images of the tar- get species (e.g., 5-10 images); regime where training classifier would typically fail due to data scarcity. (3.) In- terestingly, adding more samples of the discovered species not only improves species-level accuracy, but also leads to "rising tide lift" of improvements across all levels of the taxonomic hierarchy. This indicates that a visual memory is well-suited for hierarchical classification tasks and settings where data for new concepts is initially scarce but becomes more abundant over time twich is often the case in ap- plications like fraud detection, personalized recommender systems, and scientific discovery.
2
40
5
10
20
20
30
40
50
kingdom
phylum
class
order
family
genus
species
Figure 4: Impact of memory bank size on top-1, k=1 accuracy across taxonomic levels on iNaturalist. Top-1 accuracy for a target species across different taxonomic lev- els the number of exemplars in the memory bank for that species increases from 0 to 50. Each line represents the average accuracy over all 10,000 species in the iNaturalist 2021 dataset, while the number of examples in visual mem- ory is fixed at 50 exemplars for all other species. The black dotted line indicates baseline accuracy from predicting the majority class.
3.6. Flexible data selection: memory pruning
The ability to flexibly remove the influence of certain data- points is not just desirable in the unleamiin sense, but also advantageous in the context of dataset pruning, an emerg- ing field that analyzes the quality of individual data points. The goal of dataset pruning is to retain only useful sam- ples, while removing those that have neutral harmful effect on model quality. The key challenge is that in stan- dard black-box models, is entirely unclear whether any given sample is helpful or harmful. The gold standard is saae-oon-ouu-trrining (for ImageNet, this would consist of training 1.28 million models); current methods seek to approximate this extremely costly approach with various heuristics (Feldman & Zhang, 2020; Chitta et al., 2021; Paul et al., 2021; Sorscher et al., 2022; Abbas et al., 2023a). By contrast, the contribution of a data sample to decision-
making in visual memory based system is straightforward. For any given query image ã, the neighbor set Neighbors( fully describes which samples contributed to the decision. Furthermore, this information also highlights whether the samples were helpful (correct label) or harmful (wrong la- bel) for the decision. We, therefore, transfer the concept of dataset pruning to memory, and propose visual mem- ory pruning. To this end, we estimate sample quality by querying the ImageNet training set against visual memory consisting of the exact same dataset (IN-train, discarding the first neighbor which is identical to the query). This approach requires no more compute than a single forward pass over the training set. We then record the number of times any given neighbor contributed wrong decision, resulting
7

---

Towards flexible perception with visual memory
query
reighber
neighbrr
neighber s
neighbor&
neighbor
query
neighbor
neighbor
neighbor
neighbor 4
neighbor S
sbap bonel
Weoden speon
Iodle
com
Edip bast
toup bost
teddy
thatch
worm fence
seashore
worm Fence
worm fence
based
0.23
Prediction
mooden spoore E28, soup sawel 0.25 pred ciar wooden apoon (wrong)
Robubalty-aa
0.22
0.77(1
6,1
peuning: soup wooden spoen 0.2% ction (comnet)
chain
soap disp.
washbasin
washbasn
tub
weshbasin
Visualization of memory-based decision- making with and without memory pruning. Given a query image, nearest neighbors are retrieved from mem- ory via Cosine similarity the embedding space of model (here: five closest neighbors from the ImageNet train set, embedded via D2 ViT-L14). The model's prediction is based on the weighted aggregation of the neighbor class labels. The rank-based weight decreases with the rank of the neighbor. For soft memory pruning, those weights are adjusted by the reliability of their neighbors. the specific example here, all five neighbors appear sensible, but they have four different labels. Since the first two neighbors contributed to wrong decisions the training set, they are downweighted via soft memory pruning, and the prediction changes to the correct class.
Figure 5:
caldron
matchhtickk
coffee mag
FB
bow lie
unioydee
Ur uricyde
unioycla
unicydle
bicycle- buil- fon-two
Figure 6: Interpretable decision-making, A visual mem- ory enables a clear visual understanding of why model makes a certain prediction. Here, we show four ran- domly selected misclassified query images from ImageNet- A (Hendrycks al., 2021) with five nearest neighbors from DinoV2 ViT-L14 using the ImageNet-IK training set as memory. Labels are from the respective datasets (query: ImageNet-A; neighbors: ImageNet-train), Visually, all neighbors appear reasonable, but not all labels do.
in a sample quality estimate. This enables us to exclude low-quality neighbors from the decision-making process by either removing them from the visual memory entirely ("hard memory pruning") or by reducing their weight com- pared higher-quality neighbors ("soft memory pruning"). Method details can be found in Appendix I. In Table 3, show that both memory pruning variants improve ImageNet validation accuracy, with soft pruning leading to larger gains than hard pruning. Figure 5 visualizes the decision-making process for a randomly selected sample where estimating sample reliability improves decision quality. Given that observing the outcome of an intervention many orders of magnitude faster than traditional seye--ou-ttrininn are optimistic that the visual memory pruning gains we ob- served with two simple strategies can be improved further in the future.
et al., 2020; Shankar et al., 2020; Yun et al., 20 rather than about model quality. This issue is quantified through human experiment in Appendix N, showing that 2 out of 5 model "errors" on this dataset are instead label errors.
In a separate human experiment (described in Appendix L), we tested whether access to five nearest neighbors helps humans predict model decisions. In this experiment, for each image we provided four label choices including the ground truth label and the model predicted label (if differ- ent). The remaining labels were plausible alternatives based on top CLIP predictions for the ImageNet-A test image. For black-box classifier (no access to neighbors), human accu- racy was 56%; this accuracy improved to 83% when given access four nearest neighbours from visual memory with D2noV2 VIT-L14 featurizer, providing strong and falsifiable evidence for the improved interpretability of visual memory system.
3.7. Interpretable & attributable decision-making
Unlike a black-box deep learning model, a visual memory offers a natural way to understand model's specific pre- dictions by attributing them to training data samples (e.g. Papernot McDaniel, 2018). Figure 6, we visualize misclassified validation set examples from the ImageNet- dataset (Hendrycks et al., 2021) using a memory of the ImageNet- K training set. These randomly selected samples illustrate that many seemingly strange enrors (e.g., predict- ing type of fence instead of teddy bear, or a unicycle instead of a bow tie) are in fact sensible given the data, raising questions about label quality of ImageNet-A similar vein as label issues identified for ImageNet (Beyer
4. Discussion
Summary. Typical neural networks are trained end-to-end: perfect for static settings, yet cumbersome update when- ever knowledge changes. This is limiting their real-world potential since our world is constantly evolving. Incor- porating a visual memory, in contrast, enables a range of flexible capabilities that embrace change: lifelong learning through incorporating novel knowledge, being able to forget,
8

---

Towards flexible perception with visual memory
Table 3: Flexible data selection: memory pruning, ImageNet validation accuracy improves when removing low-quality samples (hard pruning) or downweighting them (soft pruning). In contrast to standard black-box models, memory models (here:
using D2 ViT-I.14) offer a strikingly simple way to estimate sample quality since their decisions are based on a few retrieved memory samples.
Pruning
Plurality
Distance
Softmax Voting
Rank Voting
no pruning (standard)
83.2
83.3
83.6
83.6
hard pruning (ours)
83.3
83.4
83.6
83.7
soft pruning (ours)
83.6 (+0.4%)
83.6 (+0.3%)
83.9 (+0.3%)
84.1 (+0.5%)
remove and unlearn obsolete knowledge, flexible data selec- tion through memory pruning, and an interpretable decision- making paradigm which one can intervene to control its behavior. We systematically explored a simple visual mem- ory that decomposes the task of image classification into two primitives, image similarity (from a pre-trained embed- ding representation) and search (via fast, scalable nearest neighbor search from a vector database). Our results demon- strate that technical improvements like Rank Voting improve kNN accuracies for both D2 and CLIPP over the widely used Softmax Voting method that is sensitive to two hyper- parameters (temperature 7 and neighbors k). Our approach also narrows the accuracy gap between a nearest neighbor memory (best flexibility, perfect unlearning, improved in- terpretability) and a fixed linear probe (highest accuracy on static image classification). Importantly, we show that visual memory enables flexible perceptual capabilities.
stronger its limitations are felt. While the static nature of end-to-end trained networks can easily be forgotten when focusing on fixed academic benchmarks, the real world is anything but static. Data is constantly evolving, leading to the dreaded "model drift" where once-optimal models grad- ually become less effective (Bayram et al., 2022). Incorpo- rating an explicit visual memory appears to be a promising way forward for real-world tasks where flexibility is key. While the specific approach we employ here might well be improved through more complex systems, we hope that the flexible capabilities we demonstrated might inspire and contribute to conversation on how knowledge ought to be represented in vision models.
Impact Statement
The use of flexible visual memory as proposed in this arti- cle has several positive societal implications: (1.) through improved machine unlearning with strong guarantees (by design), it becomes possible for certain classifiers to easily remove data that is no longer considered safe (e.g., comply- ing with user requests or in order to fix data safety issues that are discovered after a model is deployed); (2.) increased transparency and trust through a decision-making mecha- nism that can be inspected visually and intervened on as opposed to opaque black-box machine learning system; (3.) potentially increased privacy by storing sensitive data in memory (e.g. on-device) as opposed to training models on the data; (4.) resource efficiency: since fast similarity search can be done on CPUs with a low energy footprint, this sug- gests possibilities for reducing a system's 's carbon footprint (e.g., combining small model with larger database).
Limitations and future work. First, we only considered the task of image classification across a broad range of datasets. It will be interesting to extend the approach to other visual tasks, such as object detection, image segmentation, instance recognition and to image generation where a visual memory would be desirable, too (since it is prohibitively expensive to re-train large generative models every time data needs to be removed or added). Secondly, our approach relies on fixed, pre-trained model; strong distribution shifts may require updating the embedding. Self-supervised mod- els are a particularly flexible choice, but it is an open ques- tion whether one could train smaller models that excel at their task with the help of larger memory database. Con- ceptually, if model needs to save less information in its weights, it might be possible reduce the computational footprint of such model. Additionally, for scalable vector search, adding/removing samples can require adapting the search index, though the amortized cost is low. Furthermore, we sometimes observe a trade-off between flexibility and accuracy. The use of memory pruning weights as a data se- lection criterion in the context of dataset pruning (Sorscher et al., 2022) might be an interesting avenue for future work.
Given that the system we explore is used for classification, standard classifier- risks apply, such as for instance the potential for a system to discriminate, amplify biases, or displace human jobs. In addition, the general dual-use problematic of ML models applies: the same system can often be used for beneficial as well as harmful purposes.
We expect that the societal benefits outweigh the harms since the potential harms are shared with all ML classifiers, while the benefits are specific to our approach.
Outlook. Deep learning is increasingly becoming a vic- tim of its own success: the more widely it is deployed, the
9

---

Towards flexible perception with visual memory
Code availability. Code to replicate experiments from this paper is available https://g wttb.i.hooooom. google deepmidd//iiuull
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101-mining discriminative components with ran- dom forests. In ECCV 2014, pp. 446-461. Springer, 2014.
Lucas Bourtoule, Varun Chandrasekaran, Christopher Choquette- Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine learning. In 2021 IEEE Symposium Security and Pri- vacy (SP), pp. 141-159 IEEE, 2021.
Acknowledgements
We would like to thank Kevin Swersky, David Fleet, Been Kim, Hagen Soltau, Majid Hadian, Kaifeng Chen, Mojtaba Seyedhosseini, Ameya Prabhu and Sung Jin Hwang for feedback and discussions, and Mike Mozer for helpful com- ments on our article draft. We also thank our anonymous reviewers who provided valuable feedback that improved the paper.
Wieland Brendel and Matthias Bethge. Approximat- ing CNNs with bag-of-local-features models works surprsingly well on ImageNet. arXiv preprint
00Kiv:/90400070000 2019.
References
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp. 463 480. IEEE, 2015.
Amro Kamal Mohamed Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, and Ari S Morcos. Effective pruning of web-scale datasets based complexity of concept clusters. In The Twelfih International Conference on Learning Representations, 2023a.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transform- ers. In Proceedings of the IEEE/CVF international con- ference computer vision, pp. 9650-9660, 2021.
Amro Kamal Mohamed Abbas, Kushal Tirumala. Daniel Simig, Surya Ganguli, and Ari S Morcos. SemDeDup: Data-efficient learning at web-scale through semantic deduplication. In ICLR 2023 Workshop on Mathemati- cal and Empirical Understanding of Foundation Models, 2023b.
Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuan- jie Liu, Zengzhong Li, Mao Yang. and Jingdong Wang. Spann: Highly-efficient billion-scale approximate nearest neighborhood search. Advances in Neural Information Processing Systems, 34:5199-522 2021.
Yanbei Chen, Xiatian Zhu, and Shaogang Gong. Semi- supervised deep learning with memory. In Proceedings of the European conference on computer vision (ECCV), pp. 268-283, 2018.
David W Aha, Dennis Kibler, and Marc K Albert. Instance- based learning algorithms. Machine learning, 6:37-66, 1991.
M Saiful Bari, Batool Haider, and Saab Mansour. Nearest neighbour shot learning for cross- lingual classifica- tion. arXiv preprint arXiv:2109.0222// 2021.
Kashyap Chitta, José M Álvarez, Elmar Haussmann, and Clément Farabet. Training data subset search with en- semble active learning. IEEE Transactions on Intelligent Transportation Systems, 23(9): 2021.
Firas Bayram, Bestoun S Ahmed, and Andreas Kassler. From concept drift to model degradation: An overview on performance-aware drift detectors. Knowledge-Baud Based Systems, 245: 20S:22 2022.
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Pi- otr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alab- dulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Amab, Xiao Wang, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey Gritsenko, Vighnesh Birodkar, Cristina Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetić, Dustin Tran, Thomas Kipf, Mario Lučić, Xiao- hua Zhai, Daniel Keysers, Jeremiah Harmsen, and Neil Houlsby. Scaling vision transformers to 22 billion param- eters. In International Conference on Machine Learning, 2023.
Liron Bergman, Niv Cohen, and Yedid Hoshen. Deep nearest neighbor anomaly detection. arXiv preprint arXiv:2002.10445, 2020.
Lueas Beyer, Olivier J Hénaff, Alexander Kolesnikov, Xi- aohua Zhai, and Aäron van den Oord. Are we done with imagenet? arXiv preprint wrXiv:2006.07159 2020.
Julian Bitterwolf, Maximilian Müller, and Matthias Hein. In or out? Fixing ImageNet out-of-distributin detec- tion evaluation. In International Conference on Machine Learning, pp. 2471-2506 PMLR, 2023.
10

---

Towards flexible perception with visual memory
Stephen Dopkins and Theresa Gleason. Comparing exem- plar and prototype models of categorization. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 51(3):212 1997.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large lan- guage models. arXiv preprint M.XXv:2203.155555 2022.
Andrew Drozdov, Shufan Wang, Razich Rahimi, Andrew McCallum, Hamed Zamani, and Mohit lyyer. You can't pick your neighbors, or can you? when and how to rely on retrieval in the k nn-lm. arXiv preprint arXiv:2210. 15859, 2022.
NaturalistTeam. iNaturalist 2021 competition dataset. http:///wwniiiiiaaoooooot/e///ea
2021.
Ahmet Iscen, Thomas Bird, Mathilde Caron, Alireza Fathi, and Cordelia Schmid. A memory transformer network for incremental learning. arXiv preprint 4rX1v:2210.04485, 2022.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influ- ence estimation. Advances in Neural Information Pro- cessing Systems, 33:2881-2888 1-891, 2020.
Ahmet Iscen, Mathilde Caron, Alireza Fathi, and Cordelia Schmid. Retrieval-ehhanced contrastive vision-text mod- els. arXiv preprint arXiv:2306.07196 2023.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Bren- del. ImaaeNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In ICLR, 2019.
Frank Jäkel, Bernhard Schölkopf, and Felix A Wichmann. Generalization and similarity in exemplar models of cate- gorization: Insights from machine learning. Psychonomic Bulletin & Review, 15:256-271, 2008.
Jianping Gou, Taisong Xiong, Yin Kuang, al. A novel weighted voting for k-nearest neighbor rule. Journal of Computers, 6(5):333-840, 2011.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion- scale similarity search with gpus. IEEE Transactions on Big Data, 733)5535-547, 2019.
Zhongrui Gui, Shuyang Sun, Runjia Li, Jianhao Yuan, Zhao- chong An, Karsten Roth, Ameya Prabhu, and Philip Torr. KNN-CLIP: Retrieval enables training-free segmen- tation on continually expanding large vocabularies. arXiv preprint arXiv:2404. 4444444 2024.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scal- ing laws for neural language models. arXiv preprint arXiv:2001.0836/1 2020.
Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. Accelerating large-scale inference with anisotropic vector quantization. In International Conference on Machine Learning, pp- 3887-3896. PMLR, 2020.
Saim Khan, Somesh Singh, Harsha Vardhan Simhadri, Jyothi Vedurada, et al. BANG: billion-scale approxi- mate nearest neighbor search using a single gpu. arXiv preprint arXiv:2401.11324 2024.
Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharif--Malvaeeddi, and Chris Waites. Adaptive machine unlearning. Advances in Neural Information Processing Systems, 1111619--6338 2021.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettle- moyer, and Mike Lewis. Generalization through memo- rization: Nearest neighbor language models. In Interna- tional Conference Learning Representations, 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Ka- davath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robust- ness: A critical analysis of out-o--disti generaliza- tion. arXiv preprint arXiv:2006.1622 2020.
John K Kruschke. ALCOVE: an exemplar-based connec- tionist model of category learning. In Connectionisl psy- chology, pp. 107-138. Psychology Press, 2020.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Stein- hardt, and Dawn Song. Natural adversarial examples. In CVPR. pp. 15262-15271, 2021.
Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unleam- ing. Advances in Neural Information Processing Systems, 36. 2024.
Joel Hestness, Sharan Narang, Newsha Ardalani, Gre- gory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv: 00409, 2017.
Ritchie Lee, Justin Clarke, Adrian Agogino, and Dimitra Giannakopoulou. Improving trust in deep neural networks with nearest neighbors. In A/AA Scitech 2020 Forum, pp. 2098, 2020.

---

Towards flexible perception with visual memory
Ken Ziyu Liu. Machine unlearning in 2024, Apr 2024. URL https: //ai. stanford.d..o.o... wttprrdeeddd///////l...oo
Ameya Prabhu, Zhipeng Cai, Puneet Dokania, Philip Torr, Vladlen Koltun, and Ozan Sener. Online continual learning without the storage constraint. arXiv preprint arXiv:2305.09253, 2023.
unlearning.
Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Joao Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE transactions on knowledge and data engi- neering, 31(12):2346-2363, 2018.
Ross Quinlan. Combining instance-based and model- based learning. In Proceedings of the tenth international conference machine learning, pp. 236-243, 1993.
Douglas L Medin and Marguerite M Schaffer. Context theory of classification learning. Psychological review, 85(3):207, 1978.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural lan- guage supervision. In ICML, pp. 8748-8763, 2021.
Kengo Nakata, Youyang Ng, Daisuke Miyashita, Asuka Maki, Yu-Chieh Lin, and Jun Deguchi. Revisiting a knn-based image classification system with high-capacity storage. In European Conference on Computer Vision, pp. 457-474. Springer, 2022.
Nazneen Fatema Rajani, Ben Krause, Wengpeng Yin, Tong Niu, Richard Socher, and Caiming Xiong. Explaining and improving model behavior with k nearest neighbor representations. arXiv preprint arXiv:2010.09030) 2020.
Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXiv:2209.02299, 2022.
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean- baatttap Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrit- twieser, al. Gemini 1.5: Unlocking multimodal un- derstanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 2024.
Robert M Nosofsky. Attention, similarity, and the seeniffcationn-attegorriiai relationship. Journal of ex- perimental psychology: General, 115(1):39, 1986.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpa- thy, Aditya Khosla, Michael Bernstein, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115:211-2551 252, 2015.
Robert M Nosofsky. The generalized context model: An exemplar model of classification. Formal approaches in
categorization, pp. 18-39, 2011.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaacldin El-Nouby, al. DINOv2: Learning robust visual features without su- pervision. Transactions on Machine Learning Research, 2023.
Ayush Sckhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Processing Systems, 34:18075 18086, 2021.
Nicolas Papernot and Patrick McDaniel. Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. arXiv preprint S053Iv::803.0476660 2018.
Nazanin Mohammadi Sepahvand, Vincent Dumoulin, Eleni Triantafillou, and Gintare Karolina Dziugaite. Data selection for transfer unlearning. arXiv preprint WrXi:22405.10425 2024.
German Parisi, Ronald Kemker, Jose I. Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural networks, 113: 24-79,2019,
Vaishaal Shankar, Rebecca Roelofs, Horia Mania, Alex Fang, Benjamin Recht, and Ludwig Schmidt. Evaluating machine accuracy on ImageNet. In International Con- ference on Machine Learning, pp. 8634-8644. PMLR, 2020.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dzi- ugaite. Deep learning on a data diet: Finding important examples early in training. Advances in Neural Informa- tion Processing Systems, 34:20596-20607 2021.
Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, and Pang Wei Koh. Scaling retrieval-based language models with a trillion-token datastore. arXiv preprint arXiv:2407.12854 2024.
Tobias Plötz and Stefan Roth. Neural nearest neighbors networks. Advances in Neural information processing systems, 31, 2018.
12

---

Towards flexible perception with visual memory
Thalles Silva, Helio Pedrini, and Adín Ramírez Rivera. Learning from memory: Non-parametric memory aug- mented self-supervised learning of visual features. In Forty-first International Conference on Machine Learn- ing, 2024.
Yan Wang, Wei-Lun Chao, Kilian Q Weinberger, and Lau- rens Van Der Maaten. Simpleshot: Revisiting nearest- neighbor classification for few-shot learning. arXiv preprint arXiv:1911.04662 2019b.
Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. arXiv preprint arXiv: 110.3311 2014.
Chawin Sitawarin and David Wagner. On the robustness of deep k-nearest neighbors. In 2019 IEEE Security and Privacy Workshops (SPW), Pp. 1-7. IEEE, 2019.
Yuhuai Wu, Markus Norman Rabe, DeLesley Hutchins, and Christian Szegedy. Memorizing transformers. In International Conference on Learning Representations, 2021.
Sivic and Zisserman. Video Google: text retrieval ap- proach to object matching in videos. In Proceedings of the IEEE International Conference Computer Vision, pp. 1470-1477. IEEE, 2003.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non-paramen non-parametric instance discrimination. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pp. 3733 2018.
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35:19523- 19536, 2022.
Yang, Xiaoting Nan, and Bin Song. D2n4: A discrimi- native deep nearest neighbor neural network for few-shot space target recognition. IEEE Transactions on Geo- science and Remote Sensing, 58(5)36667-3676 2020.
Yiyou Sun, Yifei Ming. Xiaojin Zhu, and Yixuan Li. Out-of- distribution detection with deep nearest neighbors. In In- ternational Conference on Machine Learning, pp. 20827- 20840. PMLR, 2022.
Sangdoo Yun, Seong Joon Oh, Byeongho Heo, Dongyoon Han, Junsuk Choe, and Sanghyuk Chun. Re-labeling ImageNet: from single to multi-labels, from global to localized labels. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 2340 2350,2350,20 2021.
Eleni Triantafillou, Peter Kairouz, Fabian Pedregosa, Jamie Hayes, Meghdad Kurmanji, Kairan Zhao, Vincent Du- moulin, Julio Jacques Junior, Ioannis Mitliagkas, Jun Wan, et al. Are we making progress in unlearning? find- ings from the first neurips unlearning competition. arXiv preprint ar arXiv:2406.0900
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12104-12113, 2022.
2024.
Alexey Tsymbal. The problem of concept drift: definitions and related work. Computer Science Department, Trinity College Dublin, 106(2):58, 2004.
Haibo Zhang, Toru Nakamura, Takamasa Isohara, and Kouichi Sakurai. A review on machine unlearning. SN Computer Science, 4(4):337, 2023.
Matthew A Turk and Alex P Pentland. Face recognition us- ing eigenfaces. In Proceedings of the IEEE Computer So- ciety Conference on Computer Vision and Pattern Recog- nition, pp. 586-587, IEEE Computer Society, 1991.
Jiaxin Zhuang, Jiabin Cai, Ruixuan Wang, Jianguo Zhang, and Wei-Shi Zheng. Deep kNN for medical image clas- sification. In Medical Image Computing and Computer Assisted Intervention-MICCAA 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part 1 23, pp. 127-136. Springer, 2020.
Enayat Ullah, Tung Mai, Anup Rao, Ryan A Rossi, and Raman Arora. Machine unlearning via algorithmic stabil- ity. In Conference on Learning Theory, pp. 4126-4142. PMLR, 2021.
Eric Wallace, Shi Feng, and Jordan Boyd-Graber. Inter- preting neural networks with nearest neighbors. EMNLP 2018, pp. 136, 2018.
Haohan Wang, songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penaliz- ing local predictive power. In Advances in Neural Infor- mation Processing Systems, pp. 17506-10518, 2019a.
13

---

Towards flexible perception with visual memory
Appendix
We here provide the following supplemental information:
Appendix A Aggregation method comparison ImageNet-1K
Appendix B Aggregation method comparison on iNaturalist
Appendix C Hyperparameter sensitivity analysis
Appendix D Robustness towards label corruption
Appendix E Hit rate analysis as an upper bound on aggregation accuracy
Appendix F Scaling law details
Appendix G OOD analysis for NINCO dataset
Appendix H Reliability of retreived neighbors
Appendix Memory pruning details
Appendix Linear probe details
Appendix K Latency and storage
Appendix I. Human experiment to predict model behavior with memory system
Appendix Calibration analysis
Appendix N ImageNet-A error analysis
Appendix O Compositionality analysis
Appendix P Connection to bias removal and shortcut learning
14

---

Towards flexible perception with visual memory
A. Aggregation method comparison (ImageNet-IK)
Table Benchmarking different aggregation variants at different & thresholds, DinoV2 VIT-L14.
Aggregation
@10
@20
@30
@40
@50
@66
@ 70
@80
@90
100
Plurality Voting
83.2
82.9
82.6
82.4
82.1
82.0
81.8
81.6
81.5
81.4
Distance Votiing
83.3
83.0
82.7
82.4
82.2
82.1
81.9
81.7
81.6
81.5
Softmax Voting
83.5
83.5
83.4
83.3
83.2
83.1
83.1
83.0
82.9
82.9
Rank Voting
83.5
83.6
83.6
83.5
83.5
83.4
83.3
83.3
83.3
83.3
Table Benchmarking different aggregation variants at different & thresholds, DinoV2 ViT-B14.
Aggregation
@10 @0
@ 20
@30
@40
@50
@ 60
@ 70
@80
@90
@ 100
Plurality Voting
81.8
81.4
81.1
80.9
80.7
80.4
80.2
80.0
79.8
79.6
Distance Voting
81.9
81.5
81.2
81.0
80.8
80.5
80.3
80.0
79.9
79.7
Softmax Voting
82.0
82.0
81.9
81.8
81.7
81.7
81.6
81.5
81.3
81.3
Rank Voting
82.1
82.2
82.1
82.0
82.0
82.0
81.9
81.9
81.9
81.9
Table 6: Benchmarking different aggregation variants at different & thresholds, D2noV2 V4T-S14.
Aggregation
@ 10
@20
@ 30
@40
@ 50
@60 60
@ 70
@ 80
@90
@ 100
Plurality Voting
78.6
78.2
77.8
77.4
77.1
76.8
76.5
76.3
76.1
75.9
Distance Voting
78.8
78.4
77.9
77.5
77.2
76.9
76.6
76.4
76.2
76.0
Sofumax Voting
78.9
78.9
78.7
78.6
78.5
78.3
78.1
78.0
77.9
77.7
Rank Voting
78.9
79.1
79.0
78.9
78.9
78.9
78.9
78.8
78.8
78.8
Table 7: Benchmarking different aggregation variants at different & thresholds, CLIP VIT-L14.
Aggregation
@
10
@ @0
@30
@40
@50
60
@70
80
@90
@ 100
Plurality Voting
79.0
78.7
78.3
78.0
77.8
77.6
77.4
77.4
77.2
77.0
Distance Voting
79.2
78.9
78.5
78.2
78.0
77.8
77.6
77.5
77.3
77.1
Softmax Voting
79.3
79.3
79.1
78.9
78.8
78.7
78.5
78.5
78.4
78.2
Rank MankVoting
79.3
79.6
79.7
79.7
79.7
79.7
79.7
79.7
79.7
79.7
Table 8: Benchmarking different aggregation variants at different thresholds, CLIP ViT-B16.
Aggregation
@10
@20
@ 30
@40
@50
@60
@70
80
@90
100
Plurality Voting
72.8
72.6
72.3
72.0
71.7
71.4
71.2
70.9
70.8
70.5
Distance Voting
73.1
72.9
72.6
72.3
71.9
71.6
71.4
71.1
70.9
70.6
Softmax Voting
73.3
73.3
73.1
72.9
72.7
72.5
72.3
72.1
71.9
71.7
Rank Voting
73.0
73.7
73.8
73.8
73.8
73.8
73.7
73.7
73.7
73.7
15

---

Towards flexible perception with visual memory
82.5
79.5
79.0
81.5
(96
(96
78.0
81.0
77.5
80.5
Top-1
Top-1
76.5
79.5
PluralityVoding
PluralityVating
DistanceVoting
76.0
DistanceVating
79.0
SoftmaxVeting
SoftmaxVeting
RankVoting
RankVoting
78.5
75.0
0
20
40
60
80
100
0
20
40
60
80
100
Number of neighbors (k)
Number neighbors (k)
(a) D2 ViT-B14
(b) D2 ViT-S14
80
Top-1 (96
79-
Top-1 accuracy (96
78
70
PluraliyVoting
PluralityVoting
DistanceVoling
DistanceVoting
SoftmaxVoting
SoftmaxVating
RankVating
RankVoting
20
40
60
80
100
20
40
60
80
100
Number of neighbors (k)
Number neighbors (k)
(c) CLIP ViT-L14
(d) CLIP V1T-B16
Figure 7: Aggregation method comparison on the ImageNet- 1K validation set (same as Figure 2a but for other models).
16

---

Towards flexible perception with visual memory
Table 9: Benchmarking different aggregation variants on ImageNet-IK,
Model
Aggegation
IN-val acc (%)
CLIP VIT-LI4
CLIP paper (zero-shot)
75.3
CLIP VIT-L14
aggregation
76.0
CLIP VIT-LI4
Plurality Voting
79.2
CLIP VIT-L14
Distance Voting
79.4
CLIP VIT-L14
Softmax Voting
79.6
CLIP ViT-L14
Rank Voting
79.9
D2noV2 ViT-L14
DinoV2 paper (kNN Softmax)
83.5
DinoV2 ViT-L14
aggregation
81.1
DinoV2 ViT-L14
Plurality Voting
83.2
DinoV2 ViT-L14
Distance Voting
83.3
DinoV2 ViT-L14
Softmax Voting
83.6
DinoV2 ViT-L14
Rank Voting
83.6
B. Aggregation method comparison (iNaturalist)
60
55
Top-1 accuracy (96)
(96)
58
40
54
35
DinoV2 VIT-L/1
Pluraliy/Voling
DinoV2 VIT-B/14
DistanceVoting
Dino22 VIT-5/14
50
SoftmaxVeting
VIT-M1
RankVoting
VITB/16
48
40
60
80
100
20
40
60
80
100
Number of neighbors (k)
Number neighbors (k)
(a) Comparing aggregation methods, DinoV2 ViT-L14
(b) Rank (across models)
Figure 8: Aggregating information across retrieved memory samples on iNaturalist. Same as Figure 2 but for iNatu- ralist instead of ImageNet. (left) Existing aggregation methods (Plurality Voting, Distance DistanceVoting and Softmax Voting) are overconfident in distant neighbors, resulting in the paradox of decaying iNaturalist accuracy with more information. (right) This is not the case for Rank Voting which shows strong and stable performance across models and choices of k.
17

---

Towards flexible perception with visual memory
C. Hyperparameter sensitivity analysis
re/2 WilUU
Diviz VIFUE4
Tops
ne02 VIT844
MIEA
MIT1/14
24.
CLA 1IT822
PVITA/S
25
75
22.5
12.0
27.5
20.0
16
à
0.
hyperparometer
Hyperparameter
Hyperparameter
(a) Rankaanoooi Voting (a)
(b) Distance Voting (5)
(c) Softmax Voting (7)
Figure 9: Sensitivity to hyperparameters for different aggregation methods. Apart from Plurality all aggregation methods described in Section 2.2 have hyperparameter (a for RankVoting, T for Softmax Voting). For each model and method, we here plot the maximum performance when aggregating using a certain method, sweeping over the number of neighbors from 100, as function the hyperparameter. This analysis performed understand how sensitive the respective method is to the choice of the hyperparameter. Note that the range different since for instance the temperature parameter in Softmax Voting ranges from [0, 1] while Rank Voting for We therefore evaluate broad range for each method and find that all methods have regime in which they relatively stable irrespective of the hyperparameter choice. Since Distance Voting as implemented by Khandelwal et al. (2019) does not have a hyperparameter, we added temperature-style parameter & for the purpose of this comparison by setting w, = exp dist(E, =
is undefined (division by zero).
D. Robustness towards label corruption
Top-1 accuracy (%)
60
40
20
40 of corrupted labels
100
Figure 10: Robustness towards label corruption. How robust a visual memory towards corrupted labels in the memory bank? This plot shows top-1 Rank Voting accuracy on ImageNet validation set as a function how many labels the memory (containing ImageNet-1K training set features via D2 ViT-L/14) are corrupted, i.e., assigned random class. Intriguingly, performance stays almost unchanged all the way about 60% (!) corrupted (random) labels in the database.
18

---

Towards flexible perception with visual memory
E. Hit rate analysis
Hit rate (% true label among neighbors)
100
95
90
85
80
DinoV2 ViT-L/14
75
DinoV2 ViT-B/14
DinoV2 ViT-S/14
70
CLIP VIT-L/14
CLIP ViT-B/16
65
0
20
40
60
80
100
Neighbor index
Figure 11: Hit rate. This plot shows the probability of the true label being contained in list of labels of the first retrieved neighbors on ImageNet-IK, for five different models and & E [1, 100]. With 100 neighbors, the hit rate approaches 98% for the best model. Conceptually, this a very high upper bound on the performance that can be achieved by a given featurizer via nearest neighbor retrieval.
F. Scaling law
As mentioned in Section 3.3, we found that logarithmic form fits the data well between (memoryy size) and lerro rate). Specifically, found the following functional forms for DinoV2 VIT S14 and DinoV2 ViT L14 respectively via np polyfit (x, y, dim=1):
V2 VIT L14:
-0.9434 log1o(<) 2.0704
D2noV2 VIT S14:
-1.0942 log10(x) + 2.3187
where x = llog (memory-sizz) and y = (errorrar),,,,,), where memory-size E [10*, 10 and error-rate in [0, 100).
19

---

Towards flexible perception with visual memory
G. OOD analysis for NINCO dataset
Mean distance to first 100 neightors
0.8
Median distance to first 100 neighbors
0.8
0.7
0.7
0.6
0.5
0.4
0.3
0.3
0.1
0.1
0.0
in-distribution (ImageNet)
OOD (NINCO)
In-distribution (ImageNet)
(NINCO)
(a) Mean
(b) Median
Distance comparison: the NINCO OOD samples are indeed out-of-distributiin for the model. In Sec- tion 3.1, we described that we can simply plug new out-of-distribution classes into memory and still perform well on both existing data as well the new classes. This boxplot confirms that the added samples from the NINCO dataset (Bitterwolf al., 2023) are indeed out-o--distribution for DinoV2 ViT-L14: The mean (left) and median (right) distances from query the first 100 neighbors are substantially lower for ImageNet validation images than for OOD samples from NINCO.
Figure 12:
Figure 12 confirms that there is distribution difference between in-distribution data (ImageNet- 1K) and OOD data (NINCO). That said, while distribution shift exists, it is possible that individual NINCO samples were part of the training set for DinoV2. Test-set contamination is generally concern when working with models trained on large-scale datasets, since test samples may occur as exact, semantic or near-duplicates in large training datasets (e.g. Abbas et al., 2023b). For instance, NINCO contains samples from Food-101 (Bossard et al., 2014) which are also part of 1VD--42M dataset used to train DinoV2. That said, the NINCO samples belong classes which are definitely not part of the ImageNet-train set which serves memory bank for our experiments, as ensured by the NINCO dataset collection process (Bitterwolf al., 2023).
H. Reliability of retrieved memory samples
80
DinoV2 VIT-L/4
DinoV2 B/T4/14
50-
DinoV2 VIT-S/14
Neighbor accuracy (%)
Neighbor accuracy (9))
VIT-114
70
VIT-6/16
40-
65
30
55
DinaV2 VIT-L/14
20
V/T4/14
50
DinoV2 ViT-S/14
10
CLIP VIT-L/14
CLIP VIT-8/16
Neighbor index
Neighbor index
(a) ImageNet
(b) iNaturalist
Figure 13: Reliability of retrieved memory samples. This plot visualizes ImageNet (left) and iNaturalist (right) top-1 validation accuracy of single retrieved neighbor depending on the index of the neighbor (index O: nearest neighbor). In both datasets and across models, the decrease in accuracy with increasing neighbor index follows smooth trajectories and be approximated by two-parameter logarithmic fit (black lines). Figure 13a is the same Figure 1.
20

---

Towards flexible perception with visual memory
I. Memory pruning
For memory pruning from Section 3.6, we implemented two pruning methods: removing unreliable neighbors from memory entirely ("hard memory pruning"), and reducing their weight ("soft memory pruning"). We report results on the ImageNet validation set with a (potentially pruned) ImageNet-train set memory. For hard pruning, we excluded images from memory wrong decision at least 128 times (this meant excluding 26,257 images for D2 ViT-L14), based on
that contributed querying the ImageNet-train set against memory consisting of the ImageNet-train set and querying 100 neighbors for each sample. In order to obtain a fair comparison, instead of reporting accuracies for an arbitrary choice of & (the number of neighbors) we instead evaluate accuracy for each k in [1, 100] and report the maximum accuracy obtained in Table 3. This ensures that differences in observed accuracy can indeed be attributed to memory pruning, as opposed to a choice of k. For soft pruning, instead of excluding unreliable neighbors entirely as in hard pruning, the neighbor weights (1.0 for Plurality Voting, or rank-based weight in case of Rank Voting) are instead multiplied by a reliability factor y with y ..v where is the number of times the image contributed to wrong decision on the ImageNet-traie set, 1 to avoid division by zero, and d 1.75. This results, for instance, in 7 0.88 for images that only contribute to single wrong decision; in y = 0.16 for images that contribute to ten wrong decisions, and in 7 0.02 for images that contribute to 100 wrong decisions on the training set. Images that never contributed to any wrong decision are assigned = 1.0, i.e. their default weight remains unchanged.
J. Linear probe details
For the linear probe results reported in the paper, directly used the results that were reported in the D2 and CLIP papers. For V2, the authors froze the model backbone and trained the linear layers for 12500 iterations using SGD, Instead of training single time, they performed a full grid search sweep over three settings (output layers in 1, 4: pooling token concatenation in yes, no, and 13 different learning rates), resulting in 52 linear probes. Then, the authors evaluated the ImageNet validation accuracy for all of those 52 probes and only reported the highest one, as described in Appendix B.3 of the D2 paper. Some may call this test set tuning or double dipping, the Dino V2 paper describes as "common practice" (Oquab et al., 2023, p. 31). CLIP linear probe results are based on a logistic regression classifier learned using scikit-learn's L-BFGS implementation, and hyperparameter sweeps are performed held-out set not used for evaluation, according to Radford et al. (2021).
K. Latency and storage
Latency. Nearest neighbor retrieval, fortunately, does not need to reinvent the wheel but can, instead, build on top of highly optimized workloads and libraries such as the ScaNN library (Guo al., 2020). The ScaNN github README shows latency comparison; with the requirement of perfect recall a million-size memory can handle roughly 500-600 queries per second. It may be worth mentioning that searching large database can be done on CPUs and can be heavily parallelized.
Model
IN-train features (GB)
IN-val features (MB)
D2 VIT-L/14
4.9
197
D2 VIT-B/14
3.7
148
DinoV2 ViT-S/14
1.9
75
CLIP ViT-L/14
3.7
148
CLIP VIT-B/16
2.5
100
Table 10: Storage requirements for ImageNet features. Storing features in a memory database requires only about 1-3% of the space that needed to store the dataset (154.6 GB for ImageNet-train, 6.0 GB for ImageNet--alidaaton))
Storage. In addition to latency, storage is another very practical consideration: How much does it take to store features for a large database? To put things into perspective, the ImageNet training dataset requires 154.6 GB of storage, and the ImageNet validation dataset requires 6.0 GB of storage. In comparison, as shown in Appendix K, storing D2no or CLIP features for the entire ImageNet training dataset only requires between 1.9 and 4.9 GB of storage space. Thus compared to storing the training dataset, the model features account for only 1 1-3% of this size. This means that after constructing the memory, one may decide to keep the dataset which adds 1-3% of storage, or one may decide to delete the dataset only
21

---

Towards flexible perception with visual memory
keeping the features which saves 97-99% of storage (compared the dataset storage requirement). The ratio of features requiring 1-3% of the dataset size doesn't t change with dataset scale since it only depends the embedding model, thus this ratio would hold for very small datasets just it would for billion-scale dataset.
L. Human experiment to predict model behavior with memory system
conducted a small human experiment to quantify how much, all, a memory-basee system improves interpretability as operationalized by helping humans predict model behavior (as opposed a standard black-box model). Figure 14 outlines the experimental setup.
Condition A: model without neighbors
"Prddict how an Al model classifies the image on the left."
bow
suit
unicycle
bicycle
Condition B: model with neighbors
"Predict how an Al model classifies image the left."
suitt
unicycle
bicycle
The model finds these images similar to image on left.
Falsifiable prediction:
Human acouracy in condition is statistically significantly higher than human accuracy in condition A. that's not the case, we cannot claim that a visual memory system "more interpretable" than a standard black-box model.
Figure 14: Human experiment setup. Conditions A and were presented on computer screen in separate trials.
Given 4 label choices (guessing accuracy 25%), human accuracy is 56% in the case of black-box predictions (no neighbor information). With access to four nearest neighbor images from our memory-based system (just the neighbor images but not their labels), human accuracy is at 83%. This represents an absolute improvement of +27% and a relative improvement of +67% in human prediction accuracy, providing strong, falsifiable evidence in favor of the statement that a memory-based model is more interpretable.
Experimental details:
The accuracy difference is statistically significant (p 0.001).
Featurizer Dino V2 ViT-L14 (i.e. the best performing model).
Dataset: randomly selected ImageNet-A test images
Nearest neighbors for condition B: from ImageNet-train.
4 label choices per trial including ground truth label, model-predicted label (if different), and the remaining 2-3 labels were plausible alternatives based top CLIP predictions for the test image. Label order randomized.
22

---

Towards flexible perception with visual memory
M. Calibration analysis
100
cinov2_ytt14
100
dnoo2 vill1
80
Pluralty voting accuracy (8))
80
60-
Accuracy
0
20
40
60
80
100
0.0
0.2
0.4
0.6
0.8
1.0
Count plurality
Confidence
(b) DinoV2 ViT-L14, KNN classification
(a) Dino V2 ViT-L14, linear classification
dinovz_yee iinov_yitbl44 vitb14
100
Plurality voting accuracy (56
Accuracy (%)
60
60-
40
40-
20
40
60
100
0.2
0.4
0.6
0.8
1.0
Count plurality class
Confidence
(d) DinoV2 VIT-B14, KNN classification
(c) D2 VIT-B14, linear classification
100
cinov2_vitsh v
100
dnow2_yits14
80
Plurality voting securacy (96)
Accuracy
20-
0
0.0
0.2
0.6
0.8
1.0
20
Count plurality dlass
40
80
80
100
Confidence
(f) DinoV2 ViT-S14, kNN classification
(c) Di VIT-S14, classification
Figure 15: How well are predictions calibrated? Left column: Accuracy vs. confidence from Softmax of linear classifier for three DinoV2 variants. Right column: Accuracy count of plurality class among first 100 neighbors for the same three Dino variants. A D2no kNN classifier is well calibrated, is the Dino V2 softmax.
23

---

Towards flexible perception with visual memory
N. ImageNet-A error analysis
As shown Figure 6, many "errors" on ImageNet-A appear to be perfectly reasonable predictions that are caused by dataset label issues as opposed to model mistakes. More randomly selected ImageNet-A samples, along with nearest neighbors, are shown Figure 16. To quantify the issue, we performed human experiment on randomly selected subset of ImageNet-A images (N=100) where the dataset label and the prediction from DinoV2 ViT-L14 with JFT memory disagree. We presented the image alongside the original ImageNet-A label and our model-predicted label to three human observers, asking them to identify which of the labels best describes the image (of course, without telling them which of the labels is the dataset label), The result was that in 39.3% (!) of cases (std: +1.25%), the Dino label was assessed as being better/more suitable than the original dataset label-ie,, ie., roughly out of 5 model "errors" are in fact dataset label errors, quantifying the ImageNet-A label quality issue we alluded in Figure This percentage can used to estimate how correcting problematic labels influences performance. Instead of the original model's 61.1% accuracy on ImageNet-A, due label errors the 'corrected" accuracy is instead 76.4% (a delta of +153% in absolute terms +25.0% in relative terms).
query (fox squirrel)
neighbor
neighbor
neighbor
neighbor
neighbor 5
Neighbor classes:
paddle
boathcuse
spesdboat
canoe
canoe
query (sandal)
neighbor
neighbar
neighbar 3
neighbor
neighbor 5
Neighbor classes:
samoyed
samoyed
samoyed
samoyed
samoyed
query (rhinoceros beetle)
neighbor
neighbor 2
neighbor 3
neighbor 4
neighbor
Neighbor classes:
ant
ant
ant
ant
samoyed
query (banana)
neighbor 1
neighbor 2
neighbar 3
neighbor 4
neighbor
Neighbor classes:
tripod
tripod
tripod
tripod
tripod
Figure 16: Interpretable decision-making. A retrieval-basee visual memory enables a clear visual understanding of why model makes certain prediction. Here, we show four randomly selected misclassified query images from ImageNet-A (Hendryeks et al., 2021) along with five nearest neighbors from DinoV2 ViT-L14 using the ImageNet-1K K training set as visual memory. All labels are from the respective datasets (ImageNet-A for query and ImageNet-train for neighbors). While all neighbors visually look reasonable, not all labels do.
24

---

Towards flexible perception with visual memory
O. Compositionality analysis
flexible visual memory also provides a path to analyze representations of various models, particularly, how different models represent multiple concepts in an image. We study this for ImageNet-train visual memory of Dino'2 ViT-L14 and CLIP ViT-L14. We use manually selected query images from outside the ImageNet dataset that have multiple objects from the ImageNet labels. We query the visual memory for nearest neighbors of the query image. Subsequently, we obtain the residual image by subtracting the features of the nearest neighbor from the features of the query image. We, then, obtain the nearest neighbors for the residual image from the visual memory. We plot the results in Figure 17 which shows that D2 ViT-L14 and CLIP ViT-L14 represent concepts in their features in a different manner. The nearest neighbors for Di are mostly images with a single concept (or object) from the query image. The residual image, subsequently, leads to nearest neighbors dominated by another single object the query image. contrast, CLIP often results in nearest neighbors that are generally a blend of concepts from the query image. These qualitative explorations are simple demonstrations of the advantages of interpretable Docision-making process provided by flexible visual memory.
query
neighbor
neighbor
neighbor
residual
residual
residual 3
Dino- Pino-ViTzL. 1A
CLIP- -
Figure 17: Compositionality of representations. The first column indicates query image; the next three columns are the three nearest neighbors from the training set. The last three columns are the residual images, obtained by subtracting the features of the nearest neighbor (2nd column from the left) from the features of the query image (1st column from the left). The nearest neighbors for DinoV2 are mostly images with a single concept (or object) from the query image. The residual image, subsequently, leads to nearest neighbors dominated by another single object in the query image. In contrast, CLIP often finds neighbors that are blend of concepts from the query image.
P. Connection to bias removal and shortcut lcarning
This section contains brief discussion connection between our method and bias iinoval-since the additional flexibility that our memory approach brings (compared to a standard, inflexible classifier), it is open question whether this could enable better bias removal.
If the image encoder exploited shortcut during training, this will influence image similarity and thus nearest neighbor selection. There are cases where bias removal is possible, and cases where it is impossible:
Removal impossible: the encoder is biased towards textures, a test image of "cat shape + elephant texture" (cf. Geirhos al., 2019, Figure 1) would pull up elephant nearest neighbors, and removing all elephants from memory
25

---

Towards flexible perception with visual memory
would come unreasonably high cost (not being able to identify elephants anymore). Here, encoder-level debiasing is necessary.
Removal possible: only part of the memory is biased, memory-level debiasing is feasible. "fingers" are shortcut predictors for "fish" (due to a dataset bias from proud fishermen holding their catch into the camera, cf. Brendel & Bethge, 2019, Figure 3), then this bias could indeed be rectified by removing the biased "fish+finger" subset from memory. Afterwards, images with "fingers" would longer lead to "fish" nearest neighbors, demonstrating successful bias removal.
26


Reviews:
[{"review_id": "lK2DExWzkZ", "reviewer": "Reviewer_Em8N", "summary": "The authors observe that it is hard to edit knowledge acquired by deep models during training, because this knowledge is encoded in a vast number of interconnected weights. To address this issue, they suggest keeping a pre-trained model frozen, and enhancing it with a visual memory and a simple KNN algorithm to make classification decisions; the visual memory essentially corresponds to a database of feature vectors from the frozen pre-trained model. Given such a visual memory, the knowledge used by the model for classification decisions can be edited as easily as entries can be added or removed from a database. Also, the authors explore different ways to aggregate labels from the k nearest neighbors of a query, proposing a ranking-based weighing strategy.  \n\nThe authors acknowledge that similar systems already exist in the literature, however, they conduct multiple new experiments to highlight a number of capabilities, aiming to show the relevance of such a system in modern applications, and inspire further research in this direction. In the experimental evaluation of the method, the authors use ImageNet-1K (IN-1K) as the default visual memory, and show that visual memory can handle out-of-distribution (OOD) samples (they use NINCO dataset) without harming performance on existing classes, while memory can efficiently increase to billion-scale data. They also show that the influence of memory entries can be controlled through hard or soft pruning, which corresponds to weighing memory entries based on offline estimation of their impact in classification decisions. In addition, the authors show that classification decisions can be interpreted by inspecting the k nearest neighbors in the visual memory, along with their aggregation weights. Finally, the authors provide a number of additional experiments in the Appendix, where they further explore the behavior of their system, e.g., hierarchical classification, and prediction calibration.\n\n## update after rebuttal\nThe authors addressed most of my review comments, so I increased my score from 2 to 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer Em8N,\n\nThank you very much for your detailed review. We’re glad to hear you appreciated the **“well-designed experiments”**, **“very well organized presentation / clarity”** and **“very well written manuscript”**.\n\n*Abstract: “nearly impossible” is pretty strong*  \nNoted - we’ll change this to “editing the knowledge in a network is hard”.\n\n*Generalization to other tasks like segmentation / detection:*  \nIt’s possible to extend the approach to other tasks. One can pool features into multiple embedding clusters instead of a single cluster using classical or learned clustering methods. As a proof of concept, we tested object segmentation based on a visual memory of DinoV2 features; visualized here: https://ibb.co/92dM0B2. This puts features from a single image into memory (a car in the example) based on 8 feature clusters and uses these to identify similar features in the second (test) image, thereby creating a segmentation mask. Such multi-vector representations of images (which can be expanded to image + text) are a natural extension to our work, enabling tasks such as object retrieval or detection from images containing multiple objects, or provide coarse patch-level semantic segmentation. Finer-grained segmentation masks can be obtained with further training of the pooling methods. We hope this provides an intuition for how other tasks can be approached, and we will add the proof of concept example to the appendix. If other tasks are tackled, our main argument about the increased flexibility of a memory-based approach (and the benefits / capabilities enabled by this flexibility) directly translate to those tasks as well (& could be plugged into canonical architectures like Fast-RCNN).\n\n*Baselines, e.g. Table 1:*  \nWe fully agree on the importance of baselines. Due to space limitations, some comparisons were moved to the appendix, like Table 9 (RankVoting 79.9%, CLIP zero-shot 75.3%). We’d be happy to mention and display baselines more prominently in the main paper. For Table 1, we did not include a comparison to linear probes since the goal of the table is to show a lifelong learning evaluation, i.e. understand which performance can be reached on OOD data without re-training anything. If NINCO classes are evaluated with a DinoV2 model and its default linear classifier, the performance would be 0.00% because the classifier is static and cannot transfer what it has learned without further fine-tuning and change of architecture/layer.\n\n*Novelty:*  \nAgreed, while we provide technical improvements like RankVoting, we build on well-established methods with a long history in ML and seek to be very transparent about this throughout. Instead, our main focus is a broad evaluation of flexible capabilities as mentioned in your review. We believe there is community interest in seeing solid capability evaluations (as e.g. evidenced by xHyy describing it as “one of the most exciting papers I’ve seen this year in this space”).\n\n*How is the reliability factor selected?*  \nIt’s directly related to the number of times the training image contributed to a wrong decision on ImageNet-train; see https://ibb.co/2320r1Ld.\n\n*What’s the dotted line in Fig 11?*  \nIt highlights a chance accuracy baseline (no aggregation, just constantly predicting a single class). For balanced datasets, baseline guessing accuracy can be calculated as 1 / num_classes. Since iNaturalist is unbalanced, this could be misleading. E.g. if a dataset has just two classes, but one of them accounts for 70% of samples, then constantly predicting this class would lead to 70% accuracy. Thus a commonly used “strong guessing” baseline for unbalanced datasets is to always predict the largest class (with the most samples over the entire dataset), without encoder/aggregation. We’ll update the description to make this clear. We could add DinoV2 linear probing as well, though it might struggle to train properly given just a handful of training samples. \n\n*Section G aggregation method?*  \nSince we’re adding neighbors to memory starting from 0 exemplars, no aggregation is used here (k=1); we’ll make sure to mention this.\n\n*Algorithm 1:*  \nWe made a mistake here - Alg. 1 corresponds to an algorithm that we tried, but it didn’t work better than the much simpler kNN-based classifier with k=1. We should have removed the algorithm, apologies for the oversight. Fig. 11 indeed already corresponds to the simple kNN classification approach that you suggested we try instead.\n\n*If new samples are added, should soft pruning weights be calculated again?*  \nSince those new samples don’t have reliability weights, their reliability would indeed need to be estimated, unless a “default reliability weight” (e.g. mean reliability of existing samples) is used as a proxy. If the new samples are IID, existing weights for existing samples wouldn’t change systematically, thus those can be kept without recalculating.\n\n*Other comments / suggestions:*  \nExcellent points, thank you!", "claims_and_evidence": "- The authors motivate their research by mentioning in the Abstract, “Training a neural network is a monolithic endeavor, akin to carving knowledge into stone: once the process is completed, editing the knowledge in a network is nearly impossible, since all information is distributed across the network’s weights.” I think there is truth in this statement, but at the same time the phrase “nearly impossible” is pretty strong, because there are plenty of PEFT methods (e.g., LoRA) to address domain shifts and diverse downstream tasks, and multiple alignment strategies (e.g., DPO) of variable complexity.\n- In Section 3, the authors essentially dedicate one subsection for each of their claims, which makes for a very well organized presentation. In general, the experiments are well designed, and offer evidence for the corresponding claims. However, I think there is an issue with novelty. The idea of visual memory combined with a KNN classifier exists in many works that the authors cite, e.g., [1], so, I am not denying that there is value in additional elaborate experiments, but it doesn’t seem a major contribution, especially since KNN classifiers usually underperform compared to linear probes [2], and don’t generalize to diverse downstream tasks, e.g., segmentation and object detection. At the same time, the authors introduce a novel idea with Gemini re-ranking, but it is not really explored in the paper, even if it gives promising results in Table 2.\n\n[1] Nakata, Kengo, et al. \"Revisiting a knn-based image classification system with high-capacity storage.\" *European conference on computer vision*. Cham: Springer Nature Switzerland, 2022.\n\n[2] Oquab, Maxime, et al. \"Dinov2: Learning robust visual features without supervision.\" *arXiv preprint arXiv:2304.07193* (2023).", "methods_and_evaluation_criteria": "In most experiments, the authors evaluate different configurations of their own model. For example, in Table 1, they evaluate the relative performance of different aggregation methods, and the in- and out-of-distribution performance before and after expanding the visual memory. I think this is fine to demonstrate a capability, but to demonstrate its impact in the broader literature, I think It's important to have baselines, like a zero-shot classifier similar to the one from CLIP or a linear probe, especially since linear probes tend to perform better than KNN classifiers, as can be seen in Table 2, where the linear probe baseline outperforms the KNN classifiers without the Gemini re-ranking. Of course, a baseline like linear probe will come with the extra cost of training compute, but compute measurements can be included as well, since it is good to know the performance-compute trade-off between different relevant methods.", "theoretical_claims": "There aren't any proofs or theoretical claims.", "experimental_designs_or_analyses": "In general, the authors make a good effort to isolate the effect of different factors in order to reach stable conclusions. For example, in the pruning experiments (Section 3.5), as explained in Section J in the Appendix, they try to mitigate the effect of $k$, so they can attribute differences in the behavior to pruning.", "relation_to_broader_scientific_literature": "The authors cite a number of highly related works, e.g., ln 25-28, col 2, which indicate that the combination of a visual memory with a KNN classifier already exists in the literature. I think the main contribution is a set of new experiments to highlight the capabilities of the proposed system, and to show that it is relevant to modern applications.", "essential_references_not_discussed": "Nothing to add.", "other_strengths_and_weaknesses": "The manuscript is very well written, with clear Figures, Tables and captions. The authors sometimes have a playful tone, which I personally don’t mind, and I even find refreshing.", "other_comments_or_suggestions": "- ln 37, col 2: “ it has seven desirable capabilities”, Section 3 discusses 6 capabilities.\n- ln 94, col 2: In the definition of $D_{\\text{test}}$, I think $\\tilde{x}_n$ and $y_n$ should be a tuple $(\\tilde{x}_n, y_n)$.\n- In Section 2.1., the authors describe visual memory entries as feature maps, and in Section 2.2., as feature vectors (ln 149, col 1); I don’t think these terms should be interchanged, especially since the only distance metric used is cosine similarity, which requires feature vectors.\n- ln 150, col 1: Next to $y_{[2]}$ there is “((” instead of “)”.\n- ln 242, col 2: “test whether smaller models larger memory”, I think “with” is missing; similarly, in ln 273, col 1, something seems off with the phrase “increasing memory model size”.", "questions_for_authors": "When soft pruning is used, if new samples are added to the database, should pruning weights be calculated again?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4nV29T0KLb", "reviewer": "Reviewer_4C32", "summary": "The authors tackle the question whether splitting the ‘knowledge’ a neural model has to acquire into 1) a (small) set of learnt parameters (encoder) combined with 2) a non-parametric memory can prove superior to the classic “only-learnt” parameter approach – somewhat akin to what has been successfully pursued in the language domain. They demonstrate that approaching the image classification task via a similarity search over a large datastore can indeed yield a range of advantages, especially around flexible addition and removal of specific samples – as well as better attribution during decision making which improves interpretability.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 4C32,\n\nThank you for your helpful comments. We’re happy to hear you found our work **“well justified”**, appreciated the **“insightful experiments & valuable insights”** (even though some of them were admittedly a bit buried in the appendix), and described it as **“a timely analysis in the vision space”**.\n\n*Beyond classification, which other visual tasks might benefit from a datastore? Ways to leverage for segmentation, detection?*  \nIt’s possible to extend the approach to other tasks. One can pool features into multiple embedding clusters instead of a single cluster using classical or learned clustering methods. As a proof of concept, we tested object segmentation based on a visual memory of DinoV2 features; visualized here: https://ibb.co/92dM0B2. This puts features from a single image into memory (a car in the example) based on 8 feature clusters and uses these to identify similar features in the second (test) image, thereby creating a segmentation mask. Such multi-vector representations of images (which can be expanded to image + text) are a natural extension to our work, enabling tasks such as object retrieval or detection from images containing multiple objects, or provide coarse patch-level semantic segmentation. Finer-grained segmentation masks can be obtained with further training of the pooling methods. We hope this provides an intuition for how other tasks can be approached, and we will add the proof of concept example to the appendix. If other tasks are tackled, our main argument about the increased flexibility of a memory-based approach (and the benefits / capabilities enabled by this flexibility) directly translate to those tasks as well (& could be plugged into canonical detection architectures like Fast-RCNN).\n\n*Appendix nicely complements the manuscript but provides new insights that would deserve more visibility – esp. Appendix G (& to some extent O & P):*  \nThanks for the comment, that’s great to know. Since ICML's camera ready version allows for an additional page, we’re able to move the iNaturalist experiment (app. G) to the main paper as well as reference & hightlight the other two sections more than we currently do.\n\n*Changing samples might require a rebuild of the index:*  \nAgreed, we will add this to the limitations section (page 8). Regarding the two search approaches described on page 3, adding/removing samples is trivial for approach #1 (GPU/TPU matmul e.g. on ImageNet-scale data; here one can add or drop a row from the num_images x num_features matrix) but for approach #2 (scalable nearest neighbor search index for JFT) this would indeed require adapting the index, though the amortized cost is low.\n\n*Related work: suggesting to also express what’s different.*  \nValid point, we’re happy to incorporate this.\n\n*Reliance on a pre-trained encoder:*  \nIndeed, current encoders are limited. Since our approach is modular, any encoder can be used: as soon as more general, open-world encoders are developed they can simply be plugged in; even encoders based on generative models could become a possibility (given that generative models are a lot more open-world, open-vocab). Thus our approach is orthogonal to encoder choice. That said, even with current encoders we see promising success based on adding novel classes (NINCO experiment in Section 3.1).\n\n*Innovation:*  \nWhile we provide technical improvements like RankVoting, we fully agree that we build on well-established methods with a long history in ML and seek to be very transparent about this throughout the paper. Instead, our focus is a broad evaluation of flexible capabilities (including attribution, flexible adding/removal, flexibly increasing granularity like on iNaturalist, …). We believe there is community interest in seeing solid evaluations.\n\n*Approximate retrieval:*  \nGreat question! Approximate retrieval increases retrieval speed at the cost of recall errors. Based on data from Appendix D, approximate retrieval would not significantly degrade results. Let’s assume approximate retrieval leads to different neighbors (compared to NNs from exact retrieval). In the best case, those neighbors are still from the same class; thus nothing changes. Worst case, they’re from a different class i.e. their label is now misleading. Based on Figure 9, we know that our approach can handle up to 60% label corruption (!) without degradation of performance; thus as long as k>>1 our approach is very robust to approximate retrieval. We’ll add this discussion.\n\n*Would an automatic ‘uncertainty’ threshold (e.g. based on distance) be useful to detect generalisation issues?*  \nBased on the analysis from Appendix H, OOD data does indeed lead to higher mean+median nearest neighbor distances; thus a distance threshold would work well. There are a few works on kNN-based outlier detection (e.g. https://proceedings.mlr.press/v162/sun22d/sun22d.pdf).\n\nKindly let us know if you have any further questions, and thanks again for the great suggestions.", "claims_and_evidence": "The major claims around flexibility of data addition & removal, as well as improved interpretability are justified and substantiated by evidence/insights throughout the experiments;   \n$\\textrightarrow$ Only slight criticism would be that depending on the kind of datastore used, adding and/or removing samples might require a rebuild of the index/tree/graph – a caveat which could be worth mentioning, although it is likely amortized when compared to any other approach that requires retraining.\n\nAll claims around simplicity combined with performance are certainly well justified and substantiated.\n\nRegarding the title: \"Perception\" might be a bit overclaiming, as the only task which is demonstrated here is 'image classification'; and it is debatable if this alone justifies 'perception'.", "methods_and_evaluation_criteria": "While the selection of methods as well as alternatives for ablation purposes is well chosen to foster simplicity, the evaluation is exclusively concentrated on image classification.   \nThis is a valid choice, but somewhat limits the insights that can be gained: the ability to classify images based on their nearest neighbours is pretty well known, and has been extensively used across many tasks (e.g. few-shot learning via prototypical networks), even going back to 'classical' computer vision problems based on SIFT and other (simple) features; so the insights how having access to this much larger datastore could impact other (more complex) applications like generation would have been desirable.", "theoretical_claims": "No specific theoretical claims present in this work.", "experimental_designs_or_analyses": "As previously mentioned, the limitation to image classification is somewhat understandable to provide more detail on ablations of individual components like datastore size and voting metric, but unfortunately also limits the insights gained.  \nGiven the previous use of kNN for classification in the literature (albeit with smaller sets of reference samples), it is not extremely surprising that this works well; especially since a powerful encoder like DinoV2 is used – which is explicitly trained to capture various attributes in images based on similarity, and known to provide good and expressive feature representations.\n\nPersonally, I found the appendix of the paper much more insightful:  \n$\\textrightarrow$ Appendix G demonstrates the dependencies between attributes of a related but novel class/species and other classes in the dataset – and how step-wise addition of other exemplars of the same species influences the classification across all levels of the taxonomic hierarchy.    \n$\\textrightarrow$ Appendix P shows a compositionality analyses, which might also spark new ideas for the use of nearest neighbours for more complex multi-object visual settings.\n\nThe ablations are, however, well chosen and provide sufficient insights into the crucial components of the approach.", "relation_to_broader_scientific_literature": "Relation to broader literature is established, both in the introduction as well as related works section – however, the authors could improve in actually expressing what is *different* in their own work, as the related works section is currently mainly listing other works w/o contrast to this manuscript’s proposed approach.", "essential_references_not_discussed": "None that come to mind, the related works section provides a top-level but sufficiently broad list of related areas & works;", "other_strengths_and_weaknesses": "**Strengths**:   \n*Originality & Significance:*  \n- Replacing the classic parameter-based neural memory through similarity-based search is an important area that has shown promise in the past as well as more recently in other fields like language; so this work provides a timely analysis in the vision space\n- Analyses across aggregation methods as well as influence of #neighbours provides helpful insights for future methods building on similar structures (e.g. consistency in performance of aggregation schemes demonstrated in Tables 4-8)\n- Fig 3. / Section 3.2 supports the findings obtained in the language domain, i.e. small model with larger memory can be competitive to bigger model\n\n*Clarity:*   \n- The paper is well written and easy to read and follow; many additional supporting analyses moved to the appendix, so the paper provides a good level of depth to easily follow\n\n--- \n**Weaknesses:**\n- One key weakness that, however, seems unavoidable is the reliance on a pre-trained encoder to compress the raw images for efficient similarity search; This always raises the question about the applicability in situations with large domain gaps and actual open-world settings where genuinely ‘novel’ designs and/or materials are encountered. \n- Analysis in terms of kNN is mainly focused on the ‘ideal’ recall setting; However, many other popular kNN retrieval algorithms perform approximate retrieval; $\\textrightarrow$ See questions\n- Although 'perception' is claimed in the title, the only task demonstrated in the paper is image classification.\n- Main difference to other 'classical' kNN-based methods is the use of a large datastore and powerful encoder -- both, however, are known to work well in other closely related areas like e.g. NLP; so the number of 'novel surprising' insights is quite limited given the 'classification-only' setup of the experiments", "other_comments_or_suggestions": "Comment: I personally really enjoyed the analysis on iNaturalist presented in Appendix G as it provides genuinely ‘new’ and interesting insights; as well as the compositionality analysis (App. P).", "questions_for_authors": "1. Many other datastore-based approaches use *approximate* kNN search; I’d like the authors to comment on whether they think their findings would roughly translate to approximate retrieval (e.g. at recall 0.85+), or whether this could severely compromise results;\n2. Figure 3 starts at 1K memory, showing the performance for 1 sample per class; This is highly dependent on the sample, so I’d like to know if the authors have tried to use one or a few prototypes instead; and if so, how this would affect the performance? \n3. Do the authors have any insights/hypotheses which other visual tasks might benefit most from a datastore, beyond image classification and generation? Could there be ways to leverage this e.g. for applications like segmentation, detection, and the like?\n\nMore on the ‘experience’ side: \n  \n4. Given that encoders trained on large-scale data in a self-supervised manner capture a vast variety of data characteristics, I’d like to know whether the authors encountered any degradation when moving to slightly more specialised datasets like iNaturalist – or potentially even more specialised ones like Medical Applications, Satellite Images, etc;   \n$\\textrightarrow$ Do the authors have an intuition how translatable the features still are, and whether an automatic ‘uncertainty’ threshold (e.g. based on distance or distribution of neighbours) would be useful to detect issues in generalisation?\n\n---\n*TLDR;* The paper does present a number of valuable insights (especially the rank-voting strategy and its associated behaviour), hence my rating – however, as previously mentioned throughout the other parts: I do perceive many of the findings as somewhat unsurprising given previous works and the related success of retrieval-based methods in the language domain, as well as the history in 'classical' computer vision (based on simpler features) -- making the 'novelty' part in terms of contributed insights rather limited;        \n$\\textrightarrow$ This is obviously a subjective experience, but: As mentioned previously, some quite interesting analyses and (more) surprising findings are placed/hidden in the appendix – and might deserve a bit more ‘spotlight’, or at least a reference/hint in the main paper.\n\n---\n---\n## Update post-rebuttal: \nMain questions have been addressed; Raising my score from 3 to 4", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aUj4tSEiRx", "reviewer": "Reviewer_xHyy", "summary": "This paper introduces a retrieval-based visual memory framework, challenging the traditional paradigm of deep learning models storing knowledge in static (\"stone\") weights. \nInstead, it separates representation (via pre-trained embeddings) from memory (through fast nearest-neighbor search), enabling a dynamically editable model. The proposed method offers a scalable and flexible solution to lifelong learning, unlearning, and dataset pruning. \nBy treating classification as a retrieval problem, the authors demonstrate near state-of-the-art performance on large-scale datasets (ImageNet, JFT) with benefits such as control on the decision-making. This approach is presented as a significant step toward rethinking how knowledge should be stored in deep learning, moving away from the limitations of static weight-based models.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer xHyy,\n\nThanks for your review, we’re humbled to hear you appreciated the **“extremely well done experiments”** and found the work **“highly relevant with enormous potential for future extensions”**, & **“one of the most exciting papers I’ve seen this year in this space”** that **“could become a foundational paper”**.\n\n*Falsifiable experiment on interpretability:*  \nWe fully agree that interpretability should go beyond visualization. First of all, we don’t mean to imply our system is “fully interpretable”, rather that it is “more interpretable” compared to a standard black-box model (and we will revise the writing to make this distinction clear). Secondly, we appreciate your point that interpretability-related claims and statements are best made through a falsifiable human experiment. We outlined a possible experiment at https://ibb.co/k2jnGH85; it’s aimed at quantifying how much, if at all, a memory-based system improves interpretability as operationalized by helping humans predict model behavior. Before running the experiment, we’re keen to hear your thoughts!\n\n*Shortcut learning / bias removal:*  \nWe’re happy to add a discussion on the connection to shortcut learning and bias removal. If the image encoder exploited a shortcut during training, this will influence image similarity and thus nearest neighbor selection. There are cases where bias removal is possible, and cases where it is impossible:\n- [removal impossible] If the encoder is biased towards textures, a test image of “cat shape + elephant texture” [cf. Geirhos et al. 2019, Figure 1] would pull up elephant nearest neighbors, and removing all elephants from memory would come at an unreasonably high cost (not being able to identify elephants anymore). Here, encoder-level debiasing is necessary.\n- [removal possible] If only a part of the memory is biased, memory-level debiasing is feasible. If “fingers” are shortcut predictors for “fish” (due to a dataset bias from proud fishermen holding their catch into the camera, cf. Brendel et al. 2019, Figure 3), then this bias could indeed be rectified by removing the biased “fish+finger” subset from memory. Afterwards, images with “fingers” would no longer lead to “fish” nearest neighbors, demonstrating successful bias removal.\n\n*DinoV2 / Could alternative losses, or even different model architectures, be designed to perform even better?**  \nWe agree that DiNO-V2 is a strong vision model, facilitating retrieval. Since our approach is modular, any encoder can be used in a plug-and-play fashion within our framework. Progress on visual representation learning (architectures, losses etc.) can generally be expected to improve retrieval. For example, text-to-image generative models have proven to learn useful representations for open-world, open-vocab tasks and they could become possible image encoders for retrieval-based tasks as well and better losses & architectures developed by the field will directly translate into better visual memory systems.\n\n*Adversarial perturbations:*  \nWe share your intuition. As long as adversarial attacks can fool featurizers, neighbor selection can be attacked through perturbations as well (e.g., making model think a dog image is instead an airplane, and thus retrieve airplane neighbors leading to misclassification). Neighbor selection is nondifferentiable, but black-box or surrogate-model based attacks (e.g. replacing neighbor selection with differentiable Gumbel-Softmax) will likely work.\n\n*Suggestion for discussion: should removal or unlearning occur at the individual data point level or at the concept level?*  \nConcept-level manipulation (adding/unlearning entire concepts) is a really exciting idea that we hadn’t considered so far! Since any similarity space can be used for neighbor selection, including ACE or SAE-based spaces, this is indeed possible. Depending on the use case, both data-based and concept-based manipulation can be desirable: if a single image is corrupted / has licensing issues then unlearning the individual image is the right choice; if a concept is biased then concept-level changes seem preferable. We’ll be happy to add a brief discussion on this exciting possibility.\n\n*Data point influence = number of times it's used in the decision?*  \nYes. In the extreme case, if a sample never shows up as a neighbor, its influence would be zero. Of course, influence isn’t always a good thing - it really depends on whether the sample contributes to (= influences) a correct or a wrong decision. Fortunately, in contrast to a traditional model, that’s very easy to keep track of in a memory model, and can thus be exploited for reliability-based weighting as in Table 3.\n\n*Could this score be used to select better quality/diversity on the dataset?*  \nLikely. For ImageNet-A, in ~40% of cases the DinoV2 label was assessed as being better/more suitable than the original dataset label, thus this could be used to identify label issues and thereby improve dataset quality.", "claims_and_evidence": "The retrieval-based approach enables truly flexible model updates, allowing knowledge to be added or removed without retraining. The authors demonstrate that new classes can be integrated, and unwanted data points can be unlearned by removing them from memory. The method also scales effectively, achieving strong ImageNet top-1 accuracy, with Gemini re-ranking further enhancing performance.\n\nHowever, I remain unconvinced by the claim that this approach inherently leads to interpretable decision-making. While retrieving nearest neighbors provides insight into what influenced a prediction, interpretability should go beyond visualization. I would like to see a falsifiable human experiment where a human can predict how a modification to the memory will affect the model’s behavior and verify whether the system responds as expected. This kind of simulatability (as discussed in works by [1,2,3]) would provide stronger evidence that the model is genuinely interpretable in a way that users can act upon.\n\nThat said, aside from this concern, I found that the empirical results convincingly support the paper’s core claims.\n\n[1] Finale Doshi-Velez and Been Kim. “Towards a rigorous science of interpretable machine learning”\n\n[2] Julien Colin, Thomas Fel, Rémi Cadène, and Thomas Serre. What i cannot predict, i do not understand: A human-centered\nevaluation framework for explainability methods.\n\n[3] Peter Hase and Mohit Bansal. “Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?”", "methods_and_evaluation_criteria": "Yes, except for the interpretability claim, the experiment section is extensive and extremely well done.", "theoretical_claims": "Yes, no issue.", "experimental_designs_or_analyses": "The experiments cover large-scale datasets (JFT, IN), analyses of memory size versus accuracy, ablations on retrieval and demonstrations of unlearning and dataset pruning.", "relation_to_broader_scientific_literature": "Excellent, I wish more paper would have such an excellent related work section.", "essential_references_not_discussed": "Nothing in my opinion.", "other_strengths_and_weaknesses": "**I should say that this is one of the most exciting papers I’ve seen this year in this space**. \nIt opens up an entirely new direction that allow flexible, \"interpretable?\" and more importantly controllable vision decision. The work is well-executed, highly relevant, and has enormous potential for future extensions (e.g., concept-based unlearning, fairness improvements, multimodal retrieval, see my questions).\n\nWith minor improvements, this could become a foundational paper in AI research.\n\nTo summarize, in the strenghts, I found that:\n- the method proposed scale efficiently to billion-scale datasets while maintaining strong performance, making it highly practical for real-world deployment.\n- controllable: decisions are based on explicit memory retrieval, allowing users to inspect and potentially modify model behavior, though further validation of interpretability is needed.\n- unlearning and dataset pruning made simple: removing knowledge is as simple as deleting data from memory, attacking a key challenge in unlearning and data curation.\n- strong generalization across domains: the model demonstrates impressive out-of-distribution robustness, suggesting retrieval-based architectures can better adapt to changing data distributions.\n\nHowever, even if i really like this work, I think that it could be improved. Here are, in my opinion, the weak points of the paper, which I will group into major problems (**M**) and minor problems (**m**).\nMajor Concerns (**M**):\n- **M.1**. Claim on Interpretability: see my previous point, but i think that the lack of formal user study if a major point if you claim interpreatbility. The model's explainability seems intuitive, but without user studies, it is uncertain whether humans can predict or act on model behavior effectively.\n- **M.2** No Discussion of shortcut learning and bias removal: I think the first extension or application i would think of would be on removing shortcuts. The paper does not address whether retrieval-based learning can still exploit dataset shortcuts (I would tend to think yes), nor whether it can actively remove biases.\n- **M.3** Dependence on DinoV2: DinoV2 is an exceptionally strong vision model, particularly effective for retrieval tasks due in part to the Koleo loss—which was specifically choosen by the authors of DinoV2 for this purpose (see: https://github.com/facebookresearch/dinov2/blob/main/dinov2/loss/koleo_loss.py). This raises the question of how much the results depend on the model's embeddings. Could alternative losses, or even different model architectures, be designed to perform even better?\n\nNow for the minor (**m**):\n- **m.1** Robustness to adversarial: do you think that the effect of adversarial perturbations on retrieval remains true ? I would tend to say yes but i am curious.\n- **m.2** A final remark, more of a suggestion for discussion: should removal or unlearning occur at the individual data point level or at the concept level? In other words, when seeking for interpretable decisions, should interpretability be considered at the data point level or at the conceptual level? Could we instead compute distances in an overcomplete concept space [1] and design a more fine-grained decision-making process within this space? This approach would mean modifying only part of a point’s embedding rather than entirely removing a data point. Additionally, _storing a sparse embedding of a point could be significantly more efficient and effective_—for example, TopK sparse autoencoders (SAEs) have demonstrated the ability to reconstruct DINO representations with nearly 80% R^2 using as few as 10 concepts[2].\n\n[1] Towards Automatic Concept-based Explanations\n\n[2] Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models", "other_comments_or_suggestions": "See my previous points.", "questions_for_authors": "Regarding the last point (**m.2**), one question concerns whether this method could be used to remove biases by targeting high-level concepts rather than merely individual data points.\n\nAnother query would be can we understand a data point influence as the number of time it act / is used in the decision ?\n\n Related to this, could this score be used to select better quality/diversity on the dataset ?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Instead of fine-tuning to adapt your image classifier to a new domain, use a kNN on your new images with distances provided by the network.

As reviewers point out, the idea is not new. At the same time, it is also not popular, with fine-tuning dominating the field and this kind of method hardly even being used as a baseline. This is despite the fact that, as authors point out, the method works well. Perhaps the tagline of "visual memory" will help it become more popular.

One could imagine many practical applications that would benefit from adopting such a method. For example, ones where memory editing is crucial. Popularizing it can only help the field along.

My two main suggestions to the authors would be to 1. highlight other advantages of the method which are only shown in the appendix 2. more directly in the abstract and the introduction describe this as a revival or repositioning of something that already exists and can be very useful to readers. At the moment both overclaim by failing to note that this method is already well known. And, by doing so, cut off the reader from a rich literature about such methods.
Decision: Accept (poster)