Probing Layer-wise Memorization and Generalization in Deep Neural Networks via Model Stitching
Authors: Aishwarya Gupta
Indranil Saha
Piyush Rai
Venue: Accepted by TMLR ()

Abstract: It is well-known that deep neural networks can both memorize randomly labeled training data and generalize to unseen inputs. However, despite several prior efforts, the mechanism and dynamics of how and where memorization takes place in the network are still unclear, with contradictory findings in the literature. To address this, we aim to study the functional similarity between the layers of the memorized model and the model that generalizes. Specifically, we leverage model stitching as a tool to enable layer-wise comparison of a memorized noisy model, trained on a partially noisy-labeled dataset, to that of the generalized clean model, trained on a clean, noise-free dataset. Our simple but effective approach guides the design of experiments that help shed light on the learning dynamics of different layers in deep neural networks and why models with harmful memorization still generalize well. Our results show that early layers are as important as deeper ones for generalization. We find that ``cleaning'' the early layers of the noisy model improves the functional similarity of its deeper layers to that of the corresponding layers in the clean model. Moreover, cleaning the noise in the early layers of the noisy model can drastically reduce memorization and improve generalization. Furthermore, noise fixation up to a certain depth results in generalization similar to that of a noise-free model. However, interestingly, the reverse may not be true. That is, if early layers are noisy but deeper layers are noise-free, then perfect memorization cannot be achieved, emphasizing the dominant role of deeper layers in memorization. Our extensive experiments on four different architectures - customized CNN model, ResNet-18, ResNet-34, and ResNet-50, and three datasets - SVHN, CIFAR-10, and CIFAR-100, with varying levels of noise, consistently corroborate our findings.

Published in Transactions on Machine Learning Research (MM/YYY)
Probing Layer-wise Memorization and Generalization in Deep Neural Networks via Model Stitching
Aishwarya Gupta, Indranil Saha, Piyush Rai Deportment of Computer Science and Engineering, IIT Kanpur, India {aishwaryog, isaha, piyush Bese. islk.ac.in ac.in
Reviewed on OpenReview: https: // openreview. net/ forum? id= XXXX
Abstract
It is well-known that deep neural networks can both memorize randomly labeled training data and generalize to unseen inputs. However, despite several prior efforts, the mechanism and dynamics of how and where memorization takes place in the network are still unclear, with contradictory findings in the literature. To address this, we aim to study the functional similarity between the layers of the memorized model and the model that generalizes. Specifically, we leverage model stitching as a tool to enable layer-wise comparison of a memorized (noisy) model, trained on a partially noisy-labeled datasel, to that of the generalized (clean) model, trained on clean, noise-free dataset. Our simple but effective approach guides the design of experiments that help shed light the learning dynamics of different layers in deep neural networks and why models with harmful memorization still generalize well. Our results show that early layers are important deeper ones for generalization. We find that "cleaning" the early layers of the noisy model improves the functional similarity of its deeper layers to that of the corresponding layers in the clean model, can drastically reduce memorization and improve generalization. Furthermore, noise fixation up certain depth results in generalization similar that of a noise-free model. However, interestingly, the reverse may not be true. That is, if early layers are noisy but deeper layers noise-free, then perfect memorization cannot be achieved, emphasizing the dominant role of deeper layers in memorization. Our extensive experiments on four different architectures customized CNN model, ResNet-18, ResNet-34, and ResNet-50, and three datascts SVHN, CIFAR-10, and CIFAR-100, with varying levels of noise, consistently corroborate our findings. The code available at lhttps://gitae com/ http//ashggpttaeemorizaitiiioiot wttoclonsstttttiiniieeeeeee
1
Introduction
Deep Neural Nets (DNNs) widely used across many domains because of their impressive generalization capabilities unseen inputs. Yet, these models are also prone to memorization, and can fit randomly labeled dataset perfectly Zhang et al. (2021). A memorized input is known to be inexplicable using generalizable features Wei et al. (2026), and usually represents atypical inputs in long-tailed distributions Feldman & Zhang 2020 : Feldman 2020), or the underlying noise/ outliers in the training dataset Zhang et al. (2021) J. Although memorization may occasionally improve downstream performance in imbalanced or long-tailed distributions Maennel et al. (2020): Anagnostidis et al. (2023). , raises significant concerns, including privacy leakage and vulnerability to membership inference altacks Leino Fredrikson (2020); Carlini et. al.
(2022). Therefore, understanding when and how memorization occurs is critical for building more robust and interpretable models.
The understanding of the mechanisms and dynamics of memorization in DNNs has intrigued researchers since the seminal work of Zhang et al. Zhang et al. (2021). Numerous diverse approaches have been leveraged to study and find answers foundational questions, such as how model can simultancously memorize noise and generalize to unseen data Krueger al. (2017) Arpit et al.) (2017). whether all layers in a network converge similarly in the presence of label noise whether certain layers are more susceptible to memorization than

---

Published in Transactions on Machine Learning Research (MM/YYY)
others Stephenson et al. 22021 Wongso et al 2023 Maini et al. (2023). and on. These studies not only helped in understanding how memorization manilests in deep neural networks, but they also laid the foundational work for approaches designed to mitigate memorization. However, despite significant progress, these efforts have sometimes resulted in divergent conclusions, especially regarding the susceptibility and role of the individual layer in memorization. For instance, (Stehhenson et al. (202T) showed confinement of memorization to deeper layers of the model, while Wongso et al. (2023) showed that carly layers are also involved in memorization despite their relatively faster convergence than deeper layers. Furthermore, |Maini et al. 2023) localized memorization to a small subset of neurons spread across all layers in the network. All these findings capture different facets of memorization, highlighting the complexity of disentangling memorization from generalization.
In this work, we build on these insights and ask a fundamental question: What is the functional similarity between the layers of model that memorize and those that generalize? Specifically, our goal to assess the impact of memorization on the functionality of individual layers, which we define terms of their ability align with the behavior of their clean counterparts. To answer this, we compare a memorized model, trained randomly labeled inputs, to generalized model trained on the clean, freee-free dataset. We leverage the recently proposed stitching layer Csiszárik et al. (2021); ); Bansal et al. (2021), linear trainable layer that 'stitches' two models by aligning their latent representations, as a tool to enable the comparison of layer-wise functional similarity between models. We stitch the memorized model to the generalized one at varied depths and vice versa, and evaluate all stitched models on two downstream tasks - (i) generalization to unseen test data, and (ii) memorization accuracy on randomly labeled training inputs quantify similarity seeeartt//diveeeee /divergence between their generalizing and memorizing functional behavior. Our methodology provides insights into the following fundamental questions:
How does fixing (replacing) certain layers affect memorization and generalization?
How does memorization propagate through the network?
Are all layers equally responsible for memorization, or do some play a more critical role?
How do the inherent bias of the architecture, underlying noise level, and task complexity influence the impact of memorization at different. layers?
Our results uncover consistent and surprising patterns across architectures and datasets, offering new insights into how memorization emerges, evolves, and influences the functional behavior of individual layers. Our contributions and key findings can be summarized as:
We propose a novel framework for evaluating the impact of memorization on each layer of the DNN model using model stitching, enabling a direct layer-wise functional comparison between models trained with and without noise. To the best of our knowledge, our work the first of its kind, enabling direct one-on-one functional comparison between memorized and generalized models.
We experimentally analyze four architectures CNN, ResNet-18, ResNet-34, and ResNet-50 on three datasets - SVHN, CIFAR-10, and CIFAR-100, with varying extent of noise memorization. Our results consistently demonstrate the following:
1. We observe that noise memorization impacts each and every layer of the model, and replacing early layers with clean counterparts improves the test-time performance of the model.
2. If a sufficient number bottom layers are replaced by their clean counterparts in the noisy model, then often the memorized accuracy of the model drops a random guess. That is, despite the presence of noisy deep layers, fixing the early layers mitigates memorization. We hypothesize that this is attributed to the feedforward nature of DNNs that compounds noisc present in early layers during the forward pass. If the noise in early layers is fixed, noise cascading is also minimized, rendering noise in deeper layers benign.
3. Deeper layers are necessary for perfect memorization. The insertion of early noisy layers in the clean model shows the emergence of non-trivial memorization, but still, 100% memorization
2

---

Published in Transactions on Machine Learning Research (MM/YYYY)
is only achieved when noise is present in deeper layers as well. This corroborates the initial findings of training/ /fixig///nggg deeper layers to reduce memorization.
4. We observe the infueecc of the inherent bias of model architecture and task complexity on the extent of memorization in each layer. Specifically, the number of early layers that, if fixed (replaced with clean layers), can mitigate memorization is jointly decided by the task complexity and model architecture.
We also perform preliminary experiments on more complex and deeper architectures, such as Vision Transformer and ResNet-101, to probe layer-wise memorization and its impact on the model's end-to-end behavior. Our experiments show promising directions that could be explored at length in future works to enhance the understanding of memorization.
2 Related Work
Memorization DNNs are infamously known to perfectly memorize randomly labeled training inputs when heavily over-parameterized, exhibiting universal finite-sample expressivity Zhangg et al. (2017). In the presence of label noise, model training starts with learning simple, generalizable patterns, followed by memorizing specific, randomly labeled training imputs in the later stage Krueger et al. (2017); Arpit et al. (2017). This observation has inspired carly-stopping approaches to curb memorization while preserving generalization Liu et al. 22200; Frankle et al. (2020). Moreover, multiple factors such as the inductive bias from the model architecture Zhang et al. (2020). explicit regularization, or training protocols Arpit et al. (2017 play a significant role in how quickly and how much a model can memorize.
Beyond mitigating memorization, considerable work has been done in understanding where and how memo- rization manifests within model. A variety of approaches have been proposed to probe memorization, such prediction depth via replica- based mean-field geometry Stephenson et al. (2021) of each input, manifold analysis techniques Chung et al. (2018), sliced mutual information Wongso et al. [2023) gradient accounting, and layer retraining Maini et al. (023)- Interestingly, each of these methods has captured different facets of memorization, sometimes resulting in divergent conclusions. For instance, Stephenson et al. 20221) analysed the geometric properties of class manifolds throughout training and found that memorization is largely confined to deeper layers. They further showed that rolling back the weights of the final convolution layer to early epoch can reverse memorization to great extent. However, WWngsoo et al. (2023) showed that all layers of the model memorize and early layers stabilize earlier than deeper layers. A recent work Maini et al. (2023) combined techniques like gradient accounting, layer rewinding, and selective retaining to localize memorization within the model. They discovered that memorization is not limited to any single layer; instead, it is confined to a subset of neurons spread across various layers. This suggests that memorization affects all layers of the model (although through a relatively small subset of neurons in each layer), supporting the findings of Wongso et al. (2023).
Model Stitching Model stitching is introduced by Lene & Vodaldi (2015) to study the equivalence of representations. The stitching layer is a learnable linear translormation that connects two models by aligning their intermediate representations. Model stitching has become a popular approach for building new architectures adhering to specific resource and performance requirements by combining components from existing models of varying complexity Yang et al. (2022); Teerapittayanon et al. (2023) Pan et al. (2023 2024); He et al. (2024): Xu et al. (2024). Additionally, ly, rescarchers use stitching to probe functional similarity between models' hidden representations, assessing how well cross-wired components perform on the same task Bansal et al. (2021) Csiszárik et al. (2021). Empirical evidence shows that even when two networks' representations score low on metrics like CKA, they can still exhibit high functional similarity, demonstrating that representational and functional similarity capture complementary aspects of how representations relate Bansal el al. (2021). For instance, stitching has revealed strong functional correspondence between adversarially robust and standard (non-robust) models Balogh & Jelasity (2023), despite these networks showing low CKA-basod similarity Cianfarani et al. (0022). Accordingly, we employ the stitching layer as a tool to mcasure functional similarity between a generalized (clean) model and a memorized (noisy) model. Importantly, our setup is more challenging than Balogh & Jelasity (2023) owing to the noisy model trained on partially
3

---

Published in Transactions on Machine Learning Research (MM/YYYY)
randomly corrupted labels, and thus, receiving conflicting supervisory signals compared to the clean model, which is trained exclusively on correctly labeled data.
3 Methodology
Our goal is to investigate how memorization manifests across different layers of the model and how it affects generalization. We adopt the experimental setup of Maini et al. (2023), introducing randomly labeled inpuls into the training dataset to induce memorization. Nexi, we train a separate model on clean data without label noise so that none of its training imputs have to be specifically memorized, and use it as a bascline for gencralization. The core idca is that the clean model, owing to zero-noise exposure, exhibits ideal generalization behavior and thus, direct one-on-one comparison of internal representations with the memorized model can uncover the layer-wise functional differences that emerge due to noise memorization.
We follow the definition of functional similarity as de- fined by [Siszárik et al. [2021] Bansal el al. (2021) and leverage model stitching as a tool to maximally align internal representations between generalized (clean) and memorized (noisy) models with identi- cal architecture. Here, by "maximization of align- ment", we refer to the maximization of the possible functional aligument between represenlations with respect to the downstream task. A stitched model is constructed by joining two models fr, and fra, trained with different noise level rı and r2 respec- tively, at some intermediate layer 1. A trainable linear stitching layer S' is inserted to align the out- put of layer i of fr, as the input to layer ++1 of Srg. The stitching layer S is trained by minimizing the loss function of the stitched model (with weights of fr, and fr, frozen), and when the loss converges, the learned stitching layer offers maximum possible functional alignment between the representations.
Clean model
Nolsy model
Layer
90
hij
h
Stiching layer
Clean to noisy model stitching
Noisy to clean model stitching
Figure 1: Stitching clean and noisy model at layer idx Clean-to-noisy model stitching consists clean base, stitching layer, and noisy head, while noisy-to-clean stitched model consists of a noisy base, stitching layer, and clean head.
This results in a model of the form: ()( where of and h2d denote the base and head parts of the stitched model, respectively. Here, we represent the base or early part of the stitched model by 9, and the head or latter part of the stitched model by h. We annotate g with subscript 1ı and superscript < [ to show that part of fr, up to layer 1 forms the basc of the stitched model. Similarly, h2d shows that part of the model fra from layer (I +++ 1) onwards forms the head of the stitched model. From here on, for brevily, "stitching at layer 1" refers to inserting a trainable stitching layer between the output of layer I in the base model and the input to layer (I ++ 1) in the head model.
We experiment with two stitching directions:
Clean-to-noisy h2 h²"(((g)")) where the clean model fo up to layer forms the base, and the latter part of the noisy model Jr (starting from layer (1 + 1)) forms the head of the stitched model.
Noisy-to-clean ((((())))) where the noisy base (fr up to layer 1) is stitched to clean head (fu from layer 1+1 onwards).
The stitching process is visualized in Figure The accuracy of the stitched model reflects how well the intermediate representations of the two models can be linearly transformed and aligned. As result, the performance gap between the stitched model and its original constituent models serves proxy for functional dissimilarity caused by memorization. For example, the performance gap between hy (S(o)")) and fo arises from replacing the early layers (up to layer I) with the corresponding layers of the noisy model fr. This performance gap w.r.t fo serves as a mcasure of the collective functional similarity between the early layers (1 to 1) of the clean fo and noisy fr model. Similarly, performance comparison of h²'(S(g)")) to fr evaluates the functional similarity between the stitched part of the clean model and the unstitched part of the noisy
4

---

Published in Transactions on Machine Learning Research (MM/YYYY)
model (i.e., fo and )) However, the key component is the dataset used to train the stitching layer 8, on which intermediate representations are maximally aligned between models. In our setup, we train stitching layers S on the correctly-labeled validation dataset D unseen by both fo and fr. We choose the validation set Dual instead of the training set of either the fo or fr model so that the training of the stitching layer does not interfere with divergent representations of memorized inputs learned by the two models. Furthermore, training a stitching layer on Dval, which is disjoint from training data, ensures alignment of generalizable representations rather than memorized patterns, limiting memorization to only the stitched part of the noisy model.
4 Experiment Setup
We understand memorization dynamics in Deep Neural Networks (DNNs) using four different architectures a customized CNN comprising three convolutions and four fully connected layers, ResNet-18, Res.et-3A. and ResNet-50 He et al. (2016). Our experiment spans three datasets SVHN Netzer et al. (2011), CIFAR-10, and CIFAR-100 Krizhevsky et al. (2009). We train customized CNN on SVHN, ResNet-18. and ResNet-34 on CIFAR-10, and ResNet-34 and ResNet-50 on CIFAR-100, resulting in a total of 5 distinct models. We also perform preliminary experiments on ResNet-101 and ViT-Small trained on CIFAR-100 and mention experimental findings in Appendix [A. and AA.3 respectively. To study memorization, we introduce label noise into the training data by assigning random labels to a fraction of training images. We consider noise levels r = (5%, 10%, 15%, 20%, 25%, 37. 37.5%, 50%} and train each model on a noisy dataset D, until its training accuracy exceeds 99% ensure noise memorization. Note that each noise-injected training dataset Dr comprises both correctly labeled imputs D. and randomly labeled inputs D We also train model on the clean dataset DO (i.c., 0), where all training images are correctly labeled to serve as a generalization baseline.
A model trained on D has been exposed to zero random noise during training and does not memorize any noisy-labeled training input. We refer to this model as the noise-free or clean model and use it as a generalizod model for comparison. Contrary to this, all models trained on noisy datasets must memorize randomly assigned labels during training and thus are referred to as noisy models. Notably, while the same images appear in D and D their labels differ, inducing divergent learned representations, particularly for D By divergent, we refer to the latent representations of the randomly labeled imputs in the clean and noisy models. The clean and noisy model maps them to correct and random labels, respectively, resulting in different latent representations. Such representations are termod "divergent" as one cannot be mapped to another explicitly during training on the validation dat Our selection of validation as a training datasel for stitching layers ensures that these divergent representations are not at all explicitly aligned, restricting noise localization only to the noisy part of the stitched model, and thus, playing a crucial role in understanding the manifestation of memorization and the generalization drop in noisy models.
In our experiments, we have performed stitching for the CNN model after every convolution and fully connected layer, and for ResNet models, it is performed after every residual block. To get stitched models with the clean base and noisy head h2 (S(o)))))) we start with a noisy model fr and incrementally replace its initial layers/blocks with those from a clean model. The process begins with stitching the first layer layer/block /block of the clean model onto the remaining increasing the number of layers/blocks blocks of the clean model to be stitched onto the corresponding remaining blocks of the noisy model. So, in the first round, only the first layer block is stitched. In the second round, we stitch up to the first two hoyers/blocks, and so on, cach time replacing longer prefix of the noisy model with the corresponding clean layers/blas blocks. This process continues until all layers/ /boock,, except for the last one, of the noisy model are replaced by their counterparts from the clean model. Similarly, we start with a clean model fo and incrementally replace its initial layers/ blocks with those from a noisy model, resulting in a series of stitched models with the noisy base and clean head h (S(g)")), each corresponding to a different stitching depth 1. The training convergence plots of all stitching layers are presented in Appendix A.6
blocks of the noisy model. Next, we repeat it by incrementally
5

---

Published in Transactions on Machine Learning Research (MM/YYY)
5 Results and Findings
We organize the results inlo two main parts: the memorization behavior and the generalization performance of stitched models, aiding in the collective understanding of the functional behavior of the clean and noisy models.
5.1 Memorization
The stitching layer is trained to maximize the representation alignment. between the two models on the correctly labeled validation dataset. As a result, the stitching layer cannot align the divergent representations learned by the clean and noisy model on D dataset, limiting noise memorization in the stitched model to the constituent part of the noisy model. This raises the following key question: 1) Do stitched models comprising a partial clean model still exhibit memorization when evaluated on D 2) If yes, then to what extent? 3) Does the stitching depth impact the extent of memorization in the stitched model? We address these questions by evaluating the accuracy of stitched models on the noisy subset of the training dataset, i.e., DR, and use it as a measure to quantify and compare memorization. inputs plotted on the Y-axis as a function of stitch depth on the X-axis for stitched models, and each line plot corresponds to a specific value of noise r. Note that zero memorization means the stitched model does not recall those specific random label assignments, resulting in random accuracy (100/N, being the number of classes) memorized inputs.
all figures, the accuracy of memorized
5.1.1 Clean-to-Noisy Stitching h?? hz2(5(g0)))
The accuracy of noisy models, stitched models (S( ())) and the clean model on D is plotted in Figure 2 Each subplot corresponds to specific architecture and dataset. The key observations are summarized below.
M.1 On simple tasks, fixing the very first layer/blodd substantially reduces memorization. For smaller datasets such as SVHN and CIFAR-10, stitching in just one or two clean layers/ /bockss dramatically suppresses memorization (quantified as accuracy on randomly labeled training inputs D²), regardless of noise ratio r. For example, at r = 50%, the memorized accuracy drops below 50% for both datasets by replacing only the first noisy layer/block by its clean counterpart. In contrast, for a more challenging dataset like CIFAR-100, achieving similar drop 50% requires stitching clean layers up to at least the middle block.
M.2 Stitching up to the minimum clean prefix 1TM reduces memorization to random. Across architectures and datasels, memorization consistently decreases as more clean bottom layers are stitched in, eventually collapsing to near-random accuracy once the clean base reaches a critical depth 1mh This Ish represents the minimum clean depth needed to "forget" memorization. Notably, the stitch depth pm never lies among the final layers, indicating that noisy upper layers alone cannot sustain memorized behavior once early noisy layers are replaced by their clean counterparts.
M.3 Architecture and task complexity strongly influence 1T The required stitch depth l. is shaped by both architectural inductive biases and dataset difficulty. CNNs exhibit a steep drop in memorization after introducing the first clean layer, while ResNets on CIFAR-10 show slower decline-liked due to residual connections. On CIFAR-100, memorization initially decreases very gradually, followed by sharp drop only after sufficiently large clean prefix is stitched in. Even within the same architecture (c.g., ResNet-34), the pattern of memorization decay differs notably between CIFAR-10 and CI0AR-100. Also, higher noise ratios produce a larger set of memorized examples, making the corresponding model more sensitive to layer replacement. Consequently, stitching clean bottom layers induces sharper decline in memorization for larger values of r. Nevertheloss, for a given architecture and model, the decreasing trend of memorization exhibits a similar pattern across all noise ratios.

---

Published in Transactions on Machine Learning Research
100
100
100
Memorized Accuracy (%%
80
Memonzed Accuracy
80
10.0
Memorized Accuracy
60-
40
46
3000
20-
20
5000 oohy
Mocm dlen
stert notoy
sslayed..
000 0p3n
saat noly
mooowr
Rlayodd
Stitchnng depth
Stitching depth/
Stitching depth
(a) CNN model on SVHN
(b) ResNet-18 on CIFAR-10
(c) ResNet-34 on CIFAR-10
Memorized Accuracy (96
1P
3.0
30.0
Memorized Accuracy
750
5.0
60
20.0
25.0
6
30.0
15.0
37.5
40
50.0
40
x2-
22
s0nt- no8y
0 00.0
ond- 003n
Saver 130b
cni-doon deon
Stitching depth
Stitching depth J
(d) ResNet-34 on CIFAR-100
(e) ResNet-50 CIFAR-100
Figure 2: Clean-to-Noisy Stitching ( Accuracy of stitched models on randomly labeled training inputs D of the constituent noisy model. Each xtick corresponds to a stitched model comprising clean base up to the layer I, stitching layer, and the noisy head starting from layer [+ 1 to the last layer. The black vertical line shows the minimum depth of clean base 1tth required mitigate memorization noisy models.
5.1.2 Noisy-to- Cisye Stitching h? ()
The memorization accuracy (i.c., accuracy on randomly labeled training inputs) of stitched models (7*((( along with the clean and noisy baselines shown in Figure? 3 The key observations are:
M.4 Early noisy layers do not immediately induce memorization. While stitching in early clean layers in the noisy model significantly reduces memorization (as seen in clean-to-noisy stitching Figure [ the converse does not hold. Adding a few noisy carly layers to a clean model does not immediately increase memorization. Instead, memorized accuracy remains close to random for shallow stitching depths and begins rise slowly only when the number of noisy bottom layers exceeds the number of remaining clean upper layers.
M.5 Memorization emerges only after surpassing the minimum noisy prefix In but never reaches 100%. Once the stitching depth exceeds certain depth !", the stitched model begins though its head remains noise-free. This depth is referred
to show nontrivial memorization to as minimum noisy suffir needed for the emergence of memorization. However, noisy-to-clean stitched models never show full memorization, even if all layers except the output layer are noisy. For example, when only the output layer is clean, the CNN model trained on SVHN achieves only 20-30% accuracy on random labels, ResNet-34 achieves at most 55% on CIFAR-10 and 30% on CIFAR-100, and ResNet-50 reaches 60% on CIFAR- 100. This demonstrates the dominant role of the clean head in limiting the model's ability to memorize arbitrary labels.

---

Published in Transactions on Machine Learning Research
100
100
100
Memorized Accuracy (%%
20
Memonzed Accuracy
Memorized Accuracy
60
40
20
dlan
Mogee Lo--
9 2.00
voce nony
stbrt denn
d 0--b
g D-
33lppent.004
000 nolo
Sait- cetn
Stitchnng depth
Stitching depth
Stitching depth
2
(a) CNN model on SVHN
(b) ResNet-18 on CIFAR-10
(c) ResNet-34 on CIFAR-10
Momorized Accuracy (%)
(44)
Memoried (%)
(04)
5.0
5.0
80
10.0
10.0
15.C
15.6
60
20.5
25.6
60
25.6
20.0
37.5
37.5
40
50.0
60
50.6
20
20
skayer 00-b
S2ago22.007
und moay
desn
Staverl22 2
SMayS22353
und-nota)
SDaT
Stitching depth
Stitching depth /
(d) ResNet-34 CIFAR-100
(e) ResNet-50 CIFAR-100
Figure 3: Noisy-to-Cless Clenn Stitching (S(,")) Accuracy of stitched models on randomly labeled training inputs Dr Each xtick corresponds to a stitched model comprising noisy base up the layer stitching layer and clean head starting from layer ++ 1 to the last layer. The black vertical line shows the minimum depth of the noisy base t required for the emergence of memorization in the clean model.
5.2 Generalization
We assess the generalization of stitched models to address the following key questions: 1) Does the noisy model's generalization improve a deeper clean base is stitched in? 2) Does the clean model's generalization degrade more noisy layers are stitched in? 3) Can full generalization be mainlained even when a part of the model remains noisy? We evaluate and compare the accuracy of stitched models against both the clean and noisy models on the unseen test dataset (Dtest). Note that when the test accuracy of the stitchod model matches that the clean model, it indicates zero adverse impact of the training noise on its functional behavior, and thus, can be referred to as full generalization.
5.2.1 Clean-to-Noisy Stitching )²*(((g6))) ())
We stitch the early clean layers () the latter noisy layers (h?1)) at varied depths, and report their test. accuracy in Figure 4 The main observations are summarized below.
G.1 Replacing even the first noisy layer / block significantly improves gencralization. Across all architectures, stitching in just the first layer/block yields a notable boost. in Lest. accuracy, especially for high noise percentages. For instance, at r = 50%, stitching in the first clean convolution layer in the CNN model or the first clean residual block in ResNet-18 boosted test performance by roughly 8-10%. This indicates that the early layers of the noisy model have also learned some non non-generalizable features, and swapping them out has outsized positive effect.
8

---

Published in Transactions on Machine Learning Research
8
last Accuracy (%6)
Accuracy (%)
Accuracy
800
(1%)
5.0
5.0
1791
10.0
20.0
15.0
33.0
20.0
20.0
25.0
6:
25.0
37.5
&
37.5
50.3
20.0
09
gsayers
state naly
end den
BnCW caan
Skate
Stitching depth
Stitching depth
Stitching depth
(a) CNN model trained on SVHN
(b) ResNet-18 model trained on CIFAR-10
(c) ResNet-34 model trained on CIFAR-10
75
75
Accuracy
Accuracy
65
60
15.0
15
sur nony
9 1.00n
a n
anope
end oeen
sait: mory
0 Do
95 1>
94.992222 2.2
9 2-
0-
@Sayee4 10
eskayert
end. den
9
Stitching depth /
Stitching depth /
(d) ResNet-34 trained CIFAR-100
(e) ResNet-50 trained CIFAR-100
Figure 4: Clean-to-Noisy Stitching (²"(S( )):: Accuracy of stitched models on unseen test dataset. Each xtick corresponds a stitched model comprising clean base up to the layer I, stitching layer, and the noisy head starting from layer I+1 to the last layer. The black vertical line shows the minimum depth of the clean base 1 required for restoring (almost) full generalization the noisy model.
G.2 Stitching beyond the sufficient clean prefix Ig yields negligible gains. The test performance of the stitched models lies between that of the constituent noisy fr and clean fo model. The test accuracy consistently improves with increase in stitching depth 1 (more clean layers in the base), eventually matching the performance of the clean model once the clean base reaches critical depth gg. This 1 represents the minimam clean depth needed to restore full generalization. Interestingly, 18 never corresponds to the final layer, showing that despite the noise influence present in the head h²2' layers the model, replacing an appropriate subset of early layers is enough to achieve generalization.
G.3 Architecture and task complexity influence the depth of 1 The location of 1 depends on both architectural inductive biases and dataset difficulty. For example, in the CNN model trained on SVHN, 4 = fc2 lies near the end of the architecture, whereas in ResNet-18 trained on CIFAR-10, ig = 1ayer4.00 0 occurs much earlier relative to the model's depth. Even within the same archilecture, such as ResNet-34, the depth of ig differs across datasets. On CIFAR-10, 4, 1ayer4. O, the second last residual block (similar to ResNet-18 trained on CIFAR-10), but for the more complex CIFAR-100 dataset, performance continues improving up lo the last residual block layer4. analogous to ResNet- 50 where 4, 1 2. Strikingly, for any given architecture-dataser pair, all stitched models converge to similar test accuracy at (g, regardless of the noise r, and exhibit similar improvement trajectorics.
9

---

Published in Transactions on Machine Learning Research
8
last Accuracy (%6)
Accuracy (%)
(1S1
00
5.0
5.0
$0.0
15.0
25.0
50.0
50.0
34at> dkan
09ayer3000
gRayers
ande nony
Mart denn
3 0ywer2.rr
038yet.00
end noby
Skate
BRCH
Stitching
Stitching depth
Stitching depth
(a) CNN model on SVHN
(b) ResNet- 18 model on CIFAR-10
(c) ResNet-34 model on CIFAR-10
75
Accuracy
65-
655
60
55
15
56.0
RMr oen
g. 1000
95xype2.1x
end noby
sait: cart-des denn
3Swyer1 Do
95ayye2. 1>
9..1192.22 2.2
39-h
033y04 0-
uskayert
end. robo)
Stitching depth /
Stitching depth /
(d) ResNet-31 model on CIEAR-100
(e) ResNet-50 model CIFAR-100
Figure 5: Noisy-to-Clen Cleen Stitching A ( Accuracy of stitched models on unseen test dataset. Each xtick corresponds to a stitched model comprising noisy base up to layer stitching layer and the clean head starting from layer ++1 1 to the last layer. The black vertical line shows the minimum depth of the noisy base 1g beyond which the generalization drops steadily the clean model.
5.2.2 Noisy-to-Clest Clean Stitching h²((((),"))
We now consider the reverse direction: injecting early noisy (memorizationn-pron) layers into the clean model. Figure 5 presents the test accuracy of noisy-to-clean stitched models (y-axis) a function of stitch depth (x-axis). The key observations are summarized below.
G.4 Stitching a few early noisy layers causes minimal degradation. After stitching in a few early noisy layers, the test performance of the stitched model remains close to that of the clean model. This indicates that a limited number of noisy bottom layers does not immodiately harm generalization, likely because the majority of the upper clean layers can compensate or override the noisy representations introduced shallow depths.
G.5 Stitching beyond the minimum noisy prefix 1 degrades performance As stitching depth increases, test performance initially changes little, but then exhibits a sharp drop once the stitched noisy prefix exceeds critical depth gg. This gg referred to the minimum noisy depth that offects generalization, marks the point at which noisy early representations overwhelm the corrective influence of the clean upper layers. Beyond i test accuracy steadily declines and approaches that of the fully noisy model. Moreover, across all architectures and noise levels r, 1g consistently appears in the first half of the model. This suggests that only a small number of noisy bottom layers can be tolerated before generalization deteriorates. However, even when all layers except the final classifier are noisy, the stitched model generalizes noticeably better than the noisy model, with improvements as large
10

---

Published in Transactions on Machine Learning Research (MM/YYYY)
13-20% at 50%. This highlights the dominant role of even single-layer clean head that
significantly boost generalization, even when the entire base is noisy.
5.3 Interpretations and Inferences
We summarize the key observations from stitching clean layers noisy model and vice verse in Table |
Table 1: Summary of generalization and memorization behavior under dlean-Lo-noisy and noisy-to-cloan stitching.
Stitching Direction
Generalization (Test Accuracy)
Memorization (Random-Label Accuracy)
Clean + Noisy h?"(S(90")) )
Early clean layers give large gains.
Eaaly clean layers drops accuracy.
Improves with deeper clean base.
Forgets memorization after stitchiin depth 1""
Matches clean model after stitching depth eg.
Clean base suppresses noisy head.
Noisy - Clean (5((5))))))
Shallow noisy base has little effect.
Negligible for small noisy prefix.
Sharp decline beyond stitching depth
Emerges only after stitching depth 4
e. Always better than the fully noisy model.
Never reaches 100% due to clean head.
5.3.1 Fixing noise early layers improves generalization and reduces memorization in noisy models
Our experiments demonstrate that fixing early layers of the noisy model, starting from the very first /lock/block (see observation G.1., improves its generalization. The test accuracy of the noisy model increases steadily as more clean layers are stitched in (obs. G.2) and saturates once the clean base reaches a critical depth 1g. A similar behavior is observed for memorization, where memorized accuracy decreases as clean early layers are introduced (obs. Collectively, these observations show that even when noise exists only in the upper layers the stitched model behaves similarly to the clean model on both unseen and memorized inputs once carly noisy layers up to certain depth are replaced.
and reduces to near-random behavior once layers up to to are fixed (obs. M..).
We attribute this to the hierarchical structure of DNNs, where the deeper layers depend directly on the representations produced by carlier oncs. Any abnormality or noisc-inducod distortion in the carly layers propagates and amplisies with depth. Consequently, replacing noisy early layers with clean ones effectively removes the source of distortion, improving generalization while suppressing memorization.
5.3.2 Stitching in noisy early layers to the clean model results in memorization with reduced generalization
Replacing the first few clean layers with noisy ones up to depth 19 does not degrade generalization (obs. maintaining test accuracy close to the clean model and keeping memorization near random. This is mainly due to the presence a larger number of clean upper layers that can compensate for the noisy base. However, once the depth of the noisy base exceeds 1g, memorization emerges and test accuracy decreases, approaching the performance of the fully noisy model. This shows that although the model can tolerate a small noisy base, excessive corruption in early layers imposes substantial burden on remaining clean layers, causing generalization to drop.
G.5 M.5
However, stitched models with the noisy base and clean head never achieves perfect memorization (100% accuracy on randomly labeled training inputs), even when all layers except the last layer are noisy. Instead, depending on architecture and dataset complexity, memorized accuracy ranges from as low as 20% to at most 90%. This highlights the critical role of the head layers cnabling complete memorization. This obscrvation is consistent with existing memorization mitigation techniques, many of which modify or regularize the final layers. At the same time, the fact that noisy-to-clean stitched models still exhibit some memorization shows
11

---

Published in Transactions on Machine Learning Research (MM/YYYY)
that every layer participates in memorization. While early layers contribute, perfect memorization appears to require noise across all
the head.
5.3.3 Implicit bias of architecture and task complexity on critical stitching depths
We study the impact of architecture and task complexity on layer-wise memorization and minimum stitching depth needed to restore generalization or suppress memorization. We experiment with two different architec- tures on the same dataset. Res-eet-8 and ResNet-34 models trained on the same noisy variants of CIFAR-10, and study the behavior of stitched models. In this case, task complexity is fixed, and the model capacity varies. We observe that although the positions of critical depths (4g, (") ln and (T)) are broadly similar, the rate at which accuracy changes with stitching depth differs. For instance, while both architectures exhibit the same minimum noisy depth mn for memorization to emerge, the increase in memorized accuracy varies (see Figure
Conversely, when studying the same architecture (ResNet-34) across datasets of different complexity (CIFAR- 10 vs. CTFAR-100); we find that the change in performance with stitching depth follows similar trajectory, but the location of critical stitching depths shifts. For instance, in ResNet-34, noisy base up to layer4.0 marks the emergence of non-trivial memorizat ion CIFAR-10, while it is pushed deeper to the last residual block layer4.2 2 for CIFAR-100 (consistent with observations in ResNet-50 architecture for CIFAR-100). These patterns demonstrate that both architectural capacity and task difficulty jointly shape the positioning of critical depths and the rate of change of memorization and generalization performance.
5.3.4 Functional similarity between layers of the noisy and clean model
As the stitching depth in clean-to-noisy stitching increases, the test performance of the stitched model improves relative to the noisy model. This shows that clean layers exhibit better generalization. Conversely, in noisy-to-clean stitching, the test performance of the stitched model drops relative to the clean model after stitching beyond 12, and the degradation becomes more prominent as stitching depth increases. This suggests degradation compounds in depth. In essence, the collective functional similarity
that memorization-iniee between layers of the clean and noisy models (as measured by the performance gap of the stitched model relative to the head model) decreases with depth. This further supports the idea that the generalization capability of the carly layers in the noisy model is weaker than that of the clean model, highlighting how the negative impact of noise propagates through the network. Furthermore, these findings align with observations memorization in individual layers. All these experiments clearly mark the
made by in in-place noise influence in early layers and how fixing it can reduce memorization and improve test accuracy.
6 Discussion and Future Work
Using model stitching, we have analyzed the learning dynamics of each layer of DNN trained with random-label noise. Our empirical analysis both corroborates carlier findings in the literature well provides new insights. In particular, our experiments show that early layers are infected by training noise, and this is cascaded further in the DNN model owing to its feedforward nature. Furthermore, across datasets and models, we consistently observe that fixing early noisy layers can improve the model's performance on unseen data and reduce the accuracy on memorized inputs, even when noisy deeper layers remain untouched. To further verify and corroborate our findings, we perform another experiment (sce Appendix A. where we fix noise in each layer of the model independently and then evaluate the performance of the model on test and memorized inputs. We also experiment with the insertion of clean middle layers/ layers/blocks blocks in the noisy model via two stitching layers to study the sole influence of middle layers on the model's functional behavior in Appendix A.4 We once again observe that training and fixing noisc in carly layers do result in substantial improvement in generalization and suppresses memorization, corroborating that noise memorization starts early in the model. Furthermore, we show the dominant role of deeper layers for perfect memorization, and the influence of architecture bias and datasct complexity on the extent of memorization across layers. In the future, we plan to extend our work to transformer- basdod based VLMs to uncover memorization dynamics between transformer layers and leverage our findings to develop novel methods to mitigate memorization.
12

---

Published in Transactions on Machine Learning Research (MM/YYY)
References
Sotiris Anagnostidis, Gregor Bachmann, Lorenzo Noci, and Thomas Hofmann. The curious case of benign memorization. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.nen net, 2023. URL https////pppppeeeeeeeee/////raaa wttt://orrrmmdddcCccyYY
Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj. Asja Fischer, Aaron C. Courville, Yoshua Bengio, and Simon Lacoste-Julien. A closer look at memorization in deep networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference Machine Learning, 1CML 2017, Sydney, NSW, Australio, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 233-242. PMLR, 2017. URL html
http://proceedinnn http://ppooodnggg......p.o/o///s wttppp/s/////rrpt////.../
Andrés Balogh and Márk Jelasity. On the functional similarity of robust and non-robust neural representations. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 1614 1635. PMLR, 23 29 Jul 2023. URL https;
//proceedings. mlr. pttpss/v22//aalgg222000 html
Yamini Bansal, Preetum Nakkiran, and Boaz Barak. Revisiting model stitching to compare nen- ral representations. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advonces in Neural Information Processing Systems 34: Annuual Conference on Newral Information Processing Systems 2021, NewrIPS 2021, December 6- 14. 2021, virtual, pp. 225-236, 2021. URL htttps://proceedings.oo http://iw...rrrbpss.ppa/peee/eee 2t////2222hhhahaaaa 3ice/25be10fee3996633300bebbbbb
html
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Membership inference attacks from first principles. In 43rd IEEE Symposium on Securily and Privacy, SP 2022, Son Francisco, CA, USA, May 22-26, 2022, pp. 1897-1914. TEFE, 2022. doi: 10.100//SP421142222.88333 URL https://doi. org/ 10. /109/SP46291.1... 2022. 9833649
Sue Yeon Chung, Daniel D Lee, and Haim Sompolinsky. Classification and geometry of general percept.al manifolds. Physical Review X, 8(3):031003, 2018
Christian Cianfarani, Arjun Nitin Bhagoji, Vikash Sehwag, Ben Y. Zhao, Heather Zheng, and Prateek Mittal. Understanding robust learning through the lens of representation similarities. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022. URL http::/ppapee...nntt .nips. wwwpscc/pppppprrr....eppppppp ftt.://ppp../00222aaaaaarrrraa20 htt.://ae77eee-eottaaaa----eeeee
html
Adrián Csiszárik, Péter Korösi-Szabó, Ákos K. Matszangosz, Gergely Papp, and Dániel Varga. Similarity and matching of neural network representations. In Aurelio Ranzato, Alina Beygekimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Newral Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 5656 5668, 2021. URL httpss//ppooeeedingg www.iggs..u........ 2021 /hash/ 2202746fe540477888111116111111
Vitaly Feldman. Does learning require memorization? a short tale about a long tail. In Konstantin Makarychev, Yury Makarychev, Madhur Tulsiani, Gantam Kamath, and Julia Chuzhoy (eds.), Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, STOC 2020, Chicago, IL, USA, June 22-26, 2020, pp. 954-959. ACM, 2020. doi: 10. 00.1145/3357713333235
URL https: htt:::/dooi.ooooo orr/10
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long (ail via influence estimation. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (ods.), Advances in Neural Information Processing Systems 39: Annual Conference on Neu- ral Information Processing Systems 2020, NewrIPS 2020, December 12, 2020, virtual, 2020. URL hhtpps :
13

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Jonathan Frankle, David J. Schwab, and Ari S. Morcos. The early phase of neural network training. In 8th International Conference on Learning Represenlations, ICLR 2020, Addis Abala, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openrevierr httpps//pppaaeereeew.ereo/orrors
Haoyu He, Zizheng Pan, Jing Liu, Jianfei Cai, and Bohan Zhuang. Efficient stitchable task adaptation. In IEEE/CVF Conference on Computer Vision and Poltern Recognition, CVPR 2024, Seatlle, WA, USA, June 16-22, 2024, pp. 28555 28565. IEEE, 2024. doi: attps://don org/ 1109/CVPR52233..... 2024.02698)
URL
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Tn 2016 TEFE Conference on Computer Vision and Patlern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 770 778. IEEE Computer Society, 2016. doi: 10.1109/CCPR..206666 URL https://doi. org/ 10. 1109/CVPR. 2016.90)
Alex Krizhevsky, GeofTrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
David Krueger, Nicolas Ballas, Stanislaw Jastrzebski, Devansh Arpit, Maxinder S. Kanwal, Tegan Maharaj, Emmanuel Bengio, Asja Fischer, and Aaron C. Courville. Deep nets don't learn via memorization. In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, www./erummiiiiiiiee
2017, Workshop Track Proceedings. OpenReview.net, 2017. URL JJvZZgHYg
Klas Leino and Matt Fredrikson. Stolen memories: Leveraging model memorization for calibrated white- box membership inference. In Srdjan Capkun and Franziska Roesner (eds.), 29th USENIX Securily Symposium, USENIX Security 2020, August 12-14, 2020, pp. 1605-1622. USENIX Association, 2020. URL
https://wwil /wtpuu///wwuuusiiix.... htwpa///ww/./eeeeeeeeeeeeeeee//e
Karel Lenc and Andrea Vedaldi. Understanding image representations by measuring their equivariance and equivalence. In IEFE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pp. 991 999. IEEE Computer Society, 2015. doi: 10.1009/CVPR.2015.77555. URL https://doi orw.a/...111... 201.:/1/........111...
Sheng Liu, Jonathan Niles Weed, Narges Razavian, and Carlos Grannd.ez-Gaand. Farly-learning reg- ularization prevents memorization of noisy labels. In Hugo Larochelle, Marc' Aurclio Ranzato, Raía Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Pro- cessing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6-12, 2020, virtual, 2020. URL httpss:/prrcceedingg wewpreiiinnnnnuuns....e.p httpssccc/pppppe pwtppppp//////////////// hash// html
Hartmut Maennel, Ibrahim M. Alabdulmohsin, Ilya O. Tolstikhin, Robert J. N. Baldock, Olivier Bousquet, Sylvain Gelly, and Daniel Keysers. What do neural networks learn when trained with random labels? In Hugo Larochelle, Marc' Aurclio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Ticn Lin (eds.), Advances in Newral Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6- 12, 2020, virtual, 2020. URL http:: httpps//prroeeedings Abstract..ta.
neurips.cc/ wwtriisscccpppee///ee/ httpp/pppp///// 2tt///2222hhhhhaa (2d.://1818111137330666222222220
Pratyush Maini, Michael Curtis Mozer, Hanie Sedghi, Zachary Chase Lipton, J. Zico Kolter, and Chiyuan Zhang. Can neural network memorization be localized? In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 28-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 23536 23557. PMLR, 2023. URL https: sttps://procceedings mlr. press// v202/maini23a. html
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervisod feature learning. In NIPS workshop on deep learning and unsuperoised feature learning, volume 2011, pp. 4. Granada, 2011.
14

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Zizheng Pan, Jianfei Cai, and Bohan Zhuang. Stitchable neural networks. In IEEE/CVF Conference Computer Vision and Pallern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 16102 16112. LEEE, 2023. doi: 10.1109 BwB.0///VKR22222222...... CVPR52729 2023.01545 01545
URL
1109/
Zizheng Pan, Jing Liu, Haoyu He, Jianfei Cai, and Bohan Zhuang- Stitched vits are flexible vision backbones. In Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), Computer Vision ECCV 2024- 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part XLI, volume 15099 of Lecture Notes in Computer Science pp- 258 274. Springer, 2024. doi: 02.10779978--031--777
15. URL httpps///do. ottp:/001177797-----1----9-00990
Cory Stephenson, Suchismita Padhy, Abhinav Ganesh, Yue Hui, Hanlin Tang, and Sue Chung. On the geometry of generalization and memorization in deep neural networks. In 9th International Conference on Learning Repressentations, ICLR 2021, Virtual Event, Austria, May 9-7, 2021. OpenReview.net, 2021.
URL
Surat Teerapittayanon, Marcus Comiter, Bradley McDanel, and HT Kung. Stitchnet: Composing neural net- works from pre-trained fragments. In 2023 International Conference on Machine Learning and Applications (ICMLA), pp. 61 18- IEEE, 2023.
Jiaheng Wei, Yanjun Zhang, Leo Yu Zhang, Ming Ding, Chao Chen, Kok-Loong Ong, Jun Zhang, and Yang Xiang. Memorization deep learning: A survey. ACM Comput. Sure., 58(4):98:1-96) 18:33,, 2026. doi: 10.1145/3769077 URL htttss///ooi http://660..rr//111111/////70000
Shelvia Wongso, Rohan Ghosh, and Mchul Motani. Using sliced mutual information to study memorization and generalization in deep neural networks. In Francisco J. R. Ruiz, Jennifer G. Dy, and Jan- Willem van de Meent (eds.), International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, volume 206 of Proceedings of Machine Learning Research, pp. 11608 11629. PMLR, 2023. URL https://proceeddings mlr pp06s//206// http:////wonss2.a.....m html
Zhengqi Xu, Ke Yuan, Huiqiong Wang, Yong Wang, Mingli Song, and Jie Song. Training- free pretrained model merging. In IEEE/CVE Conference on Computer Vision and Pattern Recogreition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pp. URL cictps://doi.org// wwt..//gg//...///...... 10t.1/1/////P/3333... 2024.00565
IEEE, 2024. doi: 0t00///CVPR3223.2224..0000
Xingyi Yang, Daquan Zhou, Songhua Liu, Jingwen Ye, and Xinchao Wang. Deep model reassembly. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neurol Informa- tion Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. URL htttp:/pppperrs..oo.m .ips. cc/ Conferenctoooeeeeeeeeeeeeeoe htmll
Chiyuan Zhang, Samy Bongio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, Prance, April 24-26, 2017, Conference Track Proceedings. OpenReview.nee 2017. URL
https://openreviev..o http:///wwwwe.eetooooorrr/////ge
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Michael C. Mozer, and Yoram Singer. Identity crisis: Mem- orization and generalization under extreme overparameterization.. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview net, 2020.
URL http:////pppnrrrreeeeeeeoooononn
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Commeun ACM, 64(3):107 115, 2021. doi: 10.1145/3446776. URL https: //w.i.//oooooo www..oogg/.o.. 114...14111134166
15

---

Published in Transactions on Machine Learning Research (MM/YYY)
A Appendix
A.1 Retraining each layer of the model
So far, we assumed access to noise-free clean model and compared the functional similarity of its layers to those of the noisy model. However, in practical setting, it is more feasible to correct layers/blocks of the model by retraining than to replace them with those of the noisc-free clean model. So, we raise key question: "What would be the impact of in-place lixin//rduucig noise in each layer/block of the noisy model on its performance, and whether observed findings would be similar to those of stitching noise-free layers" However, answering this question needs a method to fix layer-wise noise in the model, which itself is challenging and beyond the scope of our work. As a result, we adopt the popular approach of finc-tuning cach layer on a dataset Dyz and evaluate the test and memorized accuracy of the end model.
We use the validation dataset Deat the fine-tuning dataset Dfc for each layer, resulting in weight updates to learn more generalizable features. This way, we can assume reduced noise in the fine-tuned layer when trained on Drat- So, after fine-tuning each layer independently (while freezing others), the model's performance is evaluated on the unseen test dataset Drest and memorized imputs D. We experiment on the Res.ett-18 model trained on multiple noisy variants of CIFAR-10 with noise ratio r and report our findings in Figure G
The observations can be summarized
Retraining the very first block on D, improves test accuracy and starts predicting correct labels to memorized inputs, and the increase in accuracy is proportional to the noise ratio r. This corroborates that noise impacts early layers, and its influence is directly proportional to the underlying noise ratio r.
As the depth of the retrained layer increases, more memorized imputs are correctly predicted. However, we do not observe more than 80% accuracy on memorized inputs, showing limitations of fine-tuning in mitigating underlying noise and the limited corrective power of each layer.
Next, we repeat the above experiment using memorized inputs but with their true labels as the fine-tuning dataset, i.e., Dyz = D.. The fine-tuning will directly interfere with the learned noise, providing the right update direction for weights to forget. random labels and correctly predict ground truth labels on the memorized inputs, resulting in reduced noise. We again train ResNet-18 on varied noisy variants of the CIFAR-10 datasct and report their training and test accuracy Figure i The experiment reveals that:
The fine-tuning of the first block can map at least 70% of previously randomly labeled inputs to correct labels, despite no change in the rest of the model. Moreover, the accuracy with respect to true labels increases with the depth of the fine-tuned layer.
Fino-tuning layer3.0 0 or any layer after it achieves 100% accuracy on memorized inputs. Notably, 1ayer3.0 is also the minimum clean suffix needed to mitigate memorization completely (see Figure 2
Fine-tuning with correct labels of memorized input shows degradation in test accuracy for all noise ratios r < 8000. However, for large noise, the test accuracy improves drastically after fino-tuning the first layer and increases with an increase in depth of the fine-tuned layer, except for the last residual block.
We hypothesize that when a layer is fine-tuned with Dr, the gradient updates weights to forget memorized inputs and learn their true class label, mossing up with existing generalized features. Morcover, during training on DR, the quality of learned generalized feat ures is better if the noise ratio is small. That's 's why, for small noise ratios, fine-tuning early layers can learn generalized features that can negatively impact previously learned generalized fcatures, while for large noise ratios, the quality of generalized features was already poor, and fino-tuning can partially improve them. This explains the drop in test accuracy for small r and continuous improvement for large
16

---

Published in Transactions on Machine Learning Research (MM/YYN
Test dataset
Correctly labeled noisy inputs
Accuracy on DX w.r.t true labels
100
n_noisy
90
2000
85
80
6000
Test Accuracy (%)
8000
10000
80
15000
60
20300
75
n_neisy
2000
70
6000
8000
20
10000
15000
20000
tayer2. 00
byerr.1
layar.0
layer2.1
laye3..
layer.11
layer4.0
layer..1
byer1.0 0
lyeer
layar..0
layer2.1
laye3.00
lyee3.1
lyyer.00
layerd.1
Pinctuning on validation dataset
Test dataset
Correctly labeled noisy inputs
Accuracy on D. w. w.r.f true labels
100
90
85
80
Test Accuracy (%)
n_nolsy
80
2000
60
6000
n_noisy
8000
2000
10
70
4000
15000
6000
20000
8000
65
10000
20
60
20000
layer1.0 0
layer1.1
layer2.0 0
layer2.1
layer3.0
layer3.1 1
layesk.
layor4.1
layor10 0
layer1.1
layer2.0
layer..1 1
layer3
layer
layers. 0
layord
baye_name nome
byer,
lyyer_aam nome
hayer
(b) Finctuning Dr with correct labels
Figure 6: ResNet-18 on CIFAR-10: The first row shows the performance of the model whose layers are independently trained on the validation dataset, and the second row corresponds to model layers being finetuned on the correctly labeled memorized inputs. In each subplot, all the residual blocks to be finetuned independently are listed on the x-axis while the model's accuracy (with exactly one finetuned block) is plotted on the y-axis. The first column shows the performance the model with finetuned layers on the unseen dataset, and the second column corresponds to the model's performance D with respect to their true class labels.
A.2 Layer-wise study of memorization in very deep networks
We apply stitching to ResNet-101 to study the layer-wise memorization and generalization in very deep networks. ResNet-101 comprises of 33 residual blocks organized into four stages {layer1, layer2, layer3, layer4) with layer3 consisting of 23 residual blocks. To probe layer-wise effects at different depths, we perform stitching at representative locations: the first residual block of each stage (layer1.0 layer2.0, layer3.0 0, layer4.0) and the final residual block (1ayer4. 2). The results of stitching are shown in Figure 7 Despite (layer1, layer2} forming small fraction of the total network depth, stitching clean prefix up
17

---

Published in Transactions on Machine Learning Research (MM/YYY)
Memorized Accuracy (%)
(%)
10.0
75
25.0
Test Accuracy (%)
60
65
60
40-
55
20
25.0
45
g
0300jer3.00
9alayar4..05
Slayar4,225
nony
Sisayrrr.00 1.00
9layerr2000
9
Slayerra.ge
end- clean
start
Stitching depth/
Stitching depth
(a) Clean-to-Non -oisy Stitching: Memorized accuracy
(b) Clean-to-Nols Stitching: Test accuracy
Memorized Accuracy (%)
(1%
10.0
75
25.0
Test Accuracy (%)
60-
65
60
40-
55
(19)
20
25.0
0siayer1.00 00
gsloyor3.00
SlayorA,220
Slayer1..00 .0
#layer2.00m
9ilayer3.006
Sibayer4.00
Slayert.gg
nolsy
start
Stitching depth
Stitching depth
(c) Noisy-to-
Nolsy-to-Clean Clean Stitching: Memorized accuracy
(d) Kolay-to-Clean Stitching: Test accuracy
Figure 7: ResNet-101 trained on CIFAR-100.
layer3. climinates more than 50% memorization for large noise ratios. fact, the of reduction in memorized accuracy slows down we stitch beyond layer3. O. Conversely, the emergence of memorization in noisy-to-clean stitched models delayed until stitch the noisy prefix up to the last few residual blocks. In fact, the coarse stitching behavior, both clean-to-noisy and noisy-to-clean, of ResNet-101 seems to resemble that of ResNet-34 and ResNet-50 trained on the CIFAR-100 100 dataset. The observed functional behavior of stitched models (evaluated as memorized and test accuracy) is consistent with all our findings.
A.3 Layer-wise study of memorization in Transformer model
In this paper, analysis is intentionally scoped to convolutional and residual architectures, where the notion of locality and hierarchical feature reuse allows controlled investigation of memorization mechanisms. To investigate whether the obsorvations also generalize to other architectures, conduct preliminary experiments on Vision Transformers (VIT). Specifically, we train a ViT-Small architecture on CIFAR-100 with and without random noise. We then stitch clean model (r = 0) to noisy model (r e (25% 50%}) after each transformer block, considering both clean-to-noisy and noisy-to-clean dircctions. The stitching layer trained on the validation dataset, and evaluation of stitched models on the unseen test dataset and memorized randomly labeled inputs is plotted in Figure The key observations are:
18

---

Published in Transactions on Machine Learning Research (M//YYY)
Memorized Accuracy (%)
100
75
80-
Test Accuracy (%)
70
65
60
60
25.0
55
40-
50
20
45
(I50
25.0
40
98block.255
9
95blocks.40
9
0.bbocks.0s
gatlocks.1-
Usbboko.25
9 35-
0 ON
9
Stitching depth/
Stitching depth
(a) Clean-to-Nolky Stitching: Memorized accuracy
(b) Clean-to-Nola tolay Stitching: Test accuracy
Memorized Accuracy (%)
100
75
80
Test Accuracy (%)
60
25.0
60
40-
55
20-
(156)
50
25.0
98blokk. 2-
"bbckks.. 6-
9atlocks.77
U5booko.256
9
ESSROCKC.ON ON
Stitching depth
Stitching depth
(c) Nolsy-to- Polsy-to-Clean Stitching: Memorized accuracy
(d) Noisy-to-Clean Stitching: Test accuracy
Figure 8: Vision Transformer (ViT-s) trained on CIFAR-100. The dotted lines in the first column show the memorized accuracy noisy models. The dotted lines in the second column show the test accuracy of noisy models. The black dashed line shows the memorized and test accuracy of the clean model.
Clean-to-Noisy Stitching: Stitching a clean base to the noisy model reduces memorization at all stitching depths except blocks. 3, where a sudden, small spike in memorized accuracy is observed. This behavior broadly consistent with our earlier findings M.T. Furthermore, stitching a sufficient. mumber of initial clean transformer blocks the noisy model improves its test performance, In this case, stitching layers of clean model at least up blocks. 2 improves the test accuracy of the noisy model for both noise ratios (r 25% and r 50%). Also, once the clean model is stitched upto blocks. 6, the test performance of the stitched model matches that of the clean model irrespective of noisy head layers and noise percentage r. This is consistent with observed findings G.1 and G.2
Noisy-to-Clean Stitching: As early noisy layers stitched into the latter clean layers, the test accuracy starts decrcasing. For a smaller noise ratio (r = 25%), the drop in test performance is gradual, while sharp decrease is observed for a larger noise ratio (r = 50%). However, irrespective of the stitching depth (i.e., the number of noisy blocks stitched into the clean model), the test accuracy of the stitched model always significantly better than that of its noisy counterpart. This aligns with the well-known fact fixing the last few layers of the noisy model improve its test performance. Interestingly, the memorization emerges only when a significant number of initial noisy blocks are
19

---

Published in Transactions on Machine Learning Research (MM/YYY)
ResNet-18 trained an CIFAR-10: Inserting middle clean layer in the naisy model
90
5.0
100
50-
1500
Test Accuracy (8g)
85-
Memorized Accuracy
20.0
25.0
40
50.0
80
5.0
75
20
70
1iyer111
1ayer2.0
1ayer2.1
layer3.
layer3.1
layer4,0
layerd.1
layer1.1
layer2.0
layer2-1
1ayer3.
layer3.1
layerd.
leyerd. 1
Name Inserted clean block noisy model
Name Inserted block noisy model
(a) Single clean middle block
ResNet-18 18 trained on CIFAR 10: Insertion of multiple middle clean layers
45
85
leya1.2 to 1yye22
40
koar2 layat.
kysr2 layak.3
Test Accuracy (5g
Memorizzd Accuracy
35
75-
70-
25
65
20
60-
15-
5.0
10.0
15.0
20.0
25.0
50.0
5.0
10.0
15.0
20.0
25.0
50.0
Noise percentage
Noise percentager
(b) Multiple clean middle blocks
Figure 9: Inserting clean middle blocks in the noisy model. First row corresponds to insertion of single clean middle blocks, and the second row corresponds to insertion of multiple clean middle blocks in the noisy model.
stitched into the clean model, and perfect memorization is never witnessed even when all the layers except the last output layer are noisy, consistently aligning with our main findings M.4 and M.5
Despite these qualitative similarities, VIT stitching also exhibits deviations from observed patterns in CNN and ResNets. For instance, the sudden increase in memorized and test accuracy when the clean model is stitched to the noisy model at blocks. 2 (subplots (a) and (b) in Figure g Similarly, the test accuracy of the noisy-to-stitched model for large noise ratio = 50% drops sharply with stitching in early noisy blocks and then fluctuates as more noisy blocks are stitched in. These irregularities warrant further investigation to determine whether they stem from architectural differences or from limitations of the stitching mochanism itself, as, unlike convolutional models, stitching in ViTs applies loken-wise linear transformation. Overall, our preliminary experiments suggest qualitatively similar trends with minor deviations, but a full analysis would require additional methodological development and is beyond the scope of the present work.
A.4 Insertion of middle clean layers in the noisy model
Model stitching stitches a clean prefix to the noisy model, fixing noise in all the layers of the prefix. As a result, understanding the noise impact of each layer independently a subset of layers other than the prefix is unaddressed. So, to delve further into noise memorization by each layer, replace the middle blocks of the noisy model with their clean counterparts. This is achieved by inserting two stitching layers, i.c., first performing noisy-to-clean stitching at layer I followed by stitching of the stitched model to the noisy model at layer (12), resulting in swapping out of noisy layers ((4 + .1....,22), by their clean counterparts. We carry out experiments on ResNet-18 trainod on CIFAR-10 for all noise ratios by a) inserting one middle clean block in the noisy model and b) inserting a consecutive subset of clean middle blocks the noisy model. ResNet- -8 consists of eight residual blocks, with seven blocks that can be considered "middle" blocks, leading to many
20

---

Published in Transactions on Machine Learning Research (MM/YYYY)
r" (%)
layer
layer1.1
layer2.0
layer2.1
layer3.0
1ayer3.1
layer4.0
layer4.1
5.0
48
62
113
206
237
432
497
10.0
49
62
110
113
206
244
431
494
15.0
55
60
98
113
184
233
391
486
20.0
106
201
240
355
473
25.0
50
62
108
113
188
224
434
484
37.5
244
369
476
50.0
49
60
95
113
165
229
259
468
Maximum rank
64
64
128
128
256
256
512
512
Table 2: Noisy-to-clean stitching: Rank of stitching layers
r. (%)
layer1.0
layer1.1
layer2.0
layer2.1
layer3.0
layer3.1
layer4.0 0
layer4.1
5.0
59
61
95
114
188
236
127
496
10.0
59
61
95
114
188
244
427
498
15.0
59
59
95
114
188
233
403
496
20.0
59
188
239
363
488
25.0
59
63
95
114
188
223
427
497
37.5
244
378
493
50.0
59
59
95
114
188
228
270
485
Maximum rank
64
64
128
128
256
256
512
512
Table 3: Clean-to-Noisy stitching: Rank of stitching layers
possible subsets. To keep the study tractable, we focus on insertion of two consecutive subsets of clean middle blocks [layer1.1, layer2.0, 0, 1ayer2.13 (implemented by inserting stitching layers at layer1.0 and layer2. 1) and (layer3.0, layer3.1, layer4.1, layer 4.2) (implemented by inserting stitching layers layer2. and layer4.2). 2). The functional behavior of the noisy model after insertion is shown in Figure We observe that inserting clean middle blocks leads to more pronounced reduction in memorized accuracy compared to replacing only early layers, consistently across all noise ratios Moreover, inserting larger consecutive subsets of middle blocks (three or four blocks) further mitigates memorization. However, a key distinction between stitching clean middle blocks vs clean-prefix stitching lies in generalization. Contrary to preff-prefix stitching, none of the explored insertions has resulted in the test accuracy similar to that of the clean model for any noise ratio. While this suggests that fixing middle layers alone may be insufficient fully recover generalization, we refrain from drawing strong conclusions, a comprehensive evaluation would require exploring large number of possible block combinations. We leave such exhaustive combinatorial analysis to future work.
A.5 Analysing stitching layers
In our experimental study, the stitching layer connects the memorization-heavy component of the noisy model to the downstream layers of the noise-free clean model and vice versa. In such a scenario, the task-loss-based optimization can encourage discarding of noisy features that might be detrimental to the downstream task performance. As result, even without any explicit regularizer enforcing low-rank, the task-based optimization result in non-invertible linear transformation if doing maximizes the functional alignment between representations (marked by converging loss during training). So, to experimentally corroborate our hypothesis, we have computed the rank of the stitching layer (considering various depths across the network where the stilching layer could be inserted). Consistent with our expectations, the stitching layer indeed lurns out to be a rank-deficient matrix across multiple stitching depths and noise levels. The Tables 2 and & report the rank of all stitching layers of the ResNet-18 model trained on CIFAR-10. The rows (excopt the last.) correspond noise level the training dataset, and the columns correspond the stitching depth layers at which stitching is performed. The last shows the maximum possible rank (size) of the matrix determined by
21

---

Published in Transactions on Machine Learning Research (MM/YYYY)
the layer dimensionality. This result demonstrates that rank-deficiency emerges naturally in stitching layers during training without any explicit regularization.
A.6 Training plots of stitching layers
The stitching layer S is trained by minimizing the cross-entropy loss of the stitched model on the held-out validation dataset. The training proceeds until loss converges, at which point the representations of the two models are maximally aligned in order to achieve the best-possible performance of the stitched model on its training (i.e., validation) dataset. The training convergence of stitching layers, both for clean-to-noisy and noisy-to-clean stitching, shown in Figure 10
and 15
22

---

Published in Transactions on Machine Learning Research
ConvNet trained
Training convergence clean-to-nolsy stitch ed models (S"g,
Stitching at conv1
Stitching at conv2
Stitching at conv3
2000
2000
2200
1.2
6200
1.0
6300
1.0
15200
0.0.8
15000
0.5
15000
50000
33000
32000
0.6
0.4
20
46
60
80
100
120
140
50
160
150
200
25
50
75
100
125
1500
175
Epachs
Epachs
Epachs
Stitching at
Stitching at
Stitching g at
0.5
1.2
9020
12000
9320
12000
9320
12000
15:00
15000
15000
0.6
0.3
0.22
0.21
(a) CNN model trained SVHN
ResNet-18 trained CIFAR-10 dataset: Training convergence of clean-to-noisy stitched models h, (("(((""")
Stitching at layer1.0
Stitching at aayer111
Sttching layer2.
2000
2300
2303
6100
0300
0303
1.2
1.4
13500
1000
1.25
13000
13500
1.0
15920
19630
20000
1.00
23000
23030
0.75
0.50
0.4
20
100
120
140
160
25
50
75
100
125
150
175
200
50
100
150
200
250
300
Epochs
Epochs
Epochs
Stitching layer2.1
Stitching layer3.0
Sutching layer33.1
2.00
1.2
600
6100
6303
2000
5200
800:
L055
13000
1.25
13000
15200
13000
0.8
0.6
0.75
0.50
50
100
150
100
200
150
200
Stitching at layer4.0
Stitching at layer4,1
0.40
2000
2000
0.35
0.230
Lo23
0.30
1000
13000
15000
20000
0.220
15000
23000
0.235
0.230
0.20
0.205
20
1.0
120
140
20
60
80
100
Epochs
Epochs
(b) ResNet-18 trained on CIFAR-10
Figure 10: Training convergence of stitching layers in clean-to-noisy stitched models
23

---

Published in Transactions on Machine Learning Research
ResNet-34 trained on CIFAR-10 dataset: Training convergence of clean-to-nolsy stitched models h, (S"
Stitching loyer.o
Stitching at layar1.1
Stitching at loyer2.0
2.00
2010
2010
2003
1.75
2.00
1.50
0910
0050
1.50
$003
15:00
10100
1.50
19200
1520c
0
1.25
15030
20920
1.00
20100
1.00
20100
20070
0.75
0.50
0.50
100
175
Epochs
Epochs
Epochs
Stitching 1ayer.11
Stitching loyara.o
Stitching
1.75
6020
4020
4020
6920
2000
1.2
1.4
600:
4002
1.50
8010
1.
8050
1.0
08
0.8
06
0.6
0.75
0.50
100
125
150
175
200
150
230
150
150
200
230
300
Epochs
Epochs
Epochs
Stitching loyer4.0
Stitching at layar4.1
2020
0.24
2020
0.25
0010
0050
1055
15300
10100
19200
15200
20300
20100
0.15
Epochs
Epochs
ResNet-34 trained CIFAR-10
ResNet-34 trained on CIFAR-100 dataset: Training convergence of clean-to-nolsy stitched models (S"g" "
Stitching layerl.a
Stitching layer1.1
Stitching at layer2.0
2.75
4000
2009
3.0
4000
2000
4500
2000
2.50
2.25
6400
2.5
2.25
2.00
20002
15003
2.0
20002
15603
2.00
20003
15000
1.50
1.50
1.55
20
40
60
80
100
120
140
25
50
75
100
125
150
175
25
50
75
100
125
150
175
Epochs
Epochs
Epochs
Stitching layer2.1
Stitching layer3.0
Stitching at layar3.1
2.50
1500
2.25
2.25
1000
2.00
100
8200
2.00
2000:
20002
15003
25000
15003
1.75
1.50
1.50
1.50
1.25
1.25
1.25
1.00
Epochs
Stitching layar4,0
Stitching at layar4.1
1.4
2000
2000
1000
1500
5000
1O05
3500
15003
20002
0.6
20002
0.4
6.4
0.2
0.2
50
100
150
200 Epochs
250
300
350
400
30
100
150
200 Epochs
250
300
350
400
(b) ResNet-34 trained on CIFAR-100
Figure 11: Training convergence of stitching layers in clean-to-noisy stitched models
24

---

Published in Transactions on Machine Learning Research
ResNet-50 trained on CIFAR-100 dataset: Training convergence of clean-to-noisy stitched models h, (
Stithhing at layer1.
Stithhing at l
Selthing layer
2.5
2033
2.5
200
2000
Loz5
2.0
2.0 2.0
2.
10392
1092
1000
1.5
15990
1.5
1309
15
13692
20000
2000
20002
1.0
20
60
1.00
120
20
60
100
60
80
100
120
Epochs
Epochs
Epochs
Stitching at 1oyer2.5
Stithhing at layer2.1
Stitching at layor2.2
2020
2.5
2000
2.5
2.0
0050
2.0
0060
10095
1009
1.5
30002
15290
1300
13407
1.0
20003
25
59
75
100
125
150
175
50
300
150
200
0
50
1.00
150
200
250
Epochs
Epochs
Epochs
Stitching layer2.3
Stitching at layor3.
Stitching layor3.1
2.5
2.5
4050
2.0
0003
2.0
2.0
0000
0020
0000
0000
1.5
0000
10230
1900
1.0
1000
15002
0.5
0.5
50
300
200
250
50
100
150
200
250
500
350
400
50
100
150
200 Epochs
250
300
350
Epochs
Epochs
layor3.2
Stithhing
Stitching layer3.4
2033
2.0
2000
4920
4000
6000
1.5
0000
1.5
0000
8020
1.0
000
31.0
0000
0.5
Epochs
Süitching layer3
Sütchhing at layer4.
Sütchingg layer4.1
2.0
2000
1.5
2000
2000
6020
6000
6000
6020
9939
000
52 0.75
0.50
20202
20002
100
250
300
400
0.0
50
100
200
350
400
50
150
200
350
Epochs
Sükchingg at layer4.2
2000
1020
1.2
10502
15005
2000:
0.5
50
100
150
200
Epochs
Figure 12: ResNet-50 trained on CIFAR-100: Training convergence of stitching layers in clean-to-noisy stitched models
25

---

Published in Transactions on Machine Learning Research
ConvNet trained
Training convergence noisy-to-clent stitch ed models hg (Slg,"
Stitchnng at convl
Stitching al conv2
Stitching at conv3
0.26
5020
3022
6010
0.28
1003
2002
3003
0.40
200:
0.24
0.24
15200
15032
75020
20200
20031
20020
0.22
0.20
0.25
0.20
0.18
0.20
20
40
60
8C
100
120
20
40
60
80
10c
120
140
25
50
75
360
125
150
175
Epochs
Epochs
Epochs
Sütthing
Stitching fc2
Stitching at
0.60
0.5
5020
5003
2002
15
12500
0.4
22030
0.50
12020
15300
:500:
25020
30
0.3
0.40
0.2
0.30
0.1
(a) CNN model trained SVHN
ResNet-18 trained CIFAR-10 dataset: Training convergence of noisy-to-clean stitched models he ((((()))"))
Stitchnng at 1
Sutching layer11
Sttching at layer2.0
2010
2010
2000
0.55
6032
2.00
8010
0.45
4000
0.45
10350
8
1.50
10300
0.40
1000
0.40
15200
20300
1.25
20100
13200
0:35
20000
0.75
0.30
0.25
0.25
25
50
75
300
125
150
0
25
50
100
125
150
25
75
100
125
150
175
200
Epochs
Epochs
Ep Epochs
1ayer2211
Sutching atlaye33.0
Sbithhing
0.7
0.6
0.40
0930
603
0.6
0.5
5020
0050
0.5
2000
4000
6600
0.35
15320
10300
0.6
3200
10:00
6.4
0.3
0.25
125
150
200
150
300
150
200
Epachs
Stitching at layer4.0
Sutching layer4.1 1
2030
0.8
2010
0.7
Lo53
10350
10300
15200
15200
20320
20200
6.6
25
75
100 Epochs
125
150
200
20
80
100
Epochs
(b) ResNet-18 trained on CIFAR-10
Figure 13: Training convergence of stitching layers in noisy-to-clean stitched models
26

---

Published in Transactions on Machine Learning Research (MM/YYN
ResNet-34 trained on CIFAR-10 dataset: Training convergence of nolsy-to-clean stitched models hg (S(g,"))
Stichinng at loyer1.o
Stithing at loye11.1
Stitching at layer2.0
0.425
2003
0.45
2035
4230
0.36
0.36
2000
0.350
0.375
0003
10930
15030
0.40
0530
15920
15310
0.30
0.32
20000
35000
0.300
2000
20310
0.26
20000
0.250
0.22
60
120
Epochs
Epochs
Epechs
Stitching 1oyer2.1
Stitching at layera.o
Sittching baye3311
0.5
2032
0.325
0003
4002
1003
5230
8030
4032
0.5
5000
3 0.275
0.3
0.250
0.2
0.200
0.1
0.1
75
100
150
175
200
100
150
200
250
300
50
100
150
200
250
300
350
Epochs
Epochs
Epochs
Stichingg at loyor4.0
Stitching at loyar4.1
2035
0.7
0.6
5532
2093
0.5
10320
15350
0.4
3007
5002
20300
0.3
Epochs
Epochs
ResNet-34 trained CIFAR-10
ResNet-34 trained on CIFAR-100 dataset: Training convergence of nolsy-to-clean stitched models h (S(g," 1
Stitching layer1.o
Stttching at layer1.1
Stttching layer?2 .0
2000
2000
2000
4200
4100
4000
1.25
1.1
1.5
1.20
15000
15603
15003
1.35
20003
20003
20002
1.30
1.0
20
40
60
80
300
120
140
0
20
60
80
100
120
25
50
100
125
150
175
Epochs
Epechs
Epochs
Stitching layer2.1
Sütching layer3.0
Sottchig layer3.1
1.4
8500
6500
12
8200
5500
5000
3000
1.2
22000
10003
20003
35003
1500)
15003
25003
20007
Epachs
Epochs
Stitching at layler4.o
Sttching layar4.1
2200
2000
4100
4500
Loss
12003
15003
22003
20003
0.4
50
100
150
200 Epochs
250
300
350
400
0
30
100
150
200 Epochs
250
300
550
400
(b) ResNet-34 trained on CIFAR-100
Figure 14: Training convergence of stitching layers in noisy-to-clean stitched models
27

---

Published in Transactions on Machine Learning Research
ResNet-50 trained on CIFAR-100 dataset: Training convergence of naisy-to-clean stitched models ho (S(g,") Stithing at layer12
Stitching layer1.0
Sütching at layer1l
2300
2002
1.2
2000
4002
1.050
1.05
1.1
1.000
10500
1.00
16020
10003
10500
15020
13003
20000
20020
20003
20
40
60
100
100
120
100
Epochs
Epochs
Epochs
Sttching at loyar2.0 o
Stitciing at layer2.1
Stitching at 1ayer2.2
2.1
2002
1.1
1.0
1,00
0300
0.9
6602
3000
10000
30020
10003
15:00
15070
0.8
13003
20003
0
25
50
75
100
125
150
175
0
50
300
150
200
250
50
100
150
200
Epochs
Epochs
Epechs
Stitciing 1oyar2.3
Stitching at layor3.0
layer3.1
1.2
2.1
4350
4002
9200
0.8
5002
5000
0.9
8350
0.6
600
3000
13200
1002
20007
15200
15003
0.7
0.2
100
150
200
250
300
50
100
150
200
250
300
350 400
50
100
150
200
250
300
350
Epochs
Epochs
Epechs
Stichhing loyer3.2
Stithinng layer3.3
1.25
1.25
2002
1.00
4200
1.00
4002
4000
0 0.75
6350
3 0.75
600
8 0.75
3000
8320
8000
5500
0.50
10007
Epochs
Süiching layer3.55
S0tching layer4.0
Stitching layerd.1
1.25
2000
2002
1.4
2000
8 0.75
4320
1.0
0002
4002
5007
1.2
4000
0.50
0.8
0.8
0.25
22000
20020
0.6
20000
0.00
50
100
200
250
50
250
400
100
250
300
350
Epochs
Epochs
Stitching laaer44.2
2020
1.0
12500
15500
0.8
20100
50
100
150
200
250
Epochs
Figure 15: ResNet-50 trained on CIFAR-100: Training convergence of stitching layers in noisy-to-clean stitched models
28


Reviews:
[{"review_id": "rf5EicCiMe", "reviewer": "Reviewer_vyRE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the extent of memorization (mem) and generalization (gen) in CNNs for image classification tasks with datasets like CIFAR and SVHN. Specifically, it studies the effect of different kinds of stitched models on the mem and gen performance, where the stitching happens between a noisy base and a clean head or vice-versa. The paper conducts several experiments to study the effect of the depth of the base on the mem and gen performance in different CNN arcitectures--with and without skip connections, and reports the observed trends. The trends appear to be consistent between the degree of generalization and that of memorization, i.e., configurations that lead to better gen lead to worse mem and vice-versa. Furthermore, the experiments indicate that the extent of gen and mem for the same depth depends on the architecture and the dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper conducts rigorous eperiments in support of its claims. Specifically, to test both memorization (mem) and generalization (gen) it experiments with different architectures and datasets. The observed trends seem to be consistent across datasets and architectures. This makes the observations well-justified.", "audience_yn": "Yes", "audience_explanation": "The paper provides useful analysis on how the degree of memorization (mem) and generalization (gen) are related to the layer count of the base and the head in stitched models with CNN architecture and simple image classification tasks with datasets like CIFAR and SVHN. This study could be valuable to the community for better understanding the mem and gen capabilities of CNNs for image classification.", "requested_changes": "Given the prevalence of transformer-based models in modern computer vision / AI and more complex tasks of interest, like question answering, generation, low-level vision, 3D vision, video understandinging, etc., I am seriously concerned about the usefulness of this work. The paper can be made stronger and more relevant by adding experiments and extending the study to cover the following aspects:\n1. other architectures like transformers, state space models\n2. more complex tasks like visual question answering, image/video generation, 3D scene reconstruction / novel view synthesis, object detection and segmentation, video understanding and captioning, or other tasks that are currently of interest in the vision community\n\nI am also curious how the study would extend to image classification with large-scale datasets like Imagenet.", "broader_impact_concerns": "Please see para 1 in \"Requested Changes\".", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7VdUSQABKH", "reviewer": "Reviewer_AK2u", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents an excellent work explaining the well-known puzzling behavior of deep neural networks: they can memorize incorrect labels in the training data but still perform well on new test data.\nThe authors use model stitching as an innovative way to identify where memorization occurs inside a neural network.\n\nStrengths\n\n1. I believe this is the first study to use model stitching to compare each layer in a clean model and a noisy model. It's a practical way to test layer-by-layer functional similarity.\n\n2. The authors provides solid testing results. They tested their method on several different models (CNN, ResNet-18/34/50) and datasets (SVHN, CIFAR-10/100), and it showed the same pattern each time.\n\n3. The authors report the following very important findings based on their solid and innovative experimental design:\n\na. The authors show that memorization is not limited to the deep layers but Early Layers also learn noise. When the noisy early layers is replaced by the clean layers, the model often stopped recalling the wrong random labels. This provides evidence that fixing the bottom of the network is an effective way to prevent noise from propagating through the model.\n\nb. The authors also show the importance of fixing the bottom layers as small errors/noise in the early layers get bigger and bigger as they move up.\n\nWeaknesses\n\n1. The experiments only check vision models (CNNs and ResNets) for classification tasks. They do not test text-based models or Transformers. But LLMs and Transformers are very popular today.\n\n2. The experiments assume already have a perfect clean model to swap layers. But a perfect noise-free model is probably not available in real-world scenarios. \n\n3. In Appendix A.1, the authors admit that swapping layers is difficult in the real world because rarely have a perfect clean model as a replacement. Therefore they tried to fix the noise by retraining the specific layers instead of stitching in new ones. But they find that retraining the specific layers is hard to do well compared to swapping it out as authors mentioned \"answering this question needs a method to fix layer-wise noise in the model, which itself is challenging and beyond the scope of our work.\" and \"However,\nwe do not observe more than 80% accuracy on memorized inputs, showing limitations of fine-tuning\nin correcting underlying noise.\" So overall, the paper successfully identifies the root cause of memorization, but it does not provide a practical solution for fixing it without a perfect clean reference model.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors provides solid testing results. They tested their method on several different models (CNN, ResNet-18/34/50) and datasets (SVHN, CIFAR-10/100), and it showed the same pattern each time.", "audience_yn": "Yes", "audience_explanation": "The findings of this paper would be of interest to the TMLR audience focused on deep learning, interpretability, and robust learning.", "requested_changes": "The paper already provides solid analysis and evidence regarding the layer-wise dynamics of memorization. It's better for the authors to address and explain more on how to resolve the limitations mentioned in the Weaknesses 1., 2., 3.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NROpv3EI85", "reviewer": "Reviewer_AVA2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the DNNs ability to generalize and memorize the training dataset using lens of functional representations similarity. Specifically, the authors apply layer-stitching method to better understand the (representational) differences and similarities of the models trained on the standard (noise-free) datasets and their noisy counterparts. \n\nKey strengths:\n- Important topic\n- Representational perspective offers a universal angle to look at the models, thus experiments could be easily applied to different scenarios (architectures, datasets, modalities)\n- Overall presentation and the scope of the work is well defined. The authors do not try to touch too many aspects at once) and clearly state the contributions of the work which are supported by the empirical evidence.\n\nWeaknesses:\n- Narrow experimental scope -- although the authors claim to use four different architectures and three different datasets, I would argue that using different sizes of ResNets shouldn't be considered as a different architecture. Regardless of the size (ResNet-18/34/50) these models share the core building blocks. This makes me wondering how broadly the findings would apply when considering size-homogenous models (like Transformers) or skipless-models like VGGs or MLPs.\n- Although the overall presentation of the material is rather clear, I find it difficult to follow the specific structure of the experimental section. I would suggest the authors to consider different presentation of the findings. In particular, I’d encourage the authors to focus on two cases (Noise -> Clean (evaluated both on Memorization and Generalization tasks) and Clean -> Noise evaluated the same way. This would give a better perspective on how stitching particular subsets of layers affect model’s performance). \n- The whole analysis relies only on the stitching approach. I think that at least partial control-experiments with some other types of functional similarity should be run to understand whether the findings are robust or just specific to stitching approach. \n- Some (minor) claims made by the authors remain speculative and need further justification or explanation to be clear (I will list them in the later sections).", "claims_and_evidence_yn": "Yes", "claims_explanation": "I voted yes, as the core claims are generally supported by the experiments (although the scope of the experiments is relatively narrow as I mentioned earlier).\n\nHowever, the paper has multiple sentences which are not clear to me and would require further elaboration. I expect that some of these claims might not be obviously true for the rest of the readers of the paper so I’d encourage the authors to elaborate a little bit on these things.\n\nIn several places (p. 3, p. 4, p. 5) the authors mention that stitching works by “aligning the representations”. Specifically, in p.4 the authors write that stitching works by“ (...) maximizing the alignment between latent representations of two models”. What do the authors actually mean by “alignment” here? And could the authors actually show that training the stitching layers leads to maximization of that alignment (however measured it is?). \n“We train the model on clean data without label noise so that none of its training inputs to be specifically memorized, and use it as a baseline for generalization.” While I understand the experimental design here and the need to have a reference of a clean dataset, I have to stress the fact that even the standard datasets with correct labels are not noise-free. It is a well-known fact that in most of the image classification datasets there is a clear hierarchy of learning easy to hard samples, where the hard ones are frequently ambiguous ones leading to a noise in the overall dataset. \nStitching vs. CKA -- the authors write that “Empirical evidence shows that even when two networks’ representations score low on metrics like CKA, they can still exhibit high functional similarity, demonstrating that representational and functional similarit y capture the complementary aspects of how representations relate.” The work cited by the authors indeed proposed an experiment (Section 6.3) where the representational similarity was decreased (around 0.7 CKA score) yet the accuracy of the stitched model remained unchanged. In this particular experiment, the authors regularized the stitching layer to decrease the CKA and keep the overall performance of the model unchanged -- such a scenario would likely never occur in real setting. Thus, in my opinion, this is not a sufficient evidence to argue that actually the functional similarity (measured with stitching) gives a different perspective than the one obtained with CKA or other types of representational similarity. Since the CKA/CCA metrics are invariant to reversible linear transformations, I would actually expect the stitching (through a linear layer) method and CKA/CCA scores to behave similarly. Such an analysis would definitely strengthen the work and help us better relate it to other findings studying the problem of memorization/generalization in a broader context.\n“(...) the stitching layer cannot align the divergent representations (...)” -- again what do you mean by “divergent” representations and how do you make sure that this in fact cannot happen (i.e. these representations cannot be aligned)?", "audience_yn": "Yes", "audience_explanation": "I believe so, the topic is still an important research direction with lots of questions unanswered and the work tries to offer novel perspective on the topic. At the same time, the limited experimental scope makes me doubt the the current form is ready for publishing the work.", "requested_changes": "Here are some minor details I would encourage the authors to apply to generally improve the readability of the work. At the same time I'd like to repeat my suggestions about restructuring the results section to ease the overall readability.\n\n\nFigure 1 is in poor quality -- upload this as a PDF instead of jpg.\n\nI believe that Figure 2 is inconsistent in terms of the placement of black vertical dashed lines. Per Figure caption these lines “(...) shows the minimum depth of clean base layer required to mitigate memorization in noise models” however in case of ResNet-34 on CIFAR-100 and ResNet-50 on CIFAR-100 the dashed lines are placed at layers which still have almost perfect memorization. Am I missing something? Actually the analogous thing happen in Figure 4 again for ResNet-34 and ResNet-50 trained on CIFAR-100.\n\nFigure 4 caption -- “(...) requiring for restoring (almost) full generalization in the noise model” -- What “almost” in this case means? These are the details the need precise treatment otherwise the reader has to guess them and could be easily mislead.\n\n\n\nI’m also a bit confused about the discrepancy between the differences of the trends presented in Figure 2 (ResNet34 trained on CIFAR-10 and CIFAR-100). The architecture is the same, however, the trends are radically different. Could you please elaborate on the source of these differences? Did you run any further analysis trying to understand it?\n\nFigure 3 -- captions of subfigures do not include the datasets.\n\n\nSection 5.3.3. -- fix the reference to the Figure/Section at the end of the page.\n\n\nCould you please elaborate a bit on why would you apply different strategy (e.g. LogitLens) in case of Transformers? Do you see any equivalence of stitching method and LogitLens or is it justified in any other way?", "broader_impact_concerns": "I have no ethical concerns about this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9Xrtr5f0ua", "reviewer": "Reviewer_FKUA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a method to investigate and localize the layers responsible for memorization in DNNs (specifically CNN, ResNet-18, and ResNet-34) via model stitching. The methodology relies on two distinct models: a \"clean\" model trained on a standard dataset and a \"noisy\" model trained on a dataset with randomly assigned labels. The authors explore two stitching directions: substituting clean layers into the noisy model (Clean-to-Noisy) and introducing noisy layers into the clean model (Noisy-to-Clean) .\n\nThrough these experiments, the authors provide a set of well-validated findings:\n- Early Layer Susceptibility: Contrary to the assumption that early layers learn robust features, the study finds they are infected by noise; however, replacing them with clean counterparts effectively mitigates memorization .\n- Asymmetry in Stitching: The authors demonstrate that the two stitching directions are asymmetric—cleaning early layers in a noisy model significantly restores performance, whereas introducing early noise into a clean model does not immediately degrade generalization .\n- Role of Model Head vs. Base: The results highlight that while early layers contribute to noise propagation, the deeper layers (head) play a dominant role and are necessary for achieving perfect memorization.\n\nThe paper validates these claims using standardized classification datasets (CIFAR-10, CIFAR-100, and SVHN). Overall, the proposed approach offers solid empirical evidence regarding layer-wise memorization behavior in classification tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "For each of the major observation provided in the paper, authors follow with a quantitative analysis on the classification datasets CIFAR-10, CIFAR-100, SVHN. Their approach is pretty standardized. Furthermore, the experimental design includes validations across architectures (CNN and ResNet) with different noise ratios (5% to 50%) for a robust evaluation protocol. The evidence is particularly convincing due to the careful choice of training the stitching layer on a held-out validation set, which prevents the model from simply learning the noise patterns(see the final paragraph in page 4). Additionally, the authors clearly support their findings through experiments in Appendix A.1 (layer-wise retraining), where each layer is trained with a different noise ratio. Overall, the proposed method is well-validated and convincing.", "audience_yn": "Yes", "audience_explanation": "The paper provides insights into one of the fundamental yet unresolved problems in deep learning: memorization of training data. Although the presented experiments involve relatively small datasets and models compared to the scale of current state-of-the-art systems, the observations provided are valuable. In particular, the identification that generalization and memorization are not symmetric behavior, demonstrated through the two-way stitching of models (between the clean and noisy models), is an important finding. Furthermore, I believe that the study offers broad insights that will benefit researchers attempting to solve the memorization problem, even if it is on scaled up models which still requires fundamental findings irrelevant from the model scale.", "requested_changes": "While the submission provides important insights on the memorization with the classification task, there are certain changes that I believe to be beneficial for the manuscript. Please find the suggested changes below:\n- **Evaluation of Intermediate Layer Intervals:** The current methodology relies exclusively on prefix-based stitching (e.g., replacing layers $0 \\dots l$ with clean counterparts). This design inherently biases the analysis towards the importance of preceding layers, as they are always the first to be cleaned in every configuration. To verify the claim that early layers are uniquely susceptible to noise, I believe experiments on distinct layer intervals, specifically, replacing intermediate blocks would be beneficial and would strengthen the argument on the significance of the preceding layers (e.g., keeping the initial layers noisy, cleaning the middle blocks, and keeping the head noisy). This is necessary to determine if the observed benefits stem specifically from fixing the early layers or simply from fixing any significant portion of the network.\n- **Validation on Larger Models:** While the experiments with ResNet-50 are appreciated, modern architectures often extend significantly deeper even for the classification task. From this point of view, I believe deeper architectures, including deeper backbones such as ResNet-101 or ResNet-152 and different architectures such as ResNeXT would be beneficial. It is important to verify if the \"early layer\" effect persists when the initial layers constitute a much smaller fraction of the total network depth, or if the noise becomes more distributed in significantly deeper networks.\n- **Generalization to more Complex Tasks (e.g., Object Detection):** The current findings are limited to relatively simple classification tasks on SVHN, CIFAR-10, and CIFAR-100. To demonstrate the broader applicability of these memorization dynamics, evaluations on more complex tasks, such as Object Detection (e.g., with COCO). Memorization dynamics may differ significantly when the model must predict bounding boxes rather than just class labels, and this would clarify if the observed asymmetry is a fundamental property of DNNs or specific to classification.", "broader_impact_concerns": "The proposed method focuses on investigating the memorization behavior on existing architectures with standardized datasets. Overall, the authors neither introduce a new model nor benchmark that may create a sensitive issue in terms of broader impact. However, since the proposed method can facilitate research on memorized sensitive content, any may possibly lead to unlearning approaches with a more careful block selection with the stitching methodology, it can even facilitate further research on existing broader impact concerns in the field.", "additional_comments": "The paper offers valuable insights into memorization dynamics over the classification task using a well-structured methodology, particularly the use of two-way model stitching between noisy and clean models. However, the reliance on prefix-based stitching introduces a potential bias that may overemphasize the role of early layers compared to intermediate ones. While I lean towards acceptance due to strong evidence presented for the technical findings, addressing the requested changes regarding interval-based evaluation, inclusion of backbones different in architecture and number of parameters, and broader task validation would increase confidence in the generalizability of the results.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)