Original: Roller et al., 2021;
Revised: Roller et al., 2021;

Original: Shazeer et al., 2017) that discuss the stale momentum problem and issues with momentum based optimizers.
Revised: Shazeer et al., 2017) that discuss the stale momentum problem and issues with momentum based optimizers.

Original: 15 We also could not find an academic citation for the sparse Adam implementation provided in PyTorch or the motivations for implementing it.
Revised: We also could not find an academic citation for the sparse Adam implementation provided in PyTorch or the motivations for implementing it.

Original: Suggestively, in TensorFlow the algorithm is called “LazyAdam” and advertised as being advantageous for computational efficiency reasons without any mention of dead neurons.
Revised: Suggestively, in TensorFlow the algorithm is called “LazyAdam” and advertised as being advantageous for computational efficiency reasons without any mention of dead neurons.

Original: This is unsurprising in light of the fact that the dead neuron problem is only an issue in the sparse regime and even in this case can go unnoticed if validation accuracy is the only metric of interest.
Revised: This is unsurprising in light of the fact that the dead neuron problem is only an issue in the sparse regime and even in this case can go unnoticed if validation accuracy is the only metric of interest.

Original: For example, training on CIFAR10 pixels with a ReLU network and Adam results in 95% dead neurons with minor effects to validation accuracy.
Revised: For example, training on CIFAR10 pixels with a ReLU network and Adam results in 95% dead neurons with minor effects to validation accuracy.

Original: E T RAINING R EGIME A BLATIONS

Original: [Figure]
Revised: [Figure]

Original: We remove the ConvMixer preprocessing and train models directly on ImageNet32 pixels before testing their continual learning abilities on Split CIFAR10 pixels.
Revised: We remove the ConvMixer preprocessing and train models directly on ImageNet32 pixels before testing their continual learning abilities on Split CIFAR10 pixels.

Original: This is a way to address concerns that the ConvMixer preprocessing was manipulating the CIFAR10 data manifold to make the continual learning task too easy.
Revised: This is a way to address concerns that the ConvMixer preprocessing was manipulating the CIFAR10 data manifold to make the continual learning task too easy.

Original: Figure 18 shows the results and Table 3.
Revised: Fig. 18 shows the results and Table 3.

Original: We only tested the best performing models with reference to Table 1 and chose EWC over MAS because it gives the best model overall and otherwise performs similarly.
Revised: We only tested the best performing models with reference to Table 1 and chose EWC over MAS because it gives the best model overall and otherwise performs similarly.

Original: We did not test the FlyModel because it assumes the presence of preprocessed latent embeddings rather than the 3072 dimensional CIFAR10 flattened image vectors.
Revised: We did not test the FlyModel because it assumes the presence of preprocessed latent embeddings rather than the 3072 dimensional CIFAR10 flattened image vectors.

Original: All training settings and parameters are kept the same as in the Table 1 tests.
Revised: All training settings and parameters are kept the same as in the Table 1 tests.

Original: Interestingly, all EWC methods fail to perform as well as they did before.
Revised: Interestingly, all EWC methods fail to perform as well as they did before.

Original: This is likely because we did not tune our loss coefficient or β parameter for training directly on pixels.
Revised: This is likely because we did not tune our loss coefficient or β parameter for training directly on pixels.

Original: As a result, the best performing model is now SDM with 10K neurons (yellow) instead of SDMLP+EWC (green).
Revised: As a result, the best performing model is now SDM with 10K neurons (yellow) instead of SDMLP+EWC (green).

Original: Also note that while SDMLP with k = 1 (pink) does worse, it fails to learn each task within the 2, epochs, e.g. at the end of 2,000 epochs the validation accuracy is ∼ 10% while the other methods maximize the accuracy within task of ∼ 20%.
Revised: Also note that while SDMLP with k = 1 (pink) does worse, it fails to learn each task within the 2, epochs, e.g. at the end of 2,000 epochs the validation accuracy is ∼ 10% while the other methods maximize the accuracy within task of ∼ 20%.

Original: This means that SDM with k = 1 could potentially do better with more epochs or a higher learning rate.
Revised: This means that SDM with k = 1 could potentially do better with more epochs or a higher learning rate.

Original: We use the same 5 random seeds and 2,000 epochs per task.
Revised: We use the same 5 random seeds and 2,000 epochs per task.

Original: Training takes longer because the images are 3x32x32=3072 pixels rather than the 256 ConvMixer embeddings and the most complicated model, SDMLP+EWC with 10K neurons fails to terminate in the 15 hours of GPU time allocated to each run but makes it to the last learning task where it’s performance can be inferrred.
Revised: Training takes longer because the images are 3x32x32=3072 pixels rather than the 256 ConvMixer embeddings and the most complicated model, SDMLP+EWC with 10K neurons fails to terminate in the 15 hours of GPU time allocated to each run but makes it to the last learning task where it’s performance can be inferrred.

Original: [Table]
Revised: [Table]

Original: The most competitive models, pretrained on ImageNet32 and tested on Split CIFAR10 without any
Revised: The most competitive models, pretrained on ImageNet32 and tested on Split CIFAR10 without any

Original: ConvMixer embedding.
Revised: ConvMixer embedding.

Original: SDMLP+EWC with 10K neurons has a * to denote that it did not finish training.
Revised: SDMLP+EWC with 10K neurons has a * to denote that it did not finish training.

Original: Average of 5 random task splits.
Revised: Average of 5 random task splits.

Original: [Figure]
Revised: [Figure]

Original: Figure 19 shows the results for the most competitive continual learning algorithms without any ImageNet32 pretraining and tested on the ConvMixer embedded Split CIFAR10.
Revised: Fig. 19 shows the results for the most competitive continual learning algorithms without any ImageNet32 pretraining and tested on the ConvMixer embedded Split CIFAR10.

Original: See Table 4 for the final validation results.
Revised: See Table 4 for the final validation results.

Original: We train for 2,000 epochs on each task and 10,000 epochs in total.
Revised: We train for 2,000 epochs on each task and 10,000 epochs in total.

Original: The GABA switch occurs well within the first task where s = 5 , 000 , 000 with GABA switching at epoch ∼ 50 and Top-K being fully implemented by epoch ∼ 100 .
Revised: The GABA switch occurs well within the first task where s = 5 , 000 , 000 with GABA switching at epoch ∼ 50 and Top-K being fully implemented by epoch ∼ 100 .

Original: Performances are lower than with the ImageNet32 pretraining, for example the best performing SDMLP+EWC is 84% versus the 86% in Table 1.
Revised: Performances are lower than with the ImageNet32 pretraining, for example the best performing SDMLP+EWC is 84% versus the 86% in Table 1.

Original: Most notable is the worse performance for the 1K neuron models that lack sufficient capacity to learn the data manifolds of new tasks without the benefits of pretraining
Revised: Most notable is the worse performance for the 1K neuron models that lack sufficient capacity to learn the data manifolds of new tasks without the benefits of pretraining

Original: [Table]
Revised: [Table]

Original: F A DDITIONAL D ATASETS F. CIFAR10 E XTRA F IGURE AND T ABLE
Revised: F A DDITIONAL D ATASETS F. CIFAR10 E XTRA F IG . AND T ABLE

Original: Table 5 shows the Split CIFAR10 results for the 1K neuron setting.
Revised: Table 5 shows the Split CIFAR10 results for the 1K neuron setting.

Original: Figure 20 shows the extent to which the SDMLP forgets previous tasks when compared to the baseline ReLU model.
Revised: Fig. 20 shows the extent to which the SDMLP forgets previous tasks when compared to the baseline ReLU model.

Original: This figure gives extra context to the headline continual learning results of Fig.3.
Revised: This figure gives extra context to the headline continual learning results of Fig.

Original: [Table]
Revised: [Table]

Original: F. CIFAR 
Revised: F. CIFAR

Original: To better assess the true memory capacity of our models, we use the same pretraining on ImageNet and then test continual learning on 50 splits of CIFAR100, shown in Fig.
Revised: To better assess the true memory capacity of our models, we use the same pretraining on ImageNet and then test continual learning on 50 splits of CIFAR100, shown in Fig.

Original: 21 and Table 6.
Revised: 21 and Table 6.

Original: We allow for 500 epochs per task and 25,000 epochs in total.
Revised: We allow for 500 epochs per task and 25,000 epochs in total.

Original: We used three random seeds instead of five, the same hyperparameters found for CIFAR10, and only tested the best performers from Table 1.
Revised: We used three random seeds instead of five, the same hyperparameters found for CIFAR10, and only tested the best performers from Table 1.

Original: It is with CIFAR100 that the SDM models including the FlyModel and SDMLP+EWC really shine over the regularization baselines.
Revised: It is with CIFAR100 that the SDM models including the FlyModel and SDMLP+EWC really shine over the regularization baselines.

Original: In the 1K neuron setting, SDMLP+EWC is the best performer with 42%.
Revised: In the 1K neuron setting, SDMLP+EWC is the best performer with 42%.

Original: Meanwhile, in the 10K setting the FlyModel takes the lead getting 58% and SDMLP+EWC behind it at 51% against the oracle of 72%.
Revised: Meanwhile, in the 10K setting the FlyModel takes the lead getting 58% and SDMLP+EWC behind it at 51% against the oracle of 72%.

Original: The next closest baseline is 40% for MAS.
Revised: The next closest baseline is 40% for MAS.

Original: The large performance jumps in the FlyModel going from 1K to 10K neurons for CIFAR10 and CIFAR highlight the importance of dimensionality for the fixed neuron addresses that perform a random projection.
Revised: The large performance jumps in the FlyModel going from 1K to 10K neurons for CIFAR10 and CIFAR highlight the importance of dimensionality for the fixed neuron addresses that perform a random projection.

Original: Interestingly, when looking at the validation accuracy on Task 1 for each of the models, which shows how it forgets the task over time, SDMLP+EWC performs better than the FlyModel, however the FlyModel retains later tasks better.
Revised: Interestingly, when looking at the validation accuracy on Task 1 for each of the models, which shows how it forgets the task over time, SDMLP+EWC performs better than the FlyModel, however the FlyModel retains later tasks better.

Original: [Table]
Revised: [Table]

Original: We bold the best performing model within the 1K and 10K neuron settings.
Revised: We bold the best performing model within the 1K and 10K neuron settings.

Original: Oracle was trained on the full CIFAR10 dataset.
Revised: Oracle was trained on the full CIFAR10 dataset.

Original: Average of 3 random task splits.
Revised: Average of 3 random task splits.

Original: F. S PLIT MNIST
Revised: F. S PLIT MNIST

Original: For the sake of completeness and providing an easier benchmark for other methods, we use the Split
Revised: For the sake of completeness and providing an easier benchmark for other methods, we use the Split

Original: MNIST dataset but keep the class incremental setting.
Revised: MNIST dataset but keep the class incremental setting.

Original: Results are presented in Table 7 and approximate the general rank ordering of performance whereby the FlyModel with 10K neurons does the best, SDMLP+EWC comes second, then SDMLP on its own and then the parameter importance regularization methods.
Revised: Results are presented in Table 7 and approximate the general rank ordering of performance whereby the FlyModel with 10K neurons does the best, SDMLP+EWC comes second, then SDMLP on its own and then the parameter importance regularization methods.

Original: We did extensive hyperparameter tuning of the EWC and MAS regularization coefficients and β parameter along with the FlyModel parameters (Appendix G.2).
Revised: We did extensive hyperparameter tuning of the EWC and MAS regularization coefficients and β parameter along with the FlyModel parameters (App. G.2).

Original: We do not use any pretraining and train directly on MNIST pixels.
Revised: We do not use any pretraining and train directly on MNIST pixels.

Original: We train for 500 epochs on each task meaning 2,500 epochs in total.
Revised: We train for 500 epochs on each task meaning 2,500 epochs in total.

Original: We use just three random seeds to initialize model weights but always use naive split where task 1 contains digits 0 and 1, task 2 contains digits 2 and 3, etc.
Revised: We use just three random seeds to initialize model weights but always use naive split where task 1 contains digits 0 and 1, task 2 contains digits 2 and 3, etc.

Original: As one of our benchmarks we used Active Dendrites (Iyer et al., 2022).
Revised: As one of our benchmarks we used Active Dendrites (Iyer et al., 2022).

Original: We used the code provided but found that it failed to generalize beyond the easier Permuted MNIST task incremental benchmark used in the paper.
Revised: We used the code provided but found that it failed to generalize beyond the easier Permuted MNIST task incremental benchmark used in the paper.

Original: NISPA has a * in the number of neurons column of Table 7 because it used three hidden layers of 400 neurons each as in its original implementation.
Revised: NISPA has a * in the number of neurons column of Table 7 because it used three hidden layers of 400 neurons each as in its original implementation.

Original: This is approximately the same number of parameters as the 1K neurons in a single layer (637600 vs 794000 ignoring bias terms).
Revised: This is approximately the same number of parameters as the 1K neurons in a single layer (637600 vs 794000 ignoring bias terms).

Original: [Table]
Revised: [Table]

Original: Models trained directly on MNIST pixels and without any pretraining.
Revised: Models trained directly on MNIST pixels and without any pretraining.

Original: We bold the best performing method.
Revised: We bold the best performing method.

Original: We run these results on three random seeds and using 500 epochs for each split (this is much higher than other baselines (Hsu et al., 2018)).
Revised: We run these results on three random seeds and using 500 epochs for each split (this is much higher than other baselines (Hsu et al., 2018)).

Original: Interestingly, the SDMLP with 1K neurons does better than with 10K neurons.
Revised: Interestingly, the SDMLP with 1K neurons does better than with 10K neurons.

Original: We also found that the SDMMLP+EWC models were still learning the last task at the end of training.
Revised: We also found that the SDMMLP+EWC models were still learning the last task at the end of training.

Original: However, increasing training times did not result in the last task being learnt better, suggesting that the model has run out of new neurons to avoid overwriting, or that the regularization coefficient for EWC should be reduced for the combination with SDM.
Revised: However, increasing training times did not result in the last task being learnt better, suggesting that the model has run out of new neurons to avoid overwriting, or that the regularization coefficient for EWC should be reduced for the combination with SDM.

Original: F. S PLIT F ASHION MNIST
Revised: F. S PLIT F ASHION MNIST

Original: We also evaluate our most successful models and baselines on the FashionMNIST dataset that has ten different classes of fashion items as grayscale 28x28 images.
Revised: We also evaluate our most successful models and baselines on the FashionMNIST dataset that has ten different classes of fashion items as grayscale 28x28 images.

Original: This task is slightly harder than
Revised: This task is slightly harder than

Original: MNIST with our oracles getting 90% instead of 99% for MNIST.
Revised: MNIST with our oracles getting 90% instead of 99% for MNIST.

Original: Table 8 shows these results as the average of 3 random seeds used to initalize the model weights.
Revised: Table 8 shows these results as the average of 3 random seeds used to initalize the model weights.

Original: The rank ordering of results agrees with that of Split MNIST Table
Revised: The rank ordering of results agrees with that of Split MNIST Table

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Models trained directly on FashionMNIST pixels and without any pretraining.
Revised: Models trained directly on FashionMNIST pixels and without any pretraining.

Original: We bold the best performing method.
Revised: We bold the best performing method

Revised: [Table]

Original: G B ASELINE I MPLEMENTATIONS G. B ETA C OEFFICIENT FOR E LASTIC W EIGHT C ONSOLIDATION AND S YNAPTIC I NTELLIGENCE
Revised: G B ASELINE I MPLEMENTATIONS G. B ETA C OEFFICIENT FOR E LASTIC W EIGHT C ONSOLIDATION AND S YNAPTIC I NTELLIGENCE

Original: The paper (Hsu et al., 2018) considers a number of baseline continual learning algorithms with different continual learning settings.
Revised: The paper (Hsu et al., 2018) considers a number of baseline continual learning algorithms with different continual learning settings.

Original: This includes the most realistic class incremental setting that we use.
Revised: This includes the most realistic class incremental setting that we use.

Original: The paper also open sourced their implementations of these different algorithms and found that the regularization methods of EWC, MAS and SI (Kirkpatrick et al., 2017; Aljundi et al., 2018;
Revised: The paper also open sourced their implementations of these different algorithms and found that the regularization methods of EWC, MAS and SI (Kirkpatrick et al., 2017; Aljundi et al., 2018;

Original: Zenke et al., 2017) all catastrophically forget Split MNIST.
Revised: Zenke et al., 2017) all catastrophically forget Split MNIST.

Original: However, in our hands by more carefully tuning the loss coefficient for the regularization term in the loss function we were able to improve performance.
Revised: However, in our hands by more carefully tuning the loss coefficient for the regularization term in the loss function we were able to improve performance.

Original: Tuning the loss coefficient for EWC and SI did not increase their performance.
Revised: Tuning the loss coefficient for EWC and SI did not increase their performance.

Original: But when we looked at the EWC and SI learning dynamics, we found that because they were getting 100% accuracy on each task split, there was no gradient from the loss that could be used to infer the importance of each weight to be regularized for future tasks.
Revised: But when we looked at the EWC and SI learning dynamics, we found that because they were getting 100% accuracy on each task split, there was no gradient from the loss that could be used to infer the importance of each weight to be regularized for future tasks.

Original: In order to give the model gradient information, we modified its cross entropy loss, introducing a β < 1 coefficient that made the model less confident in its prediction.
Revised: In order to give the model gradient information, we modified its cross entropy loss, introducing a β < 1 coefficient that made the model less confident in its prediction.

Original: Formally:
Revised: Formally:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (18) where l i ∈ R are the real valued outputs (logits) for each of the o output classes, indexed by i .
Revised: where l i ∈ R are the real valued outputs (logits) for each of the o output classes, indexed by i .

Original: And when put through the softmax distribution this gives a probability distribution where (cid:80) oi
Revised: And when put through the softmax distribution this gives a probability distribution where (cid:80) oi

Original: =1 p i = 1 .
Revised: =1 p i = 1 .

Original: In the original algorithm of (Hsu et al., 2018), β = 1 by not being a hyperparameter to tune and this β = 1 is large enough that it results in the model output for the correct class getting a probability of ∼ 1 with values of 0 for all other classes, giving a loss of 0 and no gradient.
Revised: In the original algorithm of (Hsu et al., 2018), β = 1 by not being a hyperparameter to tune and this β = 1 is large enough that it results in the model output for the correct class getting a probability of ∼ 1 with values of 0 for all other classes, giving a loss of 0 and no gradient.

Original: By having β < 1 , we force our model to be less confident in its prediction of the correct class, creating a loss and gradient information to infer which parameters are important.
Revised: By having β < 1 , we force our model to be less confident in its prediction of the correct class, creating a loss and gradient information to infer which parameters are important.

Original: Note that this β term is only used when inferring parameter importance, not when training the model.
Revised: Note that this β term is only used when inferring parameter importance, not when training the model.

Original: By tuning the β value in conjuction with the regularization loss coefficient, we were able to avoid catastrophic forgetting and again exceed the baseline reported in (Hsu et al., 2018) for EWC and SI.
Revised: By tuning the β value in conjuction with the regularization loss coefficient, we were able to avoid catastrophic forgetting and again exceed the baseline reported in (Hsu et al., 2018) for EWC and SI.

Original: While this is another hyperparameter that must be tuned, and is not a modification mentioned in the original algorithms of (Kirkpatrick et al., 2017; Zenke et al., 2017), or other literature that we are aware of, it is a straightforward modification that boosts performance and retains the original essence of the algorithm.
Revised: While this is another hyperparameter that must be tuned, and is not a modification mentioned in the original algorithms of (Kirkpatrick et al., 2017; Zenke et al., 2017), or other literature that we are aware of, it is a straightforward modification that boosts performance and retains the original essence of the algorithm.

Original: Therefore, we feeling that it is an appropriate modification to create more meaningful baselines.
Revised: Therefore, we feeling that it is an appropriate modification to create more meaningful baselines.

Original: [Table]
Revised: [Table]

Original: Testing a 1,000 neuron single hidden layer MLP on Split MNIST with 10 epochs per split we get the final validation accuracies after hand trying a few different hyperparameters shown in Table 9.
Revised: Testing a 1,000 neuron single hidden layer MLP on Split MNIST with 10 epochs per split we get the final validation accuracies after hand trying a few different hyperparameters shown in Table 9.

Original: We show the hyperparameters used and present results from the class incremental setting of Tablein (Hsu et al., 2018) for comparison.
Revised: We show the hyperparameters used and present results from the class incremental setting of Tablein (Hsu et al., 2018) for comparison.

Original: Note that the number of epochs here is fewer than in the full Split MNIST analysis of Appendix F.3.
Revised: Note that the number of epochs here is fewer than in the full Split MNIST analysis of App. F.3.

Original: This is to relate our results to other baselines that use only a small number of epochs per task (Hsu et al., 2018).
Revised: This is to relate our results to other baselines that use only a small number of epochs per task (Hsu et al., 2018).

Original: If you compare Table 9 with Table 7 of F.3 you will see that training for more epochs does affect the performance of EWC and MAS.
Revised: If you compare Table 9 with Table 7 of F.3 you will see that training for more epochs does affect the performance of EWC and MAS.

Original: G. F LY M ODEL P ARAMETERS
Revised: G. F LY M ODEL P ARAMETERS

Original: The FlyModel is unique in being trained for only one epoch and not using backpropagation.
Revised: The FlyModel is unique in being trained for only one epoch and not using backpropagation.

Original: The authors outlined a method for the model to be trained for more than one epoch on each task, however, this introduces synaptic decay that would likely reduce performance via forgetting.
Revised: The authors outlined a method for the model to be trained for more than one epoch on each task, however, this introduces synaptic decay that would likely reduce performance via forgetting.

Original: It was never implemented in (Shen et al., 2021) and so we present the strongest version of the algorithm trained for one epoch on each task here.
Revised: It was never implemented in (Shen et al., 2021) and so we present the strongest version of the algorithm trained for one epoch on each task here.

Original: We used the parameters outlined in (Shen et al., 2021) as a starting point.
Revised: We used the parameters outlined in (Shen et al., 2021) as a starting point.

Original: Because the dimensionality of our ConvMixer embeddings is 256, half of the 512 used in the original paper, we were able to cut in half the number of Kenyon cells from 20,000 to 10,000, conveniently fitting the number of neurons considered in our other experiments.
Revised: Because the dimensionality of our ConvMixer embeddings is 256, half of the 512 used in the original paper, we were able to cut in half the number of Kenyon cells from 20,000 to 10,000, conveniently fitting the number of neurons considered in our other experiments.

Original: We also varied the number of projection neuron connections between 64 and 3, finding that 32 performed better in both the 10,000 and 1,000 neuron settings.
Revised: We also varied the number of projection neuron connections between 64 and 3, finding that 32 performed better in both the 10,000 and 1,000 neuron settings.

Original: We experimented with the learning rate and found that the value of 0.005 worked best for the MNIST and CIFAR10 experiments.
Revised: We experimented with the learning rate and found that the value of 0.005 worked best for the MNIST and CIFAR10 experiments.

Original: For CIFAR100, we got better performance using a learning rate of 0.
Revised: For CIFAR100, we got better performance using a learning rate of 0.

Original: but only for the 10,000 Kenyon cell model.
Revised: but only for the 10,000 Kenyon cell model.

Original: The best results for each run across the random seeds is what is presented in the text.
Revised: The best results for each run across the random seeds is what is presented in the text.

Original: H I NVESTIGATING D IFFERENCES IN C ONTINUAL L EARNING A BILITIES
Revised: H I NVESTIGATING D IFFERENCES IN C ONTINUAL L EARNING A BILITIES

Original: We investigated why SDM is robust to catastrophic forgetting while baseline models and SDMLP models that ablate specific features all fail.
Revised: We investigated why SDM is robust to catastrophic forgetting while baseline models and SDMLP models that ablate specific features all fail.

Original: To summarize our findings:
Revised: To summarize our findings:

Original: • ReLU - neurons never learn to tile the data manifold and every neuron is activated by almost every input.
Revised: • ReLU - neurons never learn to tile the data manifold and every neuron is activated by almost every input.

Original: • Top-K - the lack of L 2 normalization and the use of a bias term means neurons don’t tile the data manifold.
Revised: • Top-K - the lack of L 2 normalization and the use of a bias term means neurons don’t tile the data manifold.

Original: This means they are not only activated by more tasks, forgetting previous ones.
Revised: This means they are not only activated by more tasks, forgetting previous ones.

Original: Additionally, there are more dead neurons reducing memory capacity.
Revised: Additionally, there are more dead neurons reducing memory capacity.

Original: • SDM Top-K Mask - masking leads to more dead neurons and the neurons that are alive being more polysemantic, failing to have subnetworks that avoid being overwritten by future tasks.
Revised: • SDM Top-K Mask - masking leads to more dead neurons and the neurons that are alive being more polysemantic, failing to have subnetworks that avoid being overwritten by future tasks.

Original: • SDM No L 2 Norm - there are few neurons with massive weight norms that are active for up to 50% of all inputs.
Revised: • SDM No L 2 Norm - there are few neurons with massive weight norms that are active for up to 50% of all inputs.

Original: These neurons are active for all tasks, resulting in memory overwriting and catastrophic forgetting.
Revised: These neurons are active for all tasks, resulting in memory overwriting and catastrophic forgetting.

Original: During Split CIFAR10 continual learning we checkpoint our model every 100 epochs, training on each data split for 300 epochs for a total of 1,500.
Revised: During Split CIFAR10 continual learning we checkpoint our model every 100 epochs, training on each data split for 300 epochs for a total of 1,500.

Original: We also track the number of times that each neuron is active across the entirety of training and visualize this on the log 10 plots shown in Fig.
Revised: We also track the number of times that each neuron is active across the entirety of training and visualize this on the log 10 plots shown in Fig.

Original: Fig.
Revised: Fig.

Original: 22a compares SDM to Top-K and ReLU where it is clear that ReLU neurons (green) are all activated many times without the Top-K activation function.
Revised: 22a compares SDM to Top-K and ReLU where it is clear that ReLU neurons (green) are all activated many times without the Top-K activation function.

Original: While Top-K has fewer activations (orange), its neurons lack the bimodal distribution of SDM (blue) that we believe corresponds to the unique subnetworks only activated for a specific task.
Revised: While Top-K has fewer activations (orange), its neurons lack the bimodal distribution of SDM (blue) that we believe corresponds to the unique subnetworks only activated for a specific task.

Original: Fig.
Revised: Fig.

Original: 22b looks at SDMLP ablations that use a
Revised: 22b looks at SDMLP ablations that use a

Original: Top-K Mask instead of subtraction and no L 2 normalization.
Revised: Top-K Mask instead of subtraction and no L 2 normalization.

Original: Note the small blip of neurons for no
Revised: Note the small blip of neurons for no

Original: L 2 norm (green) that corresponds to the greedy neurons always in the Top-K for all tasks.
Revised: L 2 norm (green) that corresponds to the greedy neurons always in the Top-K for all tasks.

Original: Also note that the Top-K mask (orange) is slightly shifted towards larger activation values.
Revised: Also note that the Top-K mask (orange) is slightly shifted towards larger activation values.

Original: For all of our analysis that follow we do not show dead neurons.
Revised: For all of our analysis that follow we do not show dead neurons.

Original: These dead neurons are only really a problem for Top-K that has 55% dead neurons and 18% for SDM without an L 2 norm.
Revised: These dead neurons are only really a problem for Top-K that has 55% dead neurons and 18% for SDM without an L 2 norm.

Original: To look at how specialized each neuron is to specific tasks and data classes, we take the final models after Split CIFAR10 continual learning and pass all of the CIFAR10 training data through them, recording which neurons are in the Top-K for each input.
Revised: To look at how specialized each neuron is to specific tasks and data classes, we take the final models after Split CIFAR10 continual learning and pass all of the CIFAR10 training data through them, recording which neurons are in the Top-K for each input.

Original: For each neuron, we count the number of times it is active for each of the 10 input classes and use this to create a probability distribution over the classes the neuron is activated by.
Revised: For each neuron, we count the number of times it is active for each of the 10 input classes and use this to create a probability distribution over the classes the neuron is activated by.

Original: We take the entropy of this distribution as a metric for how polysemantic each neuron is.
Revised: We take the entropy of this distribution as a metric for how polysemantic each neuron is.

Original: We plot the entropy of each neuron against the percentage of time it is in the Top-K in Fig.
Revised: We plot the entropy of each neuron against the percentage of time it is in the Top-K in Fig.

Original: We also combine these two metrics to weight each neurons’ entropy by the amount of time it is in the Top-K to compute the average entropy of activated neurons and
Revised: We also combine these two metrics to weight each neurons’ entropy by the amount of time it is in the Top-K to compute the average entropy of activated neurons and

Original: [Figure]
Revised: [Figure]

Original: present this in Table 10 alongside the continual learning performance of each method.
Revised: present this in Table 10 alongside the continual learning performance of each method.

Original: There is a clear inverse correlation between the average entropy of activated neurons and continual learning.
Revised: There is a clear inverse correlation between the average entropy of activated neurons and continual learning.

Original: [Table]
Revised: [Table]

Original: Figure 23 and the summary in Table 10 effectively convey the subnetwork formation by SDM that enables strong continual learning performance.
Revised: Fig. 23 and the summary in Table 10 effectively convey the subnetwork formation by SDM that enables strong continual learning performance.

Original: SDM (Fig. 23a) has the lowest mean entropy of activated neurons followed by SDM with the Top-K mask (Fig. 23b).
Revised: SDM (Fig. 23a) has the lowest mean entropy of activated neurons followed by SDM with the Top-K mask (Fig. 23b).

Original: The SDM with Top-K Mask
Revised: The SDM with Top-K Mask

Original: Fig.
Revised: Fig.

Original: 23b shows how the most polysemantic (highest entropy) neurons are also the most active which will result in forgetting across tasks.
Revised: 23b shows how the most polysemantic (highest entropy) neurons are also the most active which will result in forgetting across tasks.

Original: Figure 23c strikingly shows 13 “greedy” neurons in the top right that are not only highly polysemantic but active for many inputs (compare the y-axis going up to 6% compared to 1.4% for SDM).
Revised: Fig. 23c strikingly shows 13 “greedy” neurons in the top right that are not only highly polysemantic but active for many inputs (compare the y-axis going up to 6% compared to 1.4% for SDM).

Original: Top-K in Fig.
Revised: Top-K in Fig.

Original: 23d looks like the SDM with Top-K Mask but the average polysemantism of each neuron is much higher.
Revised: 23d looks like the SDM with Top-K Mask but the average polysemantism of each neuron is much higher.

Original: Finally, the ReLU network in Fig. ??
Revised: Finally, the ReLU network in Fig. ??

Original: , while haved a very even distribution of neuron activations, having the lowest y-axis range of 0.16%, the neurons are all highly polysemantic.
Revised: , while haved a very even distribution of neuron activations, having the lowest y-axis range of 0.16%, the neurons are all highly polysemantic.

Original: Notably, the fact that SDM in Fig.
Revised: Notably, the fact that SDM in Fig.

Original: 23a does not have every neuron activated and in the Top-K the same percentage of the time means that it’s neurons do not perfectly tile the data manifold in proportion to the density of the data.
Revised: 23a does not have every neuron activated and in the Top-K the same percentage of the time means that it’s neurons do not perfectly tile the data manifold in proportion to the density of the data.

Original: However, this may be because we are optimizing the network for classification performance instead of reconstruction loss.
Revised: However, this may be because we are optimizing the network for classification performance instead of reconstruction loss.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: We present a number of additional plots that show the manifold tiling abilities of SDM in comparison to the other methods.
Revised: We present a number of additional plots that show the manifold tiling abilities of SDM in comparison to the other methods.

Original: Figure 24 fits the UMAP projection on the weights of the pretrained on
Revised: Fig. 24 fits the UMAP projection on the weights of the pretrained on

Original: ImageNet32 models and then uses this projection for the CIFAR10 data.
Revised: ImageNet32 models and then uses this projection for the CIFAR10 data.

Original: Figure 25 does the inverse where it fits a UMAP (McInnes & Healy, 2018) projection to ConvMixer embedded CIFAR training data and uses this projection for the trained model weights.
Revised: Fig. 25 does the inverse where it fits a UMAP (McInnes & Healy, 2018) projection to ConvMixer embedded CIFAR10 training data and uses this projection for the trained model weights.

Original: Both Figures show that SDM learns to tile the data manifold most effectively with the full SDMLP being the only method to tile the manifold in both figures.
Revised: Both Fig.s show that SDM learns to tile the data manifold most effectively with the full SDMLP being the only method to tile the manifold in both figures.

Original: This shows that upon pretraining, the neurons of SDM have learnt to differentiate across the manifold of general image statistics, forming subnetworks that will be useful for continual learning.
Revised: This shows that upon pretraining, the neurons of SDM have learnt to differentiate across the manifold of general image statistics, forming subnetworks that will be useful for continual learning.

Original: We emphasize that while there is manifold tiling and subnetwork formation, the ImageNet32 pretraining does not result in SDM already knowing the CIFAR10 data or perfectly tiling the CIFAR manifold.
Revised: We emphasize that while there is manifold tiling and subnetwork formation, the ImageNet32 pretraining does not result in SDM already knowing the CIFAR10 data or perfectly tiling the CIFAR manifold.

Original: Figure 27 shows a UMAP plot with the projection fit to the SDMLP trained directly on
Revised: Fig. 27 shows a UMAP plot with the projection fit to the SDMLP trained directly on

Original: CIFAR10 pixels and projecting the same CIFAR10 data, note how much tighter the manifold tiling is here.
Revised: CIFAR10 pixels and projecting the same CIFAR10 data, note how much tighter the manifold tiling is here.

Original: Further evidence of manifold tiling is evident in this pixel based training where in Figure 26 we take the weights of ten random SDMLP neurons and reshape them into 3x32x32 dimensions to reveal the neurons have specialized to specific classes in the data.
Revised: Further evidence of manifold tiling is evident in this pixel based training where in Fig.we take the weights of ten random SDMLP neurons and reshape them into 3x32x32 dimensions to reveal the neurons have specialized to specific classes in the data.

Original: This is in stark contrast to the weights of the ReLU neurons shown adjacently.
Revised: This is in stark contrast to the weights of the ReLU neurons shown adjacently.

Original: SDM neurons specialize to not only specific image classes but even specific examples within the class, analogous to the hypothetical “grandmother neuron” (Gross, 2002; Quiroga et al., 2005).
Revised: SDM neurons specialize to not only specific image classes but even specific examples within the class, analogous to the hypothetical “grandmother neuron” (Gross, 2002; Quiroga et al., 2005).

Original: This figure shows the trained weights of randomly chosen neurons from an SDMLP trained on CIFAR10 pixels and reshaped into their image dimensions.
Revised: This figure shows the trained weights of randomly chosen neurons from an SDMLP trained on CIFAR10 pixels and reshaped into their image dimensions.

Original: A bird, frog, and multiple deer and horses are visible.
Revised: A bird, frog, and multiple deer and horses are visible.

Original: Note that if we directly visualize these L 2 normalized weights and our L 2 normalized CIFAR images, they all look black because the pixel values denoting RGB colors are between 0 and 1 while our normalization makes them much smaller.
Revised: Note that if we directly visualize these L 2 normalized weights and our L 2 normalized CIFAR images, they all look black because the pixel values denoting RGB colors are between 0 and 1 while our normalization makes them much smaller.

Original: We rescale our weights back into pixel values by first subtracting the minimum pixel value x − min ( x ) and then multiplying x ∗ 1 / (max ( x )) that is standard practice.
Revised: We rescale our weights back into pixel values by first subtracting the minimum pixel value x − min ( x ) and then multiplying x ∗ 1 / (max ( x )) that is standard practice.

Original: [Figure]
Revised: [Figure]

Original: To fully emphasize that these neurons and their interpretable receptive fields are on meaningful parts of the manifold, we take a neuron that has a frog receptive field and show where it is located on the manifold.
Revised: To fully emphasize that these neurons and their interpretable receptive fields are on meaningful parts of the manifold, we take a neuron that has a frog receptive field and show where it is located on the manifold.

Original: We then scan the CIFAR10 training data and take the 3 images that maximally activate
Revised: We then scan the CIFAR10 training data and take the 3 images that maximally activate

Original: [Figure]
Revised: [Figure]

Original: this neuron, plotting them in the top row of Figure 27.
Revised: this neuron, plotting them in the top row of Fig.

Original: We also look at the UMAP embedded images that have the smallest euclidian distance to the UMAP embedding of our neuron weights and show the closest three images.
Revised: We also look at the UMAP embedded images that have the smallest euclidian distance to the UMAP embedding of our neuron weights and show the closest three images.

Original: This UMAP apporach does contain a horse but this approach will be less precise due to the UMAP embedding mapping the 3072 dimensional images into 2 dimensions.
Revised: This UMAP apporach does contain a horse but this approach will be less precise due to the UMAP embedding mapping the 3072 dimensional images into 2 dimensions.

Original: We use our neuron UMAP embeddings to visualize the number of times that each neuron is activated in Fig.
Revised: We use our neuron UMAP embeddings to visualize the number of times that each neuron is activated in Fig.

Original: This uses the same overall activation values across continual learning presented in the histograms of Fig.
Revised: This uses the same overall activation values across continual learning presented in the histograms of Fig.

Original: 22 with yellow indicating the most activations and purple the fewest.
Revised: 22 with yellow indicating the most activations and purple the fewest.

Original: Finally, we show the data manifold tiling of SDM in Fig.
Revised: Finally, we show the data manifold tiling of SDM in Fig.

Original: 29 where we assign each CIFAR10 class a different color in the left plot and then show the positions of each neuron assigning the color that activates this neuron the most.
Revised: 29 where we assign each CIFAR10 class a different color in the left plot and then show the positions of each neuron assigning the color that activates this neuron the most.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]