reviews: 
Comments: This paper empirically studies catastrophic forgetting (CF) from the perspective of function vectors (FV) and makes the following core contributions:
1. It shows that there is a correlation between the change in function vector for a task and degradation in performance on the task.
2. It argues that CF is due to the inability to associate the input with the latent task vector rather than a degradation in the ability to solve a tasks conditioned on the input and correct latent task vector. It supports this with mechanistic interventions to the task vector. 
3. It presents a method to regularize FVs during training to mitigate forgetting.
Strengths: **Originality**: This was an interesting approach/hypothesis for studying forgetting that I quite enjoyed.
**Quality**: The authors perform a thorough series of experiments investigating their claims and evidence ranges from empirical measurements, mechanistic interventions, and training interventions. 
**Clarity**: The work is well contextualized in related work.
**Significance**: This work is a really interesting way to connect mechanistic interpretability to studying and mitigating forgetting demonstrating practical ways in which tools from mechanistic interpretability can be used to understand diverse phenomena.
Weaknesses: 1. **Clarity**: The paper dense and the writing is terse and sometimes hard to follow. The figures have a lot of information on them, are very small and it is quite difficult to extract takeaways from them. Overall the paper is often hard to follow and it is hard to trace the evidence for the claims. A lot of points through the paper seem like observations and the connection to one of the core scientific claims isn't always clear. I think this paper would benefit greatly from better organization. Probably most important is to explain the figures/tables better *in the captions* and actually point out what the takeaways are/what we should learn from them/what points it is substantiating. I understand this is challenging because of page constraints and the amount of information presented in the paper: my recommendation would be to move anything non-essential to the core scientific claims to the appendix. Also, the authors could present the results for 1 model and add the rest of the models to the appendix given that the models are all of the same scale.
2. **FV and forgetting**: One of the key claims in this paper is change in FV is correlated with forgetting, i.e. this is a useful measure that "explains" the phenomena of forgetting. The evidence did not convince me of this: in Figure 2, there are points where FV similarity is high, 5 shot performance is high but 0-shot performance is low. Object Count especially is unconvincing (0shot and 5shot switch and don't seem to be correlated with FV).
3. **Mechanistic interventions**: While the mechanistic interventions in Section 5 are quite interesting, the paper does not consider any alternated explanations for this behavior (see questions). It is also not clear to me why FV is a measure of latent task identification.
4. **Limitation of training method**: If I understand correctly, this training method regularizes the FV of particular tasks and thus prevents forgetting of the particular tasks being regularized. It doesn't affect forgetting as a whole? While this is not a weakness, I think this should be clarified and the method should be explained as such.
Recommendation: 10
Reviewer confidence: 3
Soundness/correctness: 3
Clarity: 2