Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies the underlying reason behind why fine tuning pre-trained language models works well. The paper shows that in some cases, those fine tuned models can be described using neural tangent kernel. In those cases, the kernel view provide explanation of the fine-tuning success. 

strength_and_weaknesses: Strength:
The paper tackle one of the most important questions in machine learning: understanding the reason behind the the performance of fine-tuned models, and in particular, when it should works.

Weakness
I found it hard to follow the paper's arguments. see section below for more details. 
clarity,_quality,_novelty_and_reproducibility: I found the paper unclear and hard to follow, in a way that is hard to appropriately appreciate the results. In particular I found the following points unclear:
- Framing: The paper aims to further an empirical and theoretical understanding of pre-training and fine-tuning paradigm for NLP tasks, but this goal is too broad. As a result, the paper highlight too many points, and does not focus on none of them, especially with the the page limit. Creating a coherent story by refining a single take-away message from all of the described results, will help readers to follow the paper.
- Mathematical motivation: The mathematical definitions appear without a proper motivation. Although the paper claim that some of the definitions are analogues to definitions in other papers, I believe the paper should be self-contained, and provide the intuition in high level.
- Self contains claims: All mathematical claims should be self contained within the body of the paper, where the appendix can be used for proof, and for a formal version of the claims. Most claims of the paper are indeed presented in a self-contained manner, except Theorem 7.3, in which the conditions themselves appear in the appendix, without any informal phrasing of them.
- references: it is very helpful that the paper have hyperlink to the arXiv, but it shouldn't replace mentioning the peer-review venue in which the papers appeared in. In the current format it is hard to validate that the paper is based on peered-review works.
- Acronyms: please define all acronyms, for examples SGD.


summary_of_the_review: The paper presents several theoretical and experimental results, that I don't feel creates a coherent bottom line. In addition, It is hard to evaluate the quality of those results, without reading the appendix (which I didn't read). Thus, I cannot recommend accepting the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper suggests that the success of language model fine-tuning (or with prompt) can be explained by neural tangent kernel (NTK), a tool that describes optimization behavior of infinitely wide neural networks. To bridge NTK with LM fine-tuning, the paper tries to justify that using NTK after pre-training phase (instead of initializing from lazy regime) is reasonable, and also derived empirical NTK for adam optimizer. The paper found empirical evidence that NTK behaves similarly to the original optimization methods, and also provided a theorem that shows prompt-based fine-tuning can exhibit kernel behavior
strength_and_weaknesses: Strength: 
The paper tries to explain language model fine-tuning using NTK and cleverly uses the fact that fine-tuning takes few samples (therefore one can compute eNTK in a feasible way). 

Weaknesses:
Overall, the paper is trying to bridge theory and practice. But there are a few concerns:
1. While some of the empirical results are good, some observations are quite mixed. For example, RTE linearization does not perform well for k=512 while eNTK still performs ok -- this seems confusing to me. 
2. Section 4 is unclear to me, the paper mentioned "we model Adam updates with SignGD", why can you replace Adam with SignGD and is the derived kernel exactly following Adam or is just an approximation of Adam (if later is the case, why not just approximate Adam with SGD)?
3. The theory section (Section 7): I don't see how it is related to prompt-based fine-tuning but not standard fine-tuning. The paper mentioned "The theory focuses on the prompt-based setting since we did not find empirical evidence of kernel behavior in the standard setting" (I hope theory can advance empirical work but maybe now the paradigm is shifted), this is fine, but which assumption in Theorem 7.3 is specific to prompt such that the final conclusion is "prompt-based FT of f will exhibit kernel behavior"?
4. The bigger question is whether NTK is THE theory or just A theory. Obviously a lot of practices in LM training or FT do not match with NTK behavior (e.g., learning rate warmup), therefore assuming NTK is THE theory and develop reasoning that tries to explain FT is a bit insufficient, so I think this is a weakness of the paper. 
clarity,_quality,_novelty_and_reproducibility: Clarity & Quality: The paper is overall clear, I'd be happy to get clarified on Section 4 & 7 a bit more. In terms of writing, I do think there is a better way to re-organize and better separate the assumption & theory with empirical results. Currently, empirical evidence and theory are interleaving and a bit hard to get the full picture. 

Novelty & Reproducibility: Overall relatively novel and should be easy to reproduce. 
summary_of_the_review: My feeling is a bit mixed for this paper. My current rating would be marginally below the threshold given that there are several issues worth clarifying. Regarding Weakness #4, I think different people may have different feeling on how important this is. I am relatively close to the median of that spectrum, but I can definitely see researchers having strong opinions from both sides.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies behavior of NTK on fine-tuning MLM models. Supposedly, pre-trained model is no longer iid random in parameters but still it can achieve good performance on CV tasks per previous literature. In this paper, authors extended the studies to NLP models with both regular fine-tuning and prompt-based learning cases. Authors also provide a modified eNTK version when using ADAM but not SGD. Last, authors tried to propose some hypothesis on why such model on eNTK behaves so.
strength_and_weaknesses: Strength:

- An interesting observation and continuation of previous work.
- Provide studies over various related topics with mixed empirical and theoretical analysis.

Weakness:

Main contribution seems to just be combining two major previous results and somehow it's not providing enough insight.
clarity,_quality,_novelty_and_reproducibility: The paper is not well organized. Claim many things but the results are spread out here and there. It's very hard to understand the contribution before linking them. In terms of novelty, I think the angle to approach the underlying NLP phenomena is interesting but no real technical difficulties or insightful theoretical results given.

summary_of_the_review: Overall, I like the angles and the topic of this paper. But somehow I think it's a bit disappointing after reading it as it's not providing much insight. Rather, it's more of a mixed of empirical evidence and some not that useful theoretical results. I do believe we should encourage this type of work so overall I am more positive, but there are some confusions and I hope authors could explain it.

Overall, I think the gist of the paper is a bit unclear. Mentioned too many points but I can't find the direct link between the claim and analysis. You listed many contributions in bullet points so I guess it's easier to go from there.

- Does the eNTK method directly work for NLP tasks? In Section 5, we show that eNTK does not
work well for standard FT but it achieves performance comparable to prompt-based FT (Schick
& Schutze, 2021; Gao et al., 2021) on a majority of downstream tasks that we evaluate. ¨

- So I think this is just an empirical analysis. Somehow just an extension of domain from wei el. al. 
Do you agree with this? If so, can you highlight this is more of an empirical results. in addition,  can
you also greatly re-organize your paper to make contribution of empirical results and theoretical analysis clear?

• The NTK theory was developed for gradient descent, whereas LM fine-tuning often uses
Adam (Kingma & Ba, 2014) as a standard practice (Devlin et al., 2019). Can the above method
be extended to give insight into the effect of different optimization algorithms? In Section 4,
we derive a new asymmetric “kernel” formula (Definition 4.2) that describes the dynamics of
short-term training with Adam and shows that the corresponding eNTK achieves comparable
performance as FT using Adam (Table 1).

- Here you use yet another approximation of Adam, so I don't see the purpose of it. Why does that 
matter to analyze yet another version of eNTK  just due to a different gradient method? 
In particular, Adam results in Table 1 doesn't really outperform SGD version eNTK musch. Especially,
for TREC ADAM result is much worse than eNTK one. What's the explanation? 

• Is the eNTK regression method simply an alternate model that also happens to have good
performance on the classification task, or is it actually a near-equivalent description of how
parameters of the original model evolve during fine-tuning? Section 5.3 shows that for tasks that
the eNTK can solve, the fine-tuning of the pre-trained model does exhibit behavior consistent
with the dynamics of training kernel classifiers. See Definition 3.1.

- I think this could be a real contribution. But somehow I feel not consistent as fine-tuning + eNTK 
seems not to work but here you showed its empirical performance justified the assumptions/desired
behavior of NTK in 3.1 ? It's kind of contradicting. 

• If the eNTK can solve NLP tasks, then does the eNTK give insight into phenomena such as why
prompting can improve performance and how parameter-efficient fine-tuning methods affect
optimization? Figure 1 demonstrates that adding a prompt is necessary for the eNTK to solve the
task, and in Section 6, we apply the kernel lens to provide a possible explanation for the empirical
success of subspace-based fine-tuning methods (Hu et al., 2021; Aghajanyan et al., 2021).

- Sorry I don't quite get why prompting can improve performance is related to subspace-based fine-tuning methods?
I think the connection is not very obvious to me. I was feeling strange when I read your section 6. I am not sure
why you want to analyze it. And I felt the connection to other part of paper is too weak such that it's another paper.

We conclude by proposing a rigorous mechanism through which fine-tuning of complex architectures
(e.g., Transformers (Vaswani et al., 2017)) with prompts can exhibit kernel behavior. This is done
in context of networks whose width goes to infinity, but unlike standard infinite-width NTK theory
it allows a non-random initialization that is the result of pretraining. This result uses the Tensor
Programs framework (Yang, 2019; 2020a;b; Yang & Littwin, 2021; Yang & Hu, 2021).

- I actually feel this part of analysis is the most pertinent part regarding your motivations. Why don't you put it in the front? But I am not sure the correctness and framework as I didn't know Tensor Programs before.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper analyzes the process of fine tuning of language models via an analysis using Neural Tangent Kernels. The goal of better understanding the process of fine-tuning is well motivated and the reviews agree that there is value in a result deepening the communities understanding of the process. In addition to being a well motivated problem, the paper has healthy mix of theoretical and empirical analysis. 
The paper exhibits two major weaknesses, both raised in all of the reviews. The first regards the clarity of the paper: The reviewers had a hard time understanding the main message of the paper, and the details of the theoretical claims and proofs. The authors improved the writing during the rebuttal stage, but this is acceptable when there are a handful of isolated issues, and this doesn't seem to be the case (for example, in the response to the rebuttal, reviewer Ftva still mention the incoherence of the story as weakness).
A second issue that I find more problematic regards the impact of the paper. Although the reviews do not directly mention it, they all mention that the conclusions aren't coherent, that observations are mixed, or the paper isn't providing enough insight. This may be an issue with the writing clarity, but given that all reviews make these claims in some way, there is a risk of the results not being actionable / insightful enough, making impact a weakness of the paper. 
To conclude, the paper in its current state doesn't seem to meet the bar for ICLR.

justification_for_why_not_higher_score: Agreement among all 3 reviewers that the paper is not written well enough and the results aren't conclusive enough. For this type of paper, a clear message is crucial for it to have an impact.
justification_for_why_not_lower_score: n/a
Decision: Reject