Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Since recent pre-trained language models are large, it is common to fine-tune small amount of parameters instead of tuning all the parameters when we would like to adapt the pre-trained model for downstream tasks (e.g. text classification etc.). Many methods have been proposed such as Prompt tuning, Adapter, BitFit and LoRa (summarized in appendix A). For example, Adapter, one of the simplest approach inserts extra layers (called adapter layers) in transformer blocks and train those layers with supervision from downstream tasks.

The straight forward approach is to directly optimize the downstream task loss, but instead authors proposed a regularization method. The proposed regularization objectives encourage hidden states of each layer to be able to approximate low-rank vector representation (PCA) of target tokens following the diffusion process explained in sec. 2.2 (and parameterized as in sec. 3). The proposed objective is added to the task objectives with some weights.

They evaluated their regularization method on fine-tuning BERT_large model with four parameter tuning approaches (Prompt tuning, Adapter, BitFit and LoRa) and show that the regularization lead to performance improvements on GLEU benchmarks in both full-set and few-shot scenario.
strength_and_weaknesses: Strength
- Since the proposed regularization is not dependent of parameter tuning methods, it can be used for many parameter tuning methods.
- The experiments shows improvements on GLEU benchmarks in both full-set and few-shot scenario.

Weakness
- It was not clear what optimal control perspective implies. It's common to use regularization / prior / pre-training in machine learning but the new perspective implies something special (which lead to the proposed method)?
- The method assumes that the pre-training corpus is available, but previous works do not assume so.
- The method looked like pre-training of newly introduced parameters (effective for few-shot setup). Is there any justifications why the stochastic bridges is necessary?
  - What will be the difference between stochastic bridges and pre-training the new parameters on PLM objective?
clarity,_quality,_novelty_and_reproducibility: 1. The paper was a little bit hard to read (e.g. eq.5/6 uses function not explained yet) but the idea is simple.
2. It should be made clear in the tables that proposed methods use more data than the baseline.
3. The idea to use diffusion bridge looks new but it is not clear why this is necessary because of the lack of baselines.
4. The authors do not include code and data to reproduce the results.
summary_of_the_review: Although the proposed method show improvements, it requires extra data to pre-train the parameters than baselines. And it is hard to make fair comparison with existing methods especially for the few-shot setup. Also, the use of stochastic bridges is not justified conceptually and empirically. For these reasons, I believe this paper is not ready for publication.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: 

This paper proposes a novel framework to do parameter efficient fine-tuning method for PLM (pre-trained language models). The basic idea is to hypothesize that there exists a map from the hidden states in each layer to a lower dimensional latent space, where the trajectory of this lower dimensional map from the zeroth layer to the final layer is from a diffusion bridge that ends at the PCA reduced dimension points of each output target word in the final layer.  The authors propose an interpretation of this as the running cost to augment the usual final terminal cost of the loss function. 

strength_and_weaknesses: Strengths:

The key idea is to learn a frozen non-linear dimensionality reduction map that recovers the dynamics of a stochastic bridge on the pre-training corpus and then using it in the fine-tuning stage to define an additional regularization cost. Overall, the proposal of using a diffusion bridge with an endpoint derived from the word embedding at the final layer in PLM (a projected version of it) seems novel / interesting.  Justifying the novel fine-tuning objective as a regularization cost seems intuitively plausible, even if the other motivations like considering it as running cost versus terminal cost appear to be a superficial connection. The authors also conduct experiments to show that this scheme can help marginally improve the final results in practice. In addition to evaluating the benchmarks, the authors also conduct some analysis of the hidden states of the various methods in terms of the clustering properties of a reduced dimension visualization.

Weaknesses:

- Clarity of the writing is quite unclear and leaves a lot of details to imagination.  Please see the next section for concrete suggestions.
-  The authors claim that they "reemphasize" a "running cost" in addition to the terminal cost (which is simply the model loss), however it is unclear why this running cost is important for the overall objective, especially given that this is a cost that the authors define, rather than inspired or proposed from a pre-existing consideration or loss.  
- It also seems somewhat arbitrary that this new regularization cost is specific to parameter efficient tuning, but it does not seem specific to this setting (e.g. what aspect of going from general finetuning to PET is specific to the method proposed here?)
- Optimal control formulation is often mentioned for the motivation, but the connection is limited to that and not really used in the technical contents of the paper.
- The significance of hidden state in vanilla PETS "approaching the latent bridges" is unclear to me. Specifically, given that the final reduced dimension representations of the embeddings are estimated from the pre-trained corpus, wouldn't a better fit to the end points automatically imply a better prediction accuracy? 


clarity,_quality,_novelty_and_reproducibility: The idea of interpreting interpreting the tunable parameters are control variables on top of a frozen PLM is not novel to this work, as the authors cite (Yang & Liu' 22, Ding et al. '22). 

Clarity is good in some aspects, but needs improvements in several other critical components -- for example, the formalism was incomplete as to where the continuous time aspect of the diffusion bridges is entering the regularization framework in the fine-tuning objective. 

When $\bar{h}_o^{(i)}$ is defined as "the average of all hidden states at the $i-$th layer",  what does this mean? Doesn't that make it independent of the position $o$ if interpreting "all" as a reduction over the time steps?

What is the $t_i$ in Equations (5) and (6)? Is this just a uniformly spaced grid with length equal to the number of layers? It would be good to make this clear without expecting the reader to guess things. 

The diffusion bridge formalism starts at 0 and then ends at a parameter $\beta$. What does this map to for the bridge $X_y$ used in the application (just above Equation (5)? Inspite of having a paragraph Section 3.2 on this, this aspect is still very unclear.

The Method 2 (approximating the SDE)'s description leaves several variables undefined which is concerning given that it is the best performing method in the experiment. Things like $\sigma$, $f^{T; \beta}$ ? There is also an $\tilde{X}_t$, which is supposed to be an estimate that's again undefined. 

summary_of_the_review: Overall, this paper proposes an interesting idea for a regularizer for fine-tuning a pre-trained language model (PLM). The most interesting part about this is to use the frozen model to constrain the dynamics of the hidden states to follow a distribution similar to those in the pre-training corpus (via a dimensionality reduction). The authors also allude to optimal control connections for this idea, which is not particularly significant in my view. However, the clarity of presentation is still somewhat unclear as I write above in more detail. 

The experimental results seem promising, but the claims about a novel discovery related to the hidden states forming a latent bridge might not be fully convincing. Overall, the paper has some merits in terms of novelty and motivation, but the technical details and the significance of the empirical results are not totally convincing.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a class of regularization objectives that are used to improve the effectiveness of parameter-efficient tuning methods (PETs) for pre-trained language models. Specifically, the objective consists of two main components: (i) a (potentially learnable) diffusion bridge defining a target diffusion process; (ii) a mapping function from every layer's output of the language model to a trajectory that approximates the diffusion bridge. This paper uses a fixed diffusion bridge defined by applying PCA to the output word embeddings. Experiment results show that the proposed regularizer is able to improve the performance of existing PETs.
strength_and_weaknesses: Strength:

- The diffusion bridge forces the latent representation of each layer to contain more information of the target, and can be viewed as a shortcut for learning better latent representations tailored to different tasks.

- The authors demonstrate that task performance is correlated with the distance to the diffusion bridge, which serves as an empirical evidence of the effectiveness of the proposed regularizer.

Weaknesses and questions:

- One weakness of the proposed method is that pre-training corpus is required to train the mapping $g_{\gamma}$. The problem here is two-fold: (i) pre-training corpus could be hard to access and (ii) since it is much larger than datasets of downstream tasks, training $g_{\gamma}$ could be computational demanding. For example, I would like to know whether the proposed method is still effective if we only have access to a small subset of the pre-training corpus.

- Following the previous comment, the computation cost of training $g_{\gamma}$ should be discussed. Furthermore, it would be nice if the authors can provide additional experiments by constraining the GPU time of the baselines and the proposed approach to be the same.
clarity,_quality,_novelty_and_reproducibility: The paper is well-written in general. The proposed regularizer is a clever integration of language models with diffusion bridges of the target output. Training hyperparameters are included in the appendix.
summary_of_the_review: In summery, I tend to vote for acceptance since the proposed regularizer makes clever use of target diffusion processes to improve the fine-tuning performance. However, the proposed method requires pre-training corpus, which could add significant computation cost. It is also not well discussed how much pre-training corpus is needed to receive significant performance gain.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a novel regularization for tuning large pre-trained language models (PLMs) with parameter-efficient tuning methods (PETs). This method is motivated with a connection of modeling intermediate state dynamics to optimal control literature and uses stochastic bridges for a target difffusion process as the regularizer. The experiments show improved performance on downstream finetuning tasks when applied to existing PETs.

Strengths:
- A novel regularization method using stochastic bridges on intermediate states rather than parameters for PET.

Weaknesses:
- Clarity. Multiple reviewers find the paper hard to follow, and the connection to optimal control unclear.
- Require original training corpus to train the regularizer which deserves more discussion than the current form.
- The significance of empirical results are not totally convincing.
justification_for_why_not_higher_score: Concerns on multiple weaknesses are shared by reviewers, especially on the clarity of the motivation and technical presentation. The authors' feedback is not sufficient to address the concerns.
Decision: Reject