UDAEE: Unsupervised Domain Adaptation in PLMs with Early Exits
Authors: Divya Jyoti Bajpai, Manjesh Kumar Hanawal


Reviewer paper summaries: The authors aim to address the overthinking issue in domain adaptation by training each later using early exits. They first train exit classifiers for each layer. After this, they use adversarial training with each layer's embeddings to encourage domain invariance, and use a KL loss term to set up exit classifiers for the target.
Summary of strengths: 1. Addressing the overthinking issue in domain adaptation, to the best of the reviewer's knowledge, is a novel and important problem. 
2. Strong empirical results of proposed method.
Summary of weaknesses: Contrary to the paper's claims, prior work has also addressed learning representations for adaptation at each layer of the model. These works do not target the overthinking issue, but it would be nice add a discussion comparing UDAEE to these works to the paper.

[1] https://arxiv.org/abs/2302.03194

[2] https://arxiv.org/abs/2401.17514
Comments, suggestions and typos: None.
Overall assessment scores: 3
Metareview score: 3
Metareview: The paper presents an approach for unsupervised domain adaptation, focusing on early exit and domain-adversarial training with knowledge distillation (KD) to train the target-domain model. The method includes a per-layer early exit classifier to improve the inference speed. The KD loss component between each layers of the source and target domain model helps avoid catastrophic forgetting. The paper shows nice experiment results, including a 67% speed-up in inference using early exits but with no degradation in performance.
Summary of reasons to publish: 1. The paper addresses an important problem of overthinking in domain adaptation.
2. Good experiment results, demonstrating inference speed-up with no performance degradation.