DIEK\AE: Difference Injection for Efficient Knowledge Augmentation and Editing of Large Language Models
Authors: Alessio Galatolo, Meriem Beloucif, Katie Winkle


Reviewer paper summaries: The paper propose an adapter-based approach for knowledge augmentation, with a novel training objective that uses two forward passes to efficiently train the involved extra parameter.
Summary of strengths: 1. The idea is relatively novel. 

2. The author clearly states the novelty / difference with existing methods.
Summary of weaknesses: 1. The biggest problem of this paper lies in its writing. Some contents conflict with each other, for example, line 355-356 state the proposed method targets at "pretraining", but all the rest of the paper indicates the method is for fine-tuning. Some statement is not true/accurate. For example, in line 307, the paper says "fine-tuning a 2.7b model requires as much as 80GB GPU memory". There are many grammar errors that impact readers' understanding, detailed below. Some math notations are also confusing.

2. The paper states many existing baselines in the related work, but the proposed method is not compared against them in experiments. The only compared baselines are vanilla prompting or fine--tuning. At least 1 or 2 baselines in knowledge editing should be compared in terms of task performance and training/inference efficiency.

3. The method require 2 forward pass during traininig (btw, is this required during inference is also not clarified), and involves extra parameters in the encoder part. Now that the method aims at efficiency, can the author explicitly use tables to indicate comparisons of FLOPs, Extra parameter ratios, Training times, and space, with all the extra needed baselines, during both training and inference time? Simply writing the comparison in text will greatly prohibit understanding.

4. Can the method work for other LLM architectures? 

5. The writing and figures presentation can be significantly improved.
Comments, suggestions and typos: 1. Line 36 has a grammar error

2. Line 216-221 has grammar error and is not readable

3. Line 296-301 is not readable
Overall assessment scores: 2