Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a method of quantizing the parameters of pre-trained language models by dividing the model parameters into sets (or modules) and quantizing each of them separately via techniques similar to distillation referred to as "quantization error minimization". To improve the speed of this process, the authors propose learning parameters of each module in parallel (even though in general the output of one module is the input to the next) by (1) maintaining a cache of old activation values and asynchronously updating their values and using them as desired by each module, (2) using "teaching-forcing" by providing the original activation from the unquantized model in the beginning to reduce error propagation.

With experiments on quantizing BERT-base and BERT-large to different bit-sizes with downstream evaluation on GLUE and SQuAD, the authors show improvements over baselines in terms of training speed and memory required while using less data for quantization and mostly maintaining task performances. 

strengths_and_weaknesses: Strengths:

1. The proposed solution is simple, where each component--module-based minimization, teaching forcing, and parallel computation--has sound motivation and leads to empirical improvements.
2. Clear improvements in training time and memory overhead compared to baselines. 

Weaknesses:
1. Choice of #Bits for baselines and proposed methods (In table 3) are not the same, doesn't that make the comparison unfair?
2. The proposed method still lags behind QAT. Even though they require more data and more training time, this is a one-time investment and if a quantized model however obtained performs better downstream, why would it not be preferable? While data accessibility is a valid concern, other concerns listed in 2.2 are not convincing in favor of using PTQ. 

questions: See weaknesses.
limitations: NA
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper presented another way to perform post train quantization (PTQ) on BERT models with a parallel optimization method to speed up the fine-tuning process.
strengths_and_weaknesses: Strength: 
1. The paper offers a detailed ablation study on each of the auxiliary training methodologies they are using on top of the main PTQ algorithm. 
2. The paper provides extensive experimental results covering previous SOTA methods in the field of PTQ and QAT. 
3. the paper provides a detailed explanation of the methodologies and background, helping the understanding of the paper.

Weaknesses: 
1. This work provides very few intuitive/theoretical explanations of why their method could sometimes work as well as QAT methods. In particular, since the paper adapts the well-known quantization function Q(w) = s*round(clip(w/s, MAX, MIN)) (I assume this paper uses the classical form of this quantization function as the author failed to provide explicit details), which is originally a QAT method, it would be ideal that (a) some comparison of loss, accuracy, reconstruction errors, etc. of QAT and/or (b) the STE optimization results over the calibration set is presented. So far, it is hard to see which part of the method is contributing to the most accuracy.
2. This work seems to be heavily based on _Learned Step Size Quantization_, a well-known work in the field of quantization. It seems to me that the only work this paper made is directly adapting this method to the field of PTQ and showing that it is working well with both limited data (PTQ calibration) and module-wise REM. Due to the lack of ablation study in point 1, the novelty seems to be limited as this paper failed to present how significant their optimization methods could bring benefits to LSQ.
questions: As listed under "Weaknesses".
limitations: This paper provides a decent discussion of the limitation.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes an approach for post-training quantization of pre-trained language models. The core of the idea is to run the unquantized and quantized networks in parallel for a small calibration dataset, and update the quantized network parameters such that the quantized network activations match the unquantized reference activations at certain module boundaries in the network. Similar to teacher forcing, the accumulated error in the activations as a result of quantization may be periodically reset/reduced, allowing better training in a regime where total accumulated error would otherwise be too large. Additionally, the model may be spread across multiple GPUs during quantization, and queues may be introduced to allow the GPUs to run asynchronously without waiting for stragglers.

The method results in quantized model accuracy that is noticeably higher than a baseline for post-training quantization. Compared to quantization-aware training, the proposed method is faster, avoids memory pressure by using less memory, and has better parallelization (though accuracy for the proposed method doesn't quite match QAT).
strengths_and_weaknesses: A strength of the method is its performance across the four key axes identified in the paper: training time, memory overhead, data requirements, and accuracy. The paper provides a clear description of the base method and how to parallelize it.

In terms of the weaknesses of the paper, I would appreciate the authors' clarification regarding the Questions section below; as it stands I have some confusion on how to interpret the results.

The paper also presents REM as the most suitable baseline for PTQ, but the accuracy results (Table 3) instead feature GOBO and no REM. It would be better to have more thorough experiments or exposition instead of this jump. It would also be helpful to have a PQT baseline in Table 3 that runs on more than just two GLUE tasks, because this may be too few to draw good comparisons of MREM accuracy vs. other PQT approaches.

Minor:
- superscript formatting for 1e-4/5e-5 on lines 206-207 is awkward to read (e^x usually means exp(x))
- 103: misspelled calibration
questions: Could the authors please explain when quantization with respect to being before/after/joint-with task-specific fine-tuning?

My initial assumption would be that PQT occurs after task-specific fine-tuning (though I don't think this is ever stated explicitly in the paper, so I could be wrong). However, Tables 1&2 include a "full-precision" row alongside the quantization approaches, and MREM uses less time/memory/data than full-precision fine-tuning. These numbers wouldn't be measuring the same quantity if MREM is run only after full-precision fine-tuning has already occurred. I am further confused by what "time" means in the QAT context. With QAT it's technically possible to run the entire task-specific fine-tuning with QAT enabled, or to first fine-tune a bit in the standard full-precision way and then add QAT and optimize the task loss some more. It's again not clear to me what is being done, and what columns like "time" actually represent.
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a technique named MREM to quantize a PLM without QAT, which requires huge resources (time, GPU memory, number of samples). Specifically, MREM adopts reconstruction error minimization in a module-wise manner. Each module consists of several Transformer layers, and each module is trained (almost) independent of the others by either taking full-precision output or the actual output from the previous module. To fully utilize such module-wise behavior, the paper also suggests dividing modules into each device. In addition, annealed teacher forcing is also used to simulate the error propagation through layers. The experiments demonstrate the savings (=advantages) of MREM for various resource types. Results of GLUE and SQuAD datasets show that MREM outperforms vanilla REM and reduces the gap between QAT and PTQ.
strengths_and_weaknesses: The paper is well written and easy to follow. Especially, the comparison between QAT and PTQ in Section 2.2 provides good motivation for the paper. The experiments are very well organized and support the advantages of the proposed method. Previous works are also sufficiently addressed.

Teacher forcing seems to be a good approach to dividing modules and performing separate optimization for each. The linear annealing schedule is reasonable, and the authors sufficiently support the necessity of the teacher forcing by experiments.

However, as the authors pointed out, the key idea that divides the network into several modules and applies reconstruction error minimization is already proposed in BRECQ [30]. There is some novelty in this work, but the improvements are somewhat incremental yet practical. Furthermore, there is no comparison between MREM and BRECQ in the Tables.

questions: The authors position MREM as a type of PTQ. However, I’m curious if it is okay to view the method that includes parameter update as a variant of PTQ. If this is a widely accepted classification, please ignore this question.

Can you compare the QAT case that only use the same number of (4K) samples? In that case, the resource usage gap will be much decreased, but how about the accuracy gap? Furthermore, if we allow using 4 GPUs to reduce the memory usage, can QAT also use multiple GPUs (reduce the batch size or employ model-parallel) for a fair comparison?

limitations: The authors addressed the limitations of this work.
review_title: null

Meta-review: metareview: In this paper the authors propose a practical method for post-training quantization (PTQ) of language models that divides the parameters and quantizes each separately (following BRECQ) in parallel, with asynchronous updates and a teacher forcing method to reduce error propagation. They show improvements on GLUE and SQuAD benchmarks. Reviewers agreed that the paper represents a solid practical contribution with convincing results, advancing the LM PTQ literature. The authors did a good job of addressing concerns and providing further analysis in the rebuttal. 
confidence: Certain
Decision: Accept