Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper studies post-training quantization, and propose to leverage inter-layer dependency to improve the post-training quantized network. The authors introduce a number of approaches: AR, ASoftmax, improved initialization and AMixup, and verify the effectiveness of each part accordingly. The experimental results demonstrate superiority over a number of previous strong PTQ baselines across various network architectures and quantization configurations. In summary, despite the paper is methodologically incremental compared with past literature, the experiments are thorough and solid.
strengths_and_weaknesses: 

Strengths:


- The major strength for this paper should be the empirical results, which outperforms a number of strong baselines and pushes the limit of PTQ performance on low-bit quantized networks. The ablations are also thorough and complete.

- The writing is clear and easy to follow.


Weakness:

- The paper is methodologically incremental as a large portion is built upon past efforts (BRECQ and QDROP). Activation regularization resembles knowledge distillation, a widely adopted approach in PTQ approaches utilizing calibration dataset. Asoftmax follows AdaRound and extends the range of discrete space in $h(v)$. Annealing mixup considers to add annealing process into the mixture of quantized and full-precision activations. 
 
questions: - While the paper is entitled with "leveraging layer-wise dependency for PTQ", layer-wise training still achieves the best on 2 out of 5 architectures according to Table 2. I wonder what is the effect to prolong the training iterations? Will it allow the network adjust itself to better leverage the inter-layer dependency, and alleviate the convergence issue (all variables are discretized instead of being left in continuity)?

- The empirical effect of initialization (proposed in Section 3.2.2.) is missing?

- According to Table 7, ASoftmax does not bring consistent improvement with 1024 images, and may even suffer from overfitting (L254). Is there a systematic way to determine the optimization space with the training size? Additionally, have you tried larger discrete optimization space given 10240 images?

- Typo: in L159, $\hat$ is placed improperly for $s\cdot w^0$.

- Equations should end up with "," or ".".
limitations: The authors improves the performance of post-training quantized networks under low-bit configurations. Despite there is still a large gap with the full-precision model, the empirical improvement in this paper is still an important step for futural PTQ research.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: Based on the inter-layer dependency of quantization, this paper proposes a Network-Wise Quantization (NWQ) approach. NWQ arises overfitting and discrete optimization problems. 
In this paper, activation regularization is introduced to handle the overfitting issue. To solve the combinatorial optimization problem,  the authors use Annealing Softmax and Anealing Mixup.
The experiments show that NWQ can achive significant improvements over previous works.

strengths_and_weaknesses: Strengths:
PTQ is widely studied, and the methods proposed in this paper outperfome previous SOTA by a large margin. The paper is easy to follow and the authors provides several ablation study to show the effectiveness of proposed techniques.

Weakness:
The paper shows some originality but is not novel enough. For example, Annealing Softmax and Annealing Mixup are largely based on AdaRound[32] and QDROP[41]. The authors clearly state the difference and demonstrate the effectiveness of modifications, but previous methods play a more important role in the discrete optimization problem.  Besides, the motivation and theoretical analysis of substituting RSeR with ASoftmax are absent.

From finger 3, we can learn the efficiency of NWQ. However, the cost of each iteration is not clear. Does it require less or more computation/memory than previous methods in each iteration.
questions: According to paper of AdaRound and Brecq, the inferred layer-wise frobenius loss is based on the hypothesis that the Hessian matrix H is diagonal, which means that the optimization of each layer are independent. Moreover, Brecq has validated the finer granularity brought better quantization quality, which contradict with the idea of inter-layer dependency in this paper, can you explain more on that part?
limitations: Limited technical contribution. The proposed methods are largely based on previous papers. For the new techniques, no theoretical analysis is provided. For example, no theoretical analysis of ASoftmax regarding the better performance compared with RSeR.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a post-training quantization (PTQ) technique that aims to search for a network-level quantization policy, compared to the previous works that only considers quantization policies on a layer-wise level. To tackle the challenges of solving a larger scale combinatorial optimization problem, and the issue of overfitting, the paper proposes two approaches: Activation Regularization, and Annealing Softmax and Annealing Mixup. Experiments show the proposed approach achieved higher performance especially when quantizing networks to extremely low bits. 
strengths_and_weaknesses: 1/ The insight of this paper is straight-forward but very sensible -- quantization errors can cancel each other, and optimizing with a higher granularity can achieve better performance. More importantly, based on the insight, the authors proposes effective approaches that achieved this.

2/ The paper is clearly written. The derivation and introduction of the techniques are explained very clearly. 

3/ Nice ablation study shedding lights on the hyper-parameter sensitivity of the proposed method. 
questions: When explaining the effect of AR, the author mentioned two roles: 1/ preventing overfitting and 2/ preventing the quantized values deviating from the floating point counter part. I can understand the effect of 2/, but 1/ is not clear to me. Are there more evidence supporting why overfitting is an issue? 
limitations: 1/ Seems that the performance gain is not consistent. The performance gain of the proposed method is significant for some configurations, e.g., MobileNet with 2 & 3 bit quantization. However, the gain is quite small for others, such as MobileNetV2 with 4-bit quantization. Why is that? Is it simply because MobileNetV2 2&3 bit configurations do not have a strong baseline?

2/ The proposed method seem to contain too many hyper-parameters, from the granularity of AR, to initialization scheme, to annealing schedules, etc. How difficult is it to find a good configuration for a new model and new target bits?
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: ### Overall:
To avoid the overfitting problem in quantized networks, the authors present several training strategies based on the previous Post-training Quantization (PTQ) methods, such as layer-wise activation fitting as proposed in BRECQ [1], Annealing Softmax (AS) to enlarge the searching space of AdaRound [2], and Annealing Mixup (AMixup) to smooth the training process of QDrop [3]. They conduct thorough ablation studies and experiments on several mainstream networks to fully verify the superiority of the proposed methods. 

### Reference:
[1] Brecq: Pushing the limit of post-training quantization by block reconstruction. ICLR2021

[2] Up or down? adaptive rounding for post-training quantization. ICMLR2020

[3] Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization. ICLR2022
strengths_and_weaknesses: ### Strengths:
- The experimental results seem strong. The authors conduct extensive experiments on ImageNet with different bitwidth. Especially, the ablation studies are quite clear, which should be encouraged.
- The technique part of this paper is well written. I personally think this paper is simple and easy to follow.
- "Annealing Softmax" seems new in the community of model compression.

### Weaknesses:
- Please refer to *Questions* and *Limitations*. The main concern is the novelty of this paper. 
questions: ### Major concerns:
- It seems that the proposed Activation Regularization (AR) shares the same idea with BRECQ [1]. The ablation study on layer-wise/block-wise/stage-wise fitting can also be found in [1]. Why do the authors list AR as the contribution of this paper? What is the definition of "inter-layer dependency"? What is the relationship between "inter-layer dependency" and the overfitting problem?
- The proposed methods heavily rely on hyper-parameter tuning, as reported in Section 4.2. I do notice that the authors conduct ablation studies in most settings. However, I doubt the search space can be too large. Did the authors apply the same setting (i.e., same AR, $\tau^T$, $n/m$, $P_s$) to all experiments reported in Table 1? Besides, did the authors conduct ablation studies using the validation set and also report the final results on the validation dataset in Table 1? 
- It is interesting to see that the $59.14, 26.42, 48.49, 62.85, 41.17$ widely exist in Tables 2-8. How to determine the best setting of AR, $\tau^T$, $n/m$, and $P_s$ simultaneously?

### Minor concerns:
- It seems that the selection of $n/m$ is quite sensitive to the size of the calibration dataset. Besides, the authors may further explain the reason for using $0/1$ instead of $-1/2$.
- I am not sure that it is reasonable to use 10240 calibration images in the circumstances of post-training quantization. It would be better to include a simple baseline utilizing naive Knowledge Distillation (KD) based on 10240 images to fine-tune a quantized network.
- Table 5: I am curious why ASoftmax with an integer range of $\\{-1,0,+1,+2\\}$ outperforms RSeR [3] with $\\{-1, 0, +1\\}$ in 2-bit. Note that 2-bit weights only contain $\\{-2, -1, 0, +1\\}$. The results listed in Table 5 seems to indicate that ASoftmax turns some $-2$ and $-1$ into $0$ and $+1$, is that correct? It would be better to include $n/m=-1/+1$ to make a fair comparison with RSeR.
- The QDrop [2] results reported in Table 4 seem inconsistent with Table 1 and Res18 AMixup with 59.12% Top-1 Acc. is slightly lower than Table 2.
- The ablation study on the **Initialization** of ASoftmax seems missing.

### Reference:
[1] Brecq: Pushing the limit of post-training quantization by block reconstruction. ICLR2021

[2] Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization. ICLR2022

[3] Up or down? adaptive rounding for post-training quantization. ICMLR2020
limitations: ### Comments on limitations:
As claimed in the **Abstract**, "This process pays little attention to the dependency across the sub-nets, hence is less optimal." However, the main idea has been rarely discussed in the current manuscript, which makes the submission serve more like a technical report (i.e., the descriptions and effect of different improvements). The authors are encouraged to solve the major concerns in the final version.
review_title: null

Meta-review: metareview: This paper studies post-training quantization by proposing Network-Wise Quantization (NWQ) an end-to-end quantization approach that takes into account relationships between layers rather than treating layers independently. Using this approach, the paper demonstrates compelling empirical gains across a number of architectures and compression factors. Reviewers recognized the practical success of the approach as demonstrated by these empirical results and praised the clarity of the manuscript. However, there were concerns regarding the novelty of the approach and whether the proposed method is simply a composition of previous methods. While I understand these concerns, I think there is a significant delta between this work and previous approaches, especially when taking into account the markedly improved performance and the challenges of determining how to apply these lines of thinking to end-to-end training. The authors also expanded their discussion of these works in their updated manuscript, clarifying the differences. There were also concerns regarding the hyperparameter tuning, but the authors clarified in their response that the large majority of experiments used a constant set of hyperparameters, suggesting that these results are not simply the effect of tuning. Altogether, I think this paper makes an impactful contribution and will be a valuable addition to the conference.
confidence: Certain
Decision: Accept