Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors have addressed my concerns, and clarified a misunderstanding of the baseline that I had, which I appreciate. I do think that it is a solid contribution with thorough experiments. I still keep my original rating of the paper because the method presented is heavily based on previous works, which limits the novelty of the paper. It uses previously proposed clipping activation function for quantization of neural networks, adding a learnable parameter to this function. 
_______________
ORIGINAL REVIEW:

This paper proposes to use a clipping activation function as a replacement of ReLu to train a neural network with quantized weights and activations. It shows empirically that even though the clipping activation function obtains a larger training error for full-precision model, it maintains the same error when applying quantization, whereas training with quantized ReLu activation function does not work in practice because it is unbounded.

The experiments are thorough, and report results on many datasets, showing that PACT can reduce down to 4 bits of quantization of weights and activation with a slight loss in accuracy compared to the full-precision model. 
Related to that, it seams a bit an over claim to state that the accuracy decrease of quantizing the DNN with PACT in comparison with previous quantization methods is much less because the decrease is smaller or equal than 1%, when competing methods accuracy decrease compared to the full-precision model is more than 1%. Also, it is unfair to compare to the full-precision model using clipping, because ReLu activation function in full-precision is the standard and gives much better results, and this should be the reference accuracy. Also, previous methods take as reference the model with ReLu activation function, so it could be that in absolute value the accuracy performance of competing methods is actually higher than when using PACT for quantizing DNN.

OTHER COMMENTS:

- the list of contributions is a bit strange. It seams that the true contribution is number 1 on the list, which is to introduce the parameter \alpha in the activation function that is learned with back-propagation, which reduces the quantization error with respect to using ReLu as activation function. To provide an analysis of why it works and quantitative results, is part of the same contribution I would say.
review_title: This paper proposes to use a clipping activation function as a replacement of ReLu to train a neural network with quantized weights and activations.

final_score: rating: 5: Marginally below acceptance threshold
review_content: The parameterized clipping activation (PACT) idea is very clear: extend clipping activation by learning the clipping parameter. Then,  PACT is combined with quantizing the activations. 

The proposed technique sounds. The performance improvement is expected and validated by experiments. 

But I am not sure if the novelty is strong enough for an ICLR paper. 

review_title: Is the idea strong enough?

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper presents a new idea to use PACT to quantize networks, and showed improved compression and comparable accuracy to the original network. The idea is interesting and novel that PACT has not been applied to compressing networks in the past. The results from this paper is also promising that it showed convincing compression results. 

The experiments in this paper is also solid and has done extensive experiments on state of the art datasets and networks. Results look promising too.

Overall the paper is a descent one, but with limited novelty. I am a weak reject
review_title: Review

Meta-review: All of the reviewers agree that the experimental results are promising and the proposed activation function enables a decent degree of quantization. However, the main concern with the approach is its limited novelty compared to previous work on clipped activation functions.

minor comments:
- Even though PACT is very similar to Relu, the names are very different.
- Please include a plot showing the proposed activation function as well.

Decision: Reject