Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper reintroduces the straight-through estimator with bias-variance analysis. It further discusses its relationship with some constrained optimization methods in convex optimization and 

In general, the novelty of the paper on the methodology side is not high. Its value may lie in the theoretical analysis of an existing method. However, the current theoretical analysis is not clear for the following issues:

- On page 3, it says "cannot interchange the gradient and the expectation in z". But z is a continuous variable, why it cannot?
- There is no clear explanation of what does dL(x)/dx mean. x is discrete, so this notation without re-definition is incorrect. 
- On page 3, it is unclear what "define now dx/da = 2F'(a)" mean. Why should it be defined in this way?
- On page 4 III), why g(x) is assumed to be Lipschitz continuous? 
- On page 8, eq (18), this metric seems strange since the ratio of expectation is not the expectation of ratio. And it only measures the angle between two vectors. Why the L2 norm cannot be used here?

In general, the main paper lacks formal theory statements, while the informal statement does not clearly answer the questions it comes up with.  Another major issue is the writing. The paper does not have a conclusion/discussion part which makes it incomplete. And the main paper has several missing citations/references with (??). In figure 1 caption, it is ELBO not ELOB.  With the concerns listed above, the paper in its current version looks not fully ready for publication. 


=====POST-REBUTTAL UPDATES======== 

Thanks to the authors for the response and the efforts in the updated draft. The updated paper improves writing. The response resolves a part of the queries.  The viewer yet believes the page limits should not be a good excuse to squeeze necessary information into the appendix, otherwise, as AR2 suggests, it may be more proper for other venues.  I raised my rating according to the author's response. 
review_title: The novelty is not high and the writing needs further polishing

final_score: rating: 5: Marginally below acceptance threshold
review_content: #####################################################

Summary:
This paper presents a systematic way of analysing straight through estimators and derive ST estimators for Stochastic Binary Networks. 
#####################################################


Pros:

1. The paper is theoretically strong.
2. The paper covers a large body of relevant work, covering theory behind straight through estimators and stochastic binary networks. 



#####################################################


Cons:
1. The novelty of the paper isn't clear, if this paper is an analysis paper, the empirical evidence is weak. 
2. The paper is very hard to read, and it is difficult to understand the clear motivation. The paper is too dense while missing any key takeaway point. It misses key details in the experiment section. 
3. The utility of the proposed MD estimator is unclear,  it would be helpful if the authors would clarify the interpretation of Table 1 with their write up under "Classification with Deep SBN": the authors state that their method performs as well as the empirical ST, while the table shows it performs worse than their baselines. 
4. Overall, the experiment section is scattered, with hard to understand goals. For example, the takeaway from Figure 1 bottom is difficult to understand. 


#####################################################


Typos:
Line 305 ??
The paper needs major revisions in terms of notation issues: the vectors should be bold, to distinguish the from scalars.
#####################################################


Questions:
Please address the cons mentioned above. 
review_title: marginally below acceptance

final_score: rating: 7: Good paper, accept
review_content: Summary: The paper presents a principled derivation and analysis of the straight-through (ST) estimator, which is often used to train networks with binary weights and activations.

A strong point of the paper is that the ST estimator is often used to train networks with binary weights and activations. Hence, a more theoretical investigation can be helpful to better understand the ST estimator. Furthermore, the paper seems theoretically sound. However, my assessment is unconfident due to lacking expertise in this area. The empirical experiments confirm that the ST performance improves as the number of latent bits is increased as suggested by the theoretical analysis. Another strong point is that the paper will make code available on Github, which can improve the reproducibility of the experiments.

A weak point of the paper might be its limited potential impact on future works since it mainly provides an analysis of prior empirical ST approaches. Furthermore, a substantial amount of content is only described in the supplementary material. A venue that allows longer submission may be a better fit for this work.

Minor important points:
- Missing reference in Line 305.
review_title: I recommend to accept that paper while my assessment is unconfident due to lacking experties in this area.

final_score: rating: 7: Good paper, accept
review_content: 
	1. Summary & contributions
This paper analyzes a number of existing straight through (ST) estimators and connects them using a unified view as estimators for stochastic binary networks (SBNs) with Bernoulli weights. Different estimators can be seen as design choices, either relating to the model or optimization procedure. This provides insights in the specific assumptions (implicitly) made by these estimators and allows to analyze their bias and performance.

The paper proposes an extension of ST for deep SBNs to binary weights and conducts experiments to evaluate the quality of ST gradient estimates, showing that the reliability increases with the size of the (boolean) latent space for a VAE, and providing optimization results for a Deep SBN.

	2. Strengths & weaknesses
Strengths:

This paper unifies a number of straight through estimators under a common framework which helps in motivating previously considered 'ad-hoc' rules for gradient backpropagation, and helps understanding of the conditions which may affect the bias/performance of the estimators.

The paper is technical but written in such a way that it is still relatively easy to follow, by keeping notations simple (yet clear) and deferring detailed derivations/proofs to the appendix (which I did not go through in detail). 

The paper shows theoretically and experimentally that different design choices for the estimator are possible, as long as different parts (model, initialization and training) are well aligned.

Weaknesses:

As a weakness, the paper feels a bit as an enumeration of related but separate contributions (summed up in the paragraph 'contributions'). The experiments mainly concern section 4 and show somewhat the effect of Bias Analysis IV), but it would improve the paper if (toy) experiments would have been conducted relating to the other analysis in section 2.2 as well, which could then justify the practical relevance of these results.

While the paper motivates the reintroduction of ST as principled methods, the experiments give some insights but do not fully convince of the practical use of the proposed extensions.

It is positive that the paper is largely self-contained, but this makes it a bit difficult to distinguish novel derivations from recapitulations of results from previous work. Also, the paper lacks a discussion section and as a result feels a bit unfinished.

	3. Recommendation
My current recommendation is to accept the paper.

	4. Arguments for recommendation
This paper helps in the theoretical understanding and unifying view of a variety of straight through estimators, which may help advancing these estimators for training networks with binary weights and activations, which is a relevant and difficult problem.

	5. Questions to authors
- Why is a discussion section omitted?
- Figure 1: looking at the y-axis for the top row, it seems that both ARM and ST get worse results with 256 bits than 8 or 64 bits, which hints at underfitting. How does this affect the conclusions?


	6. Additional feedback
Minor comments
- Line 152: where does the symbol f for loss come from?
- Line 305: a reference is missing: ??
- Line 331,5: reminder -> remainder?
- Line 372: ELOB -> ELBO? Or -ELBO (negative ELBO)?
- Some grammar could be improved, e.g. in abstract 'we … obtains, …, explains'

review_title: Unified view and analysis of Straight Through estimators

Meta-review: This paper collects a variety of results that cast straight-through estimators as arising as principled methods that make a linearization assumption on the loss for functions with binary arguments. R1 & R3 recommended against acceptance, citing clarity concerns and a lack of novelty. R2 & R4 recommended acceptance, but had low confidence. This paper had uncharacteristically low confidence on behalf of the reviewers, and this is my fault. I apologize to the authors for this. 

I have read the paper myself. I believe that this paper contains many interesting ideas, but I agree with R1 & R3 that the paper suffers from clarity issues. Unfortunately, these issues persist in the recent revisions, despite having been asked by R1 & R3 to improve the clarity. The authors asked for concrete reference points. Here are some:

- "proxy function" is not well-defined, despite being critical to the arguments.
- deterministic ST is not defined clearly before it is discussed.
-  The section structure of Sec 2 could be improved. At the moment it seems to flow from the loss function to the standard ST algorithm through to a disjointed list of questions addressed in the paper.
- The section titles are not particularly informative.
- It is difficult to know which results are known and which results are new.

In general, I believe this work could benefit from a significant restructuring. It would be best to delineate preceding work in its own section, then lay out the new results, making sure that all of the important concepts are clearly defined. I think many of these results are valuable for the community, but the current draft makes it challenging for these great ideas to reach their full potential.

Decision: Reject