Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes an interesting and potentially quite impactful and valuable idea, which I believe is novel.
The idea is: instead of specifying invariances by hand in the architecture of a network, we can instead specify a set of possible invariances, and regularize the model to favor more invariance.
The authors describe how to structure and regularize a DNN in this way, and provide proof-of-concept experiments.
The experiments show that the proposed method outperforms networks that are fully-invariant or non-invariant when the true data is partially-invariant.

Unfortunately, there are a number of weaknesses which lead me to recommend against acceptance.  In no particular order:
1) The crucial "unseen is forbidden" hypothesis is vague and seems to be a bit of a strawman.  
2) The framing of the paper seems to oversell the method in a way that makes the contribution less clear. 
3) The writing is not very clear.
4) The experiments seem to be only proof-of-concept in scenarios where the method is designed to work.  
5) The method seems to incur an exponential cost, but this is not discussed.

Elaborating:
1) The authors claim that, because DNN behavior is undefined on unseen datapoints, the "unseen-is-forbidden learning hypothesis is currently preventing neural networks from assuming symmetric extrapolations without evidence."  This claim is stated in various forms several times, but never made very precise, and it is crucial in motivating the authors' approach.  Roughly, I take the authors to be claiming that (i) the correct way to "extrapolate" is to assume that: transformations that were not observed to change the target distribution should be assumed to NOT change the target distribution, (ii) DNNs will not extrapolate in this way by default, and must be explicitly designed to do so.  
These claims (or whatever the authors actually mean) need(s) to be stated explicitly, and with appropriate modesty.  After all, both (i) and (ii) seem contentious.  
The claim about an "economical data generating process" supports (i), but is itself somewhat vague and dubious, and should be discussed in the introduction as motivation for (i).

2) The authors claim that their method can discover invariances without any data supporting them.  And their abstract claims: "Any invariance to transformation groups is mandatory even without evidence, unless the learner deems it inconsistent with the training data."  But in reality, the authors specify a small number of possible invariances which the method selects among (in a soft way).  And the data is used to guide this selection process.  So in reality, the designer is in charge of specifying a (restricted) set of (possible) invariances.  So like previous works on enforcing invariances, it places a  burden on the designer to identify plausible invariances.  Overall, I found the framing in the work to be "the model discovers invariances by itself without any data!" whereas a more neutral version would be "instead of enforcing a set of invariances, we propose a set of *possible* invariances, and assume that any input transformations that are not observed to affect the label should be enforced"

3) Besides the above issues (vagueness of "unseen-is-forbidden" and related discussion (1), overselling (2)), there were several other issues of clarity.  The paper is not poorly written overall, but is much harder to read and understand than it needs to be.  Some specific issues are: 
- The results in Section 4 are presented with insufficient context or intuition.  Theorems are stated without any proof intuition and should reference proofs in the appendix.  The intuition for the penalty arrived at (eqn13) is unclear.
- The flow is sometimes unclear.  For instance, "Learning CG-invariant representations without knowledge of G_I. " should be a subsection, not a (latex) paragraph, and should explain what the point of the subsection is before diving in.  The authors seem to be using (latex) paragraphs (i.e. beginning with bolded phrases) as subsections and paragraphs beginning with italicized phrases as (latex) paragraphs.  I suspect the paper was edited to fit into 8 pages without removing sufficient content.  This impedes the flow and sacrifices clarity.
- I think a graph showing the data generating process would be much clearer than the current explanations (e.g. eqn4/5)   
- it is unclear what equation 7 is saying... the text above makes it seem like a definition of a goal, but the following paragraph treats it as an assertion that the goal is possible to achieve.  
...Overall, I recommend stripping out some of the mathematical details and using more words and diagrams in the main text to describe the underlying issues/motivations/methods.
The overall story should be made clearer (e.g. by addressing (1) and (2)), and more space should be devoted to linking each part of the paper into the overall story.  

4) The experiments are synthetic tasks where the correct invariance group is included in the set of invariances being searched over.  I don't think that showing that this method can bring some benefits on a real task is an absolute requirement, given the novelty of the approach.  But without more meaningful results, the paper is held to a much higher standard.  Even for synthetic experiments, these are rather weak; for instance, it would be interesting to see whether/how the method degrades when we consider much larger sets of possible invariances.

5) It seems like the method might require including a set of parameters for each of the possible 2^m invariances.  Is this in fact the case?  If not, why not?  If so, it should be discussed as a limitation. 


-------------------------------
Suggestions/Questions:
- In Section 4 paragraph 1, are G-invariance and G_I-invariance used interchangeably?  This was confusing.
- say what I and D are as soon as they are introduced (top of page 4).
- Typo: "a somewhat a"
- Why a "nonpolynomial" activation function?
- The definition of "almost surely" at the bottom of page 4 is not correct (it is possible to sample probability 0 events), and also it should say that samples of Gamma(X^(obs)/(cf)) (not X^(obs)/(cf)) are equal with probability 1 (these are not the same statement!).
- "level of invariance" and "non-extrapolated validation accuracy", and several other phrases are not defined and should probably be replaced by something more clear and explicit.
- It seems like you might need to assume that that different x^(hid) can't be used to generate the same x^(obs) or x^(cf).  If so, this should be explicit.
review_title: [Official Review]: A nice idea being somewhat oversold; weak experimental evaluation 

final_score: rating: 7: Good paper, accept
review_content: Summary:
A method is given for training neural networks in the presence of a group of transformations, such that the network weights are invariant with respect to any transformation on the inputs which doesn't contradict the training data. Experiments on MNIST and toy sequence data are used to verify that this training method leads to improved extrapolation of predictions to unseen environments.

Strengths:
The method introduced seems promising, as it doesn't require training data to exhibit symmetry, but can still verify (or reject) the invariance of data with respect to a collection of candidate symmetry groups. The training method also seems quite lightweight, and shouldn't require significant additional resources to check for the presence or absence of symmetry.

Although the experiments are a bit limited, the authors include a detailed experiments section in the appendix with a larger selection of baselines and more information about choosing the magnitude of the applied CG-regularization.

Critiques:
The explanation of the results has a lot of room for improvement, and I would recommend the authors revise the writing to follow standard best practices, such as defining/explaining new variables when they are introduced, giving the steps associated with novel algorithms, etc. I give a few specific examples below where this lack of clarity makes the authors' results hard to understand, but there are many other examples of this not listed.

The description of the underlying causal model in section 3 (from the end of page 3 to the start of page 5) is hard to follow owing to a lack of explanation in many places. For example, the overgroups $G_D$ and $G_I$ are introduced without any insight into the distinction between these groups, or what role they play in the context of extrapolation tasks. 

Similarly, the central concept of CG-invariance lacks some crucial details in its definition (Def 2), making the following material harder to follow. In particular, it isn't stated if CG-invariance is defined relative to a specific $\tilde{U}_I$, or else requires Eq. 6 to hold for any choice of $\tilde{U}_I$ (the latent distribution which determines the counterfactual samples $X^{cf}$).

Theorem 3 is difficult to make sense of, with the subspaces $B_M$ appearing at first glance to be circularly defined (the projection in Eq. 9 used to define the $B_M$ is itself defined in terms of these subspaces). The text below and above Theorem 3 helps to interpret this circularity as an inductive algorithm for calculating these subspaces, but it would have been much clearer to define this algorithm explicitly in terms of pseudocode (along with a runtime), and then reference this definition in Theorem 3.

Recommendation:
Although the techniques seem like a timely and useful contribution, the poor presentation makes these techniques difficult to follow, and limits the usefulness of the paper for readers. I'm recommending a weak accept, but this can be improved by clarifying the presentation and making the results easier for readers to understand.


**UPDATE AFTER THE REBUTTAL:** The new material in the paper clarifies things quite a bit, especially the intuitive explanations appearing below Equation 2 and at the bottom of page 4. Thank you for adding that, I have changed my score accordingly :)
review_title: Great idea, but presentation has room for improvement

final_score: rating: 7: Good paper, accept
review_content: POST REBUTTAL UPDATE: I am increasing my confidence in this paper from 2 to 3 - I still believe the paper can use some more clarity but enough points have been explained and updated in the draft for me to feel more confident in my evaluation.  I think the ideas in this paper are quite interesting - for this reason I continue to recommend acceptance.

Summary: This paper describes an approach to embedded invariances in learned neural networks through defining linear automorphism groups. They define a fair amount of theoretical machinery for this task, proposing a model of image generation which includes a number of transformations from these defined groups, and defining the notion of a counterfactual G-invariance for the task. They discuss a method for practically learning a useful invariance even if that exact invariance you wish to have is unknown, by ordering subspaces which may be invariant, and discusses the practicalities of embedding these into neural architectures. Some experiments are described in an MNIST setting and on simulated experiments, showing success at embedding these invariances in toy-ish settings.

Recommendation: I’m recommending acceptance for the paper, since the ideas seem interesting, there appears to  be theoretical contribution and empirical evidence, and it is obviously written with care. My hesitance comes on two fronts: I may be lacking some background in the relevant group theory/invariance literature, and it is hard for me to understand a number of important ideas due to the information density in the writing (a result of ICLR restrictions but also some fault of the authors).

Strong points:
-	The paper is very detailed and interesting – while I am not particularly familiar with the group theory side of the literature, it seems like a good idea from a robustness perspective and the authors lay out their ideas carefully
-	The notion of assuming an invariance unless contradicted is interesting at a high level, and provides some meat to the oft-discussed notion of “extrapolation from a single environment”
-	I appreciate the bridge made from the theory to the practical implementation
-	The experiments mostly back up the point the authors assert in their theory – larger experiments for a mostly theoretical paper like this are not necessarily required 

Weak points:
-	My sense is the authors are having a lot of trouble fitting their ideas into the 8 pages. I sympathize but I also think they can do better on this front – the first two pages can be much more compact, with more space to explicate complex ideas that get swept over quickly
-	Ideas which do not receive enough attention or explanation (but should) include: Eq 2, Theorem 3, the design of neural network weights, and even the bare minimum of experimental details. I know the format is short but some of this stuff is necessary, and I believe the authors can do better in terms of fitting important information in. As it is I am confused about some central ideas, even after checking the appendix
-	The notion of “forbidding examples in the learner’s statistical model” carries some intuitive weight but is not precise – what is this model for a discriminative classifier? This can be more clearly explicated
-	It’s not clear how much is packed in the “economical data generation” assumption, or how that is really connected to the method. Please be more clear.
-	The definition of $T_{{U_D, U_I}}$ is not really clear – need another sentence on this indexing in the main body, as well as discussion of ordering!
-	Def 1: you don’t actually define what it means for 2 variables to be counterfactually coupled, you just show what a counterfactual coupling is. Please reword this definition.
-	Thm 3 – this is incredibly dense and I have a lot of trouble parsing this. I’m not sure how to interpret the direct sum – you’re combining all the subspaces which are supersets of M? Also not sure about the end – this is not G_j -invariant for j \in M-bar. What is M-bar? Is that a Reynolds operator? Then what does it mean for j to be an element of it?
-	How should I pick my groups G_1 … m? Not clear if this is important
-	Bottom of p6: not clear how these neuron weights are specified – does it matter which layer we are in? what does the product of B_m \omega_M,h mean – it looks like a subspace times a real number
-	Sec 5: I really am confused about the relationship between architecture and which invariances can be realizable. There’s not a lot of explanation on this.
-	Sec 6: it’s very hard to interpret anything in this section without the appendix – work more to make it stand alone
-	Proof of Thm 1: maybe I am missing some group theory background but I don’t understand this. For instance, there is no explanation of why this $\tilde{U}_I$ can always be found to couple X^cf with X^obs.
-	Proof of Lemma 1: Again, may be missing some background. But neither x nor $\bar{T}$ is mentioned in this proof, so I don’t know where it is going – please be more verbose.

Clarifications:
-	Middle of p3 – we can “compose rotations and image flips” – do you mean a union of the two sets of transformations? That’s what is shown in the notation but I may be missing some group theory background here
-	Top of p4: not clear how $G_D \cap G_I \neq 0$ is possible – is the idea that $G_i \cap G_j$ might be nonempty?
-	Below Eq 3: “the training data may contaion on a few samples of the variable” – do you mean only a few values of the variable may be observed?
-	Below Eq 3: you use the term environment without defining it, not sure how to interpret it in this context
-	Below Eq 7: should the samples of $\hat{Y}  \ X^{(obs)}$ be Y instead?
-	Below Eq 7: in (i) you say you don’t know the group – do you mean for which group Y should be considered invariant? If so, state that explicitly
-	It’s not clear how the statistical assumption at the end of Sec 3 really fits in with the argument, if you’re going to use it need more here
-	Top of p6 – you say the eigenspace of the Reynolds operator gives us a way to build an invariant NN, but the Lemma is about a linear operator. Need more description here
-	You say the method in Thm 3 is fast but the power set should grow exponentially – is that a problem?

Other feedback:
-	Not sure why you define g as going to the image of the probability – can’t it just be [0, 1]?
-	Not sure “unseen is forbidden” is quite right – wouldn’t it be “unseen is irrelevant” or something?
-	Specify whether Thm 1 only applies to linear automorphisms or if it is more general
-	The recursion at the bottom of Sec 4 is unreadable – just put this in the appendix
-	Bottom of p17 – describe more how the transformations are sampled. I shouldn’t have to work so hard to understand the experiments

review_title: Seems like a good paper, although I found it hard to follow in places (updated)

Meta-review: Pros:
- All reviewers agreed that the idea was particularly interesting/novel. I personally appreciated the perspective of unlearning invariances that prove inconsistent with the training data, rather than learning invariances that are demonstrated by the training data.
- The authors significantly improved clarity during the rebuttal period, and two out of three reviewers raised scores or confidence as a result.

Cons:
- There were significant concerns raised by reviewers about clarity of presentation, and some concern around whether the specific instantiation of the high level idea was the most sensible. From a *lightweight* reading of the paper on my part, I also feel that the writing style is unnecessarily dense, though I believe the underlying ideas are solid.
- One of the reviewers (AnonReviewer4) continues to have serious concerns. I believe the authors and AnonReviewer4 may have both become more entrenched in their positions during the discussion, in a way that wasn't particularly productive.

This paper is borderline score-wise. I believe it is particularly important to reward and encourage unusually novel work. Primarily for this reason I bias my decision upwards, and recommend acceptance.

nit: belive --> believe
Decision: Accept (Poster)