Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The authors propose a method extending Gumbel SCM for doing counterfactual reasoning.
main_review: The paper could have been written a little more clearly in places. For example, Figure 2 did not appear to be linked back to the text. It would have been helpful to have a little more explanation. “Discriminative” was misspelled line 256.

For originality, I haven't seen any other papers like it, though perhaps other reviewers have.

The quality of the theory is clear.

This is significant insofar as counterfactual reasoning is a persistent problem for causal inference. It would have been helpful if the approach had been compared to other approaches for doing counterfactual reasoning. At least a discussion could have been included as to why this covers cases different from what, say, Pearl's methods would have covered.

limitations_and_societal_impact: I don't think this work would have negative societal impact.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose a method to parametrize the functions of a certain class of structural causal models (SCM) called Generalized Gumbel-max SCMs. The aim is to train the model while minimizing the variance of treatment (counterfactual?) effects computed from them. 
The paper introduces two different families of functions (gadgets) that can be learned through an optimization process aiming to find a maximal coupling of a pair of logits vectors defining a discrete distributions. 
The experimental section compares the two gadgets with the Gumbel-max representation from Oberst and Sontag, and a cumulative distribution functions (CDF) inversion method. The experimental results seem to suggest that the proposed gadgets improve may achieve lower variance.
main_review: I am not familiar reparametrization or couplings, but as far as I can tell, their application to causal modeling tasks is novel. It seems that relevant work as been cited (Oberst and Sontag, 2019) and the differences have been addressed.

I am unable to assess the soundness of most of the technical results as they are outside my are of expertise. However, I do have some questions regarding the specifics of the causal task that is attempted which I will discuss shortly. Overall, the paper does seem to support the claims and the experiments provided seem to support the benefits from employing the proposed approach.

In terms of clarity it is hard for me to understand what exactly is the data (observational, interventional) assumed to be known for the task. Also, what is exactly the relationship between the logits vectors and that data? Does this approach work for any type of input data or are you assuming a particular form? I believe a simple running example could help ground these aspects.
The introduction touches upon the different levels of the ladder of causation and the fact that it is not possible, in general, to infer level 3 data solely from level 2 data. Nevertheless, it is not clear what are the level 3 quantities that the authors aim to estimate with minimal variance. 
It is not clear to me what kind of counterfactual is being evaluated. Also, the motivating observation "if we only care about our models being consistent with interventional data, then we have flexibility in the choice of causal mechanism" seems to disregard the level 3-level 2 impossibility mentioned in the introduction. I hope the authors could clarify these points in their response.

In terms of significance, I am unable to assess the applicability of the proposed method, mainly to the lack of a clear picture of the input data, the query of interest and the assumptions being made about the true underlying data generating process. It is very likely that this picture could be understood from the paper but I believe it should be more explicit and easy to grasp for the average reader trained in causal inference. On a related note and in contrast with Oberst and Sontag, 2019, the SCM formulation in the paper seems to assume the absence of latent confounding, is this the case?

## Update
After reading the response from the authors, I feel they have clarified my main concerns. I'm updating my score.
limitations_and_societal_impact: There is discussion on technical limitations.
There is no negative societal impact that requires discussion at this point.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper generalizes the treatment effect estimation, in terms counterfactual and observed policies, of Oberst and Sontag (2019), to couplings. They build upon the Gumbel-max SCMs from Oberst and Sontag and show that with their proposed Gumbel-max coupling, they can learns causal mechanisms that minimize the counterfactual effect variance. 
main_review: I think this paper is quite well written for an audience that has read the paper of Oberst and Sontag (2019). Since I didn't read that paper beforehand, I had no clue what problem the authors were trying to solve. For example, the authors state in the introduction "Casting the problem this way ...", without explaining what the actual problem is, or describing the problem setting there in more detail. This was quite confusing, because the authors talk later about a data distribution that determines observed outcomes and counterfactuals. However, in practice, one has almost never access to those distributions IMO. Once I arrived at the Experiments section, I saw that the authors use simulations to get access to the counterfactual distributions. I think this confusion can be easily tackled by making the actual problem statement clear from the beginning.

Although the paper is well organized and each section is written very clearly, the overall storyline of the paper can be improved IMO. I think that the different sections (and concepts) are too loosely connected. For example, clarifying how the SCM for MDPs are related to the problem that the authors are trying to solve should be very clear from the beginning onward instead of being clarified (slightly) in the Experiments section. Or, on line 12 (and also on line 45) the authors state "We propose a parametrized family of causal mechanisms that generalize Gumbel-max", without showing this in the paper, or at least. I couldn't find it. Are the authors not generalizing the treatment effect instead of the causal mechanisms? This is a bit confusing, after reading what the definition of a causal mechanism of an SCM is.

The actual contributions of this paper could IMO be emphasized more. For example, one of the main contributions of this paper, that is IMO, that seeing the average treatment effect for a counterfactual vs. an observed policy in terms of couplings, could be emphasized more. This contribution is, for example, a bit downplayed by stating (in line 70) that [Maddison et al., Oberst and Sontag] exploit Gumbel-max couplings, which I think they never showed, since the proof of this is actually in the Appendix of this paper and not theirs. Or, in the introduction the authors state the questions about what mechanisms leads to the lowest variance estimates of treatment effects, which they didn't put much emphasis on in the paper, except by stating it somewhere in the related work section. Why not highlight this more and state how optimality and minimizing variance of the treatment effects are connected explicitly. I think these things that motivate this work should really be emphasized and pop out. This can make this paper easier to read and more strong.

At last, I was wondering about how to interpret the main objective? By generalizing the objective from Oberst and Sontag (2019) from difference between the outcomes to some (different) coupling between the outcomes, doesn't that also change the definition of the average treatment effect? That is, can we still talk about average treatment effect in some way? What is the intuition that lays behind using couplings instead?

Although the storyline could be improved, the rest of the paper is quite well written. Under the condition that all the proofs are correct (which I didn't check), I believe that this could potentially be a valuable contribution to NeurIPS.

Minor comments:
 - line 126 "... approach to the Gumbel-max couplings underlying Gumbel-max SCMs": Are they really underlying Gumbel-max SCMs?
 - line 134 "... Gumbel-max couplings are counterfactually stable ...": Where is this shown?
 - line 75: "casual" -> "causal"?
 - Figure 3: What is CRN? Please explain this somewhere.

Update: I updated my score.
limitations_and_societal_impact: I think the impact of this paper can be improved by providing a better intuition about the relation between couplings and counterfactual effect estimation, especially about why couplings lead to more desirable causal mechanisms. Moreover, as said before, improving the overall storyline could make the impact of this paper stronger.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper introduces a continuously parameterized family of causal mechanisms where all members of the family are identical when used in a level 2 context but different when used in a level 3 context. The families contain the existing Gumbel-max technique proposed by Oberst & Sontag (2019), but a wide variety of other mechanisms can be learned by using gradient-based optimization. They show that the causal mechanisms can be learnt using a variant of Gumbel-softmax relaxation and that the mechanisms improve over Gumbel-max and other fixed mechanisms. Further, they show that the learnt mechanism is generalizable from a training set of observed outcomes and counterfactual queries to a testing set of observed outcomes and counterfactual queries.

main_review: Section 1 & 2 & 3. The causal problem is not defined properly. If you are trying to solve the problem of learning causal mechanisms, please define mathematically what is a causal mechanism, what data you learn it from, what is the key challenge in general. I guess you don’t have to answer (not solve) these general questions with the Gumbel-max trick.  After presenting the general causal problem clearly, it will be better to explain the following two issues mathematically or using some toy examples of structural causal models (SCMs): (1) the non-identifiability of level 3 from level 2; (2) the requirement of counterfactuals (mentioned in lines 34-35); (3) Why we need to solve the problem using the family of causal mechanisms that generalizes the existing Gumbel-max SCMs. In the current introduction, Section 2 (Background) and Section 3 (Problem statement: Learning Implicit Coupling), I don’t see a well-defined causal inference problem. I feel like this is a coupling learning paper rather than a causal inference paper.  

Overall, I suggest addressing the following three points mathematically: (1) formulate the problem of learning causal mechanisms; (2) formulate the requirement of counterfactuals; (3) highlight the advantage of learning specifically-tuned causal mechanisms.  I find the sections are well written, but the explanation for counterfactual stability in lines 88-94 is difficult to understand.

Section 4. Why do we need maximal couplings? I guess the answer is on lines 64-69. It looks like the variance of the objective function you will optimize to learn the coupling. How does this variance translate into the variance of the learnt coupling or causal mechanism? For the noise variable u, distributions p and q, could you define their space clearly?

Section 5. I am not very familiar with Gumbel-max and coupling theory. I must confess I don’t completely follow the discussion in this section. It is difficult for me to judge the novelty of the proposed method. Could you point out the novelties more clearly and explain why the continuously parameterized family of causal mechanisms you propose generalizes Gumbel-max?

Section 6. The related works on maximal coupling look good to me. I hope I know it earlier when I am reading through the paper.

Section 7. In Figure 3, you compare the variances of the estimates. Could you also compare the errors of the estimates? 

Summary:
Overall, I find this paper interesting but a bit beyond my expertise. I lower my confidence level and will adjust my score appropriately after understanding the paper better by reading through your feedback and other reviews' comments.

Other comments:

In lines 38-39. Is the variance of an estimator a computational behaviour or statistical property (behaviour)?

Line 57:
You can always draw a sample from a categorical distribution.  What special about the Gumbel-max trick is that you can draw a differentiable sample from the continuous relaxation of a categorical distribution. 

Line 81:
“that map states, actions, and noise” should be “that map actions, states and noise”

Equation (5) under line 115, a typo: “min. \theta”?



limitations_and_societal_impact: Yes, they do.
review_title: null

Meta-review: This paper explores an understudied connection between causal inference and couplings learning, building upon very recent work from Oberst & Sontag. More specifically, it considers the general problem of constructing a SCM that allows counterfactual (level 3) reasoning, when given access to all interventional (level 2) distributions. The key challenge is that there are often many different SCMs compatible with a set of level 2 distributions. To circumvent this non-identifiability issue, they find a SCM with "good" counterfactual distributions based on interventional data, where a "good" counterfactual distribution is one with minimal loss according to some particular loss function. Here, the approach they take is to extend Gumbel-max SCMs with learned couplings.

While the reviewers initially had some concerns re. clarity/significance of the contribution, the authors' replies and subsequent discussion cleared up these concerns and a majority of reviewers agreed that it is worth publishing at neurips.
Decision: Accept (Spotlight)