Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a synthesis between literatures on using matrix equations to characterize causal effect identifiability, and using proxies/surrogates to obtain identifiability when the effect of interest is not identifiable using the canonical ID algorithm. The authors highlight examples of identifiability under the various identification paradigms. They then describe notation used for their synthesis and highlight how it applies to each of the known cases. The main results entail an algorithm for determining identification of a broad class of effects via the matrix equations paradigm. The authors show that some key existing results, which had been proven sound, are not complete and show that their approach, while not yet proven complete, generalizes the existing approaches for the literatures studied.
main_review: As is the case with all identification papers, this one is dense and a lot of the notation is hard to parse/get lost in. Unfortunately I don't have any advice for the authors on how to simplify the notation. It is quite clear the authors have already thought about this very carefully (section 2 stuck out to me as a particularly efficient characterization of the necessary notation). I found that the paper flowed nicely and the building of one idea off the other was generally reasonably easy to follow (or, rather, as easy as it could be for this sort of paper). I also appreciated the authors' attempts to add clarity through examples and illustration, such as with the Euler diagram in Fig. 2. 

Candidly, this paper relies pretty heavily on 4 papers (citations 9-12) I am only familiar with at a high level and so I do not know that my review will serve as the best assessment of this paper's quality.

Detailed comments for the authors to consider follow:

**Presumptions about audience knowledge** -- given that this paper is under consideration at NeurIPS (a wide and not necessarily deep causal audience), I suspect many ideas from background won't be well-known or well-understood. Despite the authors' clear attempts at clarity, there are some opportunities for further improvement. E.g. In line 56/57, the authors state that Px(y) is not identifiable in Fig. 1d by either the proxy approach nor the matrix equations approach. Why is this the case? What is the hedge/district/(insert poorly named, non-intuitive graphical construct from the literature) that "witnesses" non-identifiability here?

This also applies to the statement in line 194. What is it about the structure of the referenced graphs that makes it infeasible to use the approach described in lines 186-193?

**Explanations of Math** -- Similar to the above point, there are cases where it would help to explain a little bit more clearly what is going on from step t to step t+1 in terms of classical notation. For instance, for the chain rule below line 119, I had to scribble on my copy of the paper for a few minutes to say "ok...so they split out b like that...then...??...they can sum it out...then..." to be able to verify my understanding (on the other hand, once I had that understanding, I was basically able to prove Lemma 1 myself so perhaps it's your intention that there be more of a learning curve?). My opinion is that when you're introducing a new (or not-widely-used) notation, it's important to make a bit more clear what's going on by not skipping as many steps. 

**Invertibility Assumptions** -- Throughout it was somewhat hard for me to grasp the interpretation of (non-)invertibility. Is it the case that non-invertibility --> non-identifiability? I don't think so because the results here are not complete. Some intuition here would be helpful, especially in the exposition-y sections (mid-section 3 is where I've written this question in the margin).

**Line 165** -- the expression below line 165 is hard to read, mostly because of all the alternating between \ and /. Is the / meant to represent some sort of division (multiplication by inverse?)? The proof of Lemma 2 in the appendix didn't shed any light on this.

**Section 6** -- I actually very much appreciated the line-by-line description provided throughout section 6. This is missing from most ID-related papers.
limitations_and_societal_impact: Not really, but this paper is so theoretical that I don't really think it matters.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper considers causal inference in the discrete case, where probability axioms can be represented with matrix equations. Specifically, the paper characterizes the relationships between certain graphically driven formulae and matrix multiplications. With such characterizations, the authors then broaden the spectrum of proxy variable-based identification conditions and further propose novel intermediary criteria based on the pseudoinverse of a matrix.

main_review: This paper studies a very important problem: causal inference with matrix equations in the discrete case. Such matrix equation characterization has been used in previous work, e.g. in [12]. So, with respect to the matrix equation, it is not a novel characterization. Instead, the novelty of this paper is that it broadens the spectrum of proxy variable-based identification conditions and proposes intermediary criteria based on the pseudoinverse of a matrix.

Some parts of the paper are not very easy to follow and I did not check the proofs.  A question: whether such generalized proxy-based conditions can be extended to the continuous case, like that in [12]?

There is no experimental validation in the paper. To show the effectiveness of the proposed algorithm, I suggest that the authors add some experiments.

Updates:
I have carefully read the authors' rebuttal. I understand it is not so intuitive to solve my concerns, but I still think the authors should take these concerns more seriously. 

limitations_and_societal_impact: yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Two primary methods exist for graph based identification. The first relies only on the assumptions in the graph while the second exploits assumptions  between unobserved confounders and the observable variables via proxy variables. This paper develops a new approach by combining both these methods. The new approach thus developed is able to solve problems that are not solvable by any of the methods alone. 
main_review: “We may employ conditional to emphasize…” Line 80, page 3. This sentence is not clearly worded. 

It seems like P_{a,B}(C,d) should be a |X_C| x |X_B| matrix instead of the |X_B| x |X_C| as specified on line 114, page 3.

“A special case of the lemma is appeared in” Line 125, page 4. Include a reference to double proxy setting on page 6 where MGT is described. 

Section 3: Your audience will already be familiar with the usual (non-matrix) chain rule, adjustment criterion etc. Why not replace this whole section with a table with the usual definitions in the first column, matrix definition in the second and a toy example for the  matrix definition in the third column?

Lines 146-161: Please try and reword this part. “As a result, we can obtain...” I am having difficulty understanding the last two sentences. An example with matrices would have been very helpful. 

A careful checking of proofs and claims in this paper will require an effort spanning multiple days. Unfortunately, this wasn’t possible. This work seems to be apt for a journal where it would benefit by multiple rounds of review by the same set of people. Additionally, there will be no space constraint in a journal. So the authors will be able to include examples and results, all in one place thus making the paper easy to follow. 

The referencing style is not uniform throughout the paper. In the abstract, it is names such as Lee & Barinboim, 2020, Kuroki & Pearl, 2014 and Miao et al 2018. In the paper the authors revert to numbers. Other relevant paper that precede the work by Kuroki & Pearl include,

Greenland, S.andLash, T.(2008). Bias analysis.In Modern Epidemiology (K. Rothman, S. Green-land and T. Lash, eds.), 3rd ed. Lippincott Williamsand Wilkins, Philadelphia, PA, 345–380

Carroll, R.,Ruppert, D.,Stefanski, L.andCrainiceanu, C.(2006).Measurement Error in Nonlinear Models: A Modern Perspective. 2nd ed.Chapman & Hall/CRC, Boca Raton, FL.

limitations_and_societal_impact: not applicable
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper considers the causal effect identification problem, and proposes an approach making use of both graphical criteria and matrix equations. More precisely, the authors connect the graphical and matrical approaches by characterizing matrix equations of probability distributions driven by graphical constraints in a causal graph. Building on this new characterization, the authors generalize proxy-based criteria and devise novel intermediary pseudoinverse criteria so as to identify a causal effect by utilizing the general inverse of a matrix and diverse collection of distributions.
main_review: This paper builds up on the work of [Miao et al.], and generalizes their results to some extent. They specially draw attention to the following points: 1) in certain circumstances, the invertibility condition of the matrices in causal identification can be dropped given certain d-separation relations, and we can work with pseudo-inverse rather than inverse of these matrices. 2) Matrix factorization of the desired causal effect can be extended to more than two factors (as in [Miao et al.]). 3) the causal identification problem when partial data or conditional distributions are provided is a line of research that needs more attention.

The authors relax some of the assumptions, namely invertibility of certain conditional probability matrices and also some d-separation relations, and show that the causal effect is still identifiable under certain conditions.
They provide a generalization of the “MGT”. However, the extra conditions they need in theorem 2, namely the surrogate set, converts the problem to the original one.

It would be better to include the definition of an “admissible set”, or at least refer to the works by Pearl or Greenland et al.

They are assuming throughout the paper that the data is from an interventional distribution P_r. I understand that this is to show that the results are general and the data doesn’t need to be observational, but this could be pointed out as a remark rather than including the irrelevant notation r, P_r, etc. in every result which reduces the readability.

The idea of the paper and the big picture sounds interesting, but the contributions on the theory side are not major in my opinion. The results are mainly based on a few examples and only one-way theorems that indeed generalize MGT in a way, but do not contain solid improvements/contribution.


 The big picture and the motivation are interesting, but the contribution and the generalization scheme are not profound enough. It sounds like a rephrasing of the main problem in a more general case but with conditions that converts it to the main case with less effort. 
limitations_and_societal_impact: Please see the main review.
review_title: null

Meta-review: This paper aims to unify (everywhere) identification theory based on graphical models, and (generic) identification theory based on proxies.

The paper was very difficult to follow, but after a fair bit of discussion, reviewers saw merit in the results presented, despite the difficulty with presentation.
Decision: Accept (Oral)