Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes "Back-to-Back" regression for estimating the causal influence between X and Y in the linear model Y=(XE+N)F, where the E denotes a diagonal matrix of causal influences. Furthermore, this work theoretically shows the consistency of B2B and the experiments also verify the effectiveness of this approach.

The writing is well and clear and there are some minors issues:
- Further analysis and explanation for using \hat{E}=Diag(H)to estimate the causal influence might be needed.
- The model defined in Fig. 1 seems the influence E should have a sparse diagonal vector. It is possible to introduce an L1 regulation in E?

##############
After reading the author's feedback and the comments from other reviewers, I keep the current rating but tend to a borderline score and it is ok if it must be rejected because of the concerns of limited applicability and the experimental.

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The authors introduce a method (B2B) for disentangling effects of correlated predictors in the context of high dimensional outcomes. Their model assumes the outcomes are constructed by a linear transformation of a set of true causes plus measurement noise. Specifically, they fit the model Y=(XE+N)F, where X are the predictors, E is a binary matrix indicating the true causes, N is a noise term, and F is a mixing term. They provide a closed form solution the model fit based on a pair of l2-regularized regressions. They simulate from the given model and provide comparisons against least-squares regression, canonical correlation analysis, and partial least squares. They also apply the method to brain imaging data containing 102 individuals reading 120 sentences plus scrambled sentences, with the goal of inferring which features of the words have an effect on imaging results.

This paper appears to be technically sound, but it should be rejected based on 1) the relatively limited applicability of the model and 2) a lack of thorough experimentation indicating that this is an appropriate method under more general circumstances. It is odd that the model assumes the outcomes are measured without error. Instead, it is assumed that the causes are measured with error, and mixed via F. A more appropriate model may be: Y=(XE+N)F+M, where an additional noise term M allows for Y to be measured imprecisely. Viewed in this light, the model starts to look a lot like canonical correlation analysis. Consider a model Y=ZF+M, X=ZG+N, if dim(Z) = dim(X) and G is invertible, this can be re-written as Y=(X inv(G)-N)A+M, and we arrive at a similar model with specific restrictions about the structure of inv(G) (E will in general not be invertible, so they are not the same). It is particularly odd, then, that the authors do no comparisons against any regularized form of CCA, which would seem to be the most natural method to use in this circumstance. Moreover, in the simulations where they show B2B outperforms CCA, they use 1000 training samples with 10-100 possible causes. In their experiments on real data, where it seems the CCA results are much closer to the results that B2B gives, they have 2700 samples and 4 possible causes. This seems to imply the real data case might be better conditioned than the simulated case, so that regularization would have less of an impact.

In conclusion, the authors present a sound method for disentangling correlated possible causes when the outcome is high-dimensional. However, the authors do not provide enough evidence that this method is generally useful and better than established methods to merit acceptance to ICLR. A comparison to regularized CCA, application to more datasets and simulations under violations of the model would greatly improve the paper. I also have two minor points. 1) the word “causal” can mean many things, and here it refers specifically to disentangling correlated predictors, rather than confounding in observations or direction of effect. It would improve the paper to add some discussion of this point. 2) The comparisons in the experiments are done between E estimated from B2B and E=sum_j H_j^2 for other methods that do not directly estimate E. However, a more natural comparison might be against EF as this also includes estimates of the strength of influence of each observation, which is implicit in the sum above.

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: 
The paper provides an iterative linear method to identify causal influences of putative cause matrix to signal matrix. The idea is a natural extension of previous forward and backward such as CCA and PLS. The paper has provided consistency guarantee and several synthetic and real data experiments as support. 

Technical questions:

1) The estimation of binary causal influence matrix E is set as \hat{E} = Diag(\hat{H}). Why is Diag(\hat{H}) guaranteed to have binary elements?

2) The Theorem 1 needs more explanation about why it proves consistency, which is currently isolated from other parts of the paper. Why E\hat{H} = \hat{H} guarantees the consistency of \hat{E} = Diag(\hat{H})?  For example, \hat{H} can have more all-zero rows than E which still satisfies E\hat{H} = \hat{H}  but  \hat{E}  is not equals to E. In an extreme case,  \hat{H} = 0 will have  E\hat{H} = \hat{H} but is clearly not consistent. 

3) How does the Eq (4),(5) give an estimation \hat{E}?

4) By Eq. (10,11), H and G seem to be determined give X,Y. Then what are the maximization of Eq. (12,13) over? 


General comments:

1) How does the problem in Eq.(1) differ from variable selection in linear regression where a plenty algorithms exist such as LASSO, spike-slab prior, SCAD, etc. ?

2) The experiments are a bit weak with a simple synthetic experiment and a real dataset with just four features. Can the experiment directly demonstrate the correctness of the theorem?

Typo:

Page 1, “are are based on”

In general, I think B2B is an algorithm that has improvement over CCA and PLS. I am looking forward to the author response to address my above concerns.

##############
I have read the author's feedback which addresses some of the confusion parts in the paper. I maintain the current rating mainly because of the experimental strength. 
review_title: Official Blind Review #1

Meta-review: The authors introduce a method for disentangling effects of correlated predictors in the context of high dimensional outcomes. While the paper contains interesting ideas and has been substantially improved from its original form, the paper still does not meet the quality bar of ICLR due to its limitations in terms of limited applicability and experiments. The paper will benefit from a revision and resubmission to another venue.
Decision: Reject