Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The setting of the paper is about making predictions from predictions -- given a deployed model and decision subjects may best respond to it, how will their target variable Y change. The goal for the authors is to predict the impact of a new model deployment before actually deploying it. Instead of trying to come up with an optimal solution (e.g. performative optimality), the authors are interested in understanding the underlying causal mechanism of the distribution shifts. 
strengths_and_weaknesses: I find the problem of identifying $E[Y| do (\hat{Y} = y), X]$ in the performative prediction setting interesting. Overall the paper makes a novel contribution by providing sufficient conditions to identify the causal effect of predictions under some assumptions. The relation to the literature on spillover effect/social network analysis is also interesting.

To me, the major weakness to me is that the paper seems to make heavy assumptions in order to have interesting and clean results. For example, the whole paper is built on assuming a particular causal structure (Figure 1),  and the definition of the extrapolation error allows a clean separation between the influence of $X$ and $\hat{Y}$, and in section 3.3, the author assumes the effect of $X$ on $Y$ and $\hat{Y}$ is separable as well. Thus it wasn't clear to me how likely this work can be extended to more complicated settings. The authors provide some empirical justification, but I would like to see some theoretical insights on how the results would change/not hold without certain assumptions, and I believe it will greatly strengthen the paper. 
questions: Can the author provide some insights on why overlapping is easier to achieve if $|\mathcal{T}|$ is small? In general, can we boost the overlapping by including more classifiers? If the underlying causal model is more complicated (e.g. have more variables $X_1, X_2..$ or more complicated causal relationships among each other), what will be a good way to ensure the identifiability of the causal structure?
limitations: Please see the strength and weaknesses section.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This work investigates performative prediction through the lens of causal inference, aiming to identify causal structures which allow for counterfactual estimation of performative effects.

Investigation in this paper assumes a specific causal structure, in which the predicted value $\hat{Y}$ acts as a mediator between the chosen prediction model $f_\theta$ and the outcome $Y$, and $f_\theta$ does not directly affect other variables (Figure 1). For the theoretical analysis, an infinite-sample setting is assumed, where it is assumed that the sample size is infinite. 
Three sets of theoretical results are presented:
* The first set result assumes a specific performative structure, and quantifies the regret from naively training a model under performative behavior (Proposition 1,2). The result aims to illustrate the negative effects of neglecting a performative causal structure. 
* For the second set of theoretical results, authors identify “overlap/positivity” as the main limiting factor in performative extrapolation. Authors point out three avenues through which this concern can be alleviated - randomization of model outputs (Proposition 3), noisy measurement of covariates (Proposition 4), and incongruence between the effect of $X$ on $Y$ and the effect of $X$ on $\hat{Y}$ in separable performative structures (“over-parameterization” in Proposition 5, discrete classification in Proposition 6).
* For the third set of results, authors initiate investigation of spillover effects between users, identifying a spillover structure $G$ through which identifiability is possible.

In the experimental evaluation, authors present results which validate their claims, showing empirically that results extend beyond the theoretical guarantees.

strengths_and_weaknesses: Strengths
* Presentation and mathematical notations are very clear. Causal assumptions are made explicit.
* Identifying “failure modes” and “success modes” of performative prediction is an approach which can establish strong theoretical foundations for this area of research, and help it further establish applicability in practice.
* The investigation points out avenues of research which may be interesting for further inquiry, such as the relation between model over-parameterization and performative prediction, or the significance of spillover effects.

Weaknesses
* The paper claims to address the general problem of "predicting from predictions", but in practice assumes a very specific causal structure and does not sufficiently establish its applicability. In particular, the paper assumes a causal model in which $f_\theta$ does not affect $X$. However, a direct causal path between $f_\theta$ and $X$ exists in many practical cases, for example in the “Actionable Recourse” setting (Uston et al. 2018). In section 2.1, the authors mention many applications in which a performative structure may be present, but it’s not clear whether they can indeed be approximated using the causal structure presented in Figure 1.
* The paper presents a collection of interesting insights, but it’s not made clear how they combine into an understanding of the “bigger picture”. Moreover, even though similar problems were investigated in previous literature (e.g “Counterfactual Risk Minimization: Learning from Logged Bandit Feedback” by Swaminathan and Joachims 2015, recent work on over-parameterization and OoD generalization in deep neural nets), these existing results are not discussed in the paper.
* Theoretical results assume that sample size is infinite, and that predictors are minimizers of the expected risk (i.e $h^*=\arg\min_{h\in\mathcal{H}} \mathbb{E}[w(h(x)-y)^2]$) - A quantity which cannot be directly optimized practice. Not clear how the formalism extends to the finite-sample setting. What would change when the dataset size is finite?

A few minor typos: Line 263 (bet -> be), line 307 (inequality expression seems incomplete - missing a random variable next to the expectation operator?), line 374 (lesson -> lessons).

questions: * When do we expect the performative causal structure assumption (Figure 1) to hold? When do we expect it not to hold? What are the implications of making a wrong assumption about the causal structure?
* What is the relation between the different propositions in the theoretical analysis? How do they combine into an understanding of a “bigger picture”? Are they "complete" in the sense that they cover all possible influence modes?
* The discussion claims that “one of the most important lessons from this work is that there is high value to logging the state of the deployed predictor when collecting data for the purpose of supervised learning”. I agree with this claim very much, and am wondering if there is a way to support or quantify it.
* What is the connection to existing work on learning under distribution shift? Literature that comes to mind is:
  * “Counterfactual Risk Minimization: Learning from Logged Bandit Feedback” by Swaminathan and Joachims 2015,
  * “Discriminative Learning Under Covariate Shift”, Bickel et al. 2009
  * “Actionable Recourse in Linear Classification”, Uston et al. 2018 
Literature on online learning and multi-armed bandits (e.g can we think of an “$\varepsilon$-greedy” strategy as an inducer of identifiability?)

limitations: * The paper makes a very strong causal assumption, but the current version does not properly contextualize it within the existing literature. I feel that the paper would benefit from discussing its limitations and implications of the causal structure assumption in more depth.
* Extension of theoretical analysis to the finite-sample case, and the relation to existing machine learning literature on similar topics.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper studies the problem of estimating counterfactual outcomes under a different predictive model, thus drawing a link between causal inference and the recent literature on performative prediction. The paper starts with hardness of performativity-agonostic learning and then describes identification results that enable the supervised learning approach to work. The authors also conduct experiments showing that the identification strategies work well in simulations and mild violations of assumptions are not of concern. 
strengths_and_weaknesses: Strengths: The writing is clear and the problem is well motivated and stated. The identification results are clearly stated with enough explanations and examples. Overall the paper is very clear in exposition. The problem studied is also having potential significance in real world examples as non-stationarity arises, which I particuarly like. The experiements are sound with code available for reproducibility.

Weakness: The scope of the work is somewhat limited since the effect of the predictive model is only assumed to affect $
\hat{Y}$ (the authors acknowledge this in section 2.1). When restricting to this particular causal graph, the first two identification results are essentially direct application of existing results, thus limiting the originality of the work. Also, it would be nice to see some real examples in the experiment section. 
questions: 1. In the overparametrization identification result, the author mentions neural network as one of the examples. I am curious about if there is any empirical results illustrating this identification result under this complex model as a lot of the real world models are indeed getting more and more complex.
2. In proposition 4, is $P(U|Y, \hat{Y})$ necessary to know or $P(U|\hat{Y})$ is enough to enable identification?
3. Could you say more about applicability of knowing $P(U|Y, \hat{Y})$ in some real world examples, like which examples you mentioned seem more reasonable assuming this?

Some possible typos: line 293 in main text, $f_\theta$ we will, removing we? line 307, should there be $Y_j$ after the expectation? 
limitations: The author is very clear about the limitation of the work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In the traditional supervised setting, we seek to predict a response $Y$ based on features $X$, say as $\hat{Y} = f_{\theta}(X)$. The question this paper asks is the following: What happens when the prediction $\hat{Y}$ itself influences $Y$? Can we then still rigorously reason about our predictive model? To answer the above question, the paper makes the following contributions:

1) It introduces a causal framework to answer the above question. Instead of the traditional regression estimand $\mathbb E[Y \mid X=x]$, the estimand of interest in this setting is the following:
$$ h(x, \hat{y}) = \mathbb E[ Y \mid X=x, do(\hat{Y}=\hat{y})] $$

2) It establishes three practical situations under which the above estimand is identified from observational data and may furthermore be estimated by supervised learning based on weighted empirical risk minimization. These three situations are the following: when noisy predictions are released (e.g., in differential privacy), when the features $X$ are noisy measurements of latent features $U$, and under separability (along with overparameterization or discrete classification).


3) A semi-synthetic study is conducted based on census data and Kaggle credit scoring data.

strengths_and_weaknesses: # Strengths

* Motivation: The setting and problem tackled in this paper are exceptionally well-motivated. It is clear that rigorous reasoning about the causal effects of predictions is of great practical and theoretical importance. 

* Clarity: The prose of the paper (but not the formal results, see below) is a joy to read and very clear.


# Weaknesses

* Previous work: I believe that the paper misses important previous references. Rigorously thinking about causal effects of predictions and seeking to estimate them is not a new idea, even within the NeurIPS community. For example, the following paper (also see references therein) addresses such a problem, and the underlying method is based on adding noise to the predictors (i.e., very similar to one of the identification strategies in the present paper):

> Wager, S., Chamandy, N., Muralidharan, O., & Najmi, A. (2014). Feedback detection for live predictors. Advances in Neural Information Processing Systems, 27.

* Formal results: Several of the formal results of the paper are not well-stated, have errors (e.g., in the proofs), and make incorrect references to previous work. See points below for a more detailed listing of such issues.

questions: ### Extrapolation error

Please clearly define what extrapolation error is (I do not think it is ever defined). It may be helpful to state the theoretical result (Proposition 1) with the middle term in the inequality replaced by $R_{f_{\phi}}(\psi(f_{\theta})) - R_{f_{\phi}}(\psi(f_{\phi}))$. Even though the 2nd term is $0$, it may be more instructive to express it as above. In Proposition 1, it seems that smoothness and strong convexity refer to the second argument of the loss function. If so, this should be clarified/explicitly stated.

### Overlap

* Front door adjustment: I did not understand the reference to front door adjustment in both the main text and the proof of Proposition 3. For example, in the proof, it is written that the front door adjustment formula is applied to $\hat{Y} \to T_{\hat{Y}} \to Y$. But note that here $X$ is actually observed, and is conditioned upon, so front door adjustment does not seem necessary? Instead it seems that the argument boils down to the fact that in this setting:
$$ \mathbb E[ Y \mid X=x, do(\hat{Y}=\hat{y})]  = \mathbb E[ Y \mid X=x, \hat{Y}=\hat{y}],$$
and the RHS is now estimable/identifiable (while is is not under the conditions of Proposition 2).


* Page 6, Line 226: $P(T_{\hat{Y}} = t \mid X)>0$: This is not correct, this should be replaced by a statement about the conditional density of $T_{\hat{Y}}$, or alternatively, stated as in Assumption 1.

### Noisy measurement of covariates

* "information about the noise distribution of the measurement error $P(U \mid X=x)$": Is the measurement error distribution not specified instead by $p(x \mid U=u)$?

* I do not understand the statement of Proposition 4. What are these weights exactly? They are stated as:
$w(x, \hat{y}) = P(U | X=x) / P(U \mid \hat{Y} = \hat{y}),$
but this does not make sense to me, since $U$ is a latent random variable (and so the above could not be the density at a fixed $u$). Also why does the beginning of the statement state that knowledge of $P(U \mid \hat{Y} = \hat{y}, Y=y)$ is required even though only $P(U \mid \hat{Y} = \hat{y})$ is included in the weight definition?

* Proposition 4: In the proof of Proposition 4, the equality in line (19) is incorrect. 

### Separability:

* Proposition 5: It would be very helpful if the specification of $\mathcal{H}$ in this statement could be more explicit.

* Example 3.1: "inferring $\beta = c_1$ and $\alpha = c_1 - c_2$": I think for $\alpha$ one would infer $\alpha=c_1$ instead of $\alpha = c_1 - c_2$.

* Proposition 6: Here the main text claims that results for the regression discontinuity design are used, but then the actual proof cites a result from Wang and Blei (2019) that is unrelated to the regression discontinuity design. Also in the proof, what is $x'$? Why does supervised learning recover the true effect?

### Minor:

* *The supervised learning approach*: This should be in bold instead of italics (to match the bold of other paragraphs in the manuscript).

* In equation (8), why are the weights allowed to depend also on $y$? Would it suffice if they only depend on $(x, \hat{y})$?

* There is a typo in the statement of Assumption 2.
limitations: Limitations of the framework are not discussed. It would be great if a paragraph could be added with possible directions for future work!
review_title: null

Meta-review: metareview: Strengths:
* problem is well motivated and stated, writing is clear
* interesting and important identification problem
* useful results on sufficient conditions
* causal assumptions are made explicit (see also weaknesses)

Weaknesses:
* very strict assumptions on causal structure (specifically that $\hat{Y}$ does not affect $X$)
* assumptions made explicit, but writing conveys more general claims at the onset
* theoretical results make interesting connections but lack clear novelty
* current version missing a thorough discussion of limitations
* concerns regarding using predictions as a feature

Summary:  
The paper presents an interesting study of the effects of predictions on outcomes. The key contributions are a clean formulation of the problem, several identifiability results, and some corollaries regarding the use of predictions as input for learning.  
Reviewers were unanimous in their appreciation of the paper’s quality—but also in their concern regarding the strong assumptions made. While making assumptions is certainly adequate, some reviewers felt that this narrows the scope of how results should be interpreted. One reviewer was also worried that assumptions simplify the problem in a way that makes the paper’s theoretical contributions derive immediately from know results in causal inference. But the main concern was that, while assumptions were indeed clearly stated, earlier sections seem to present the paper as being more general than it is, thus creating false expectations and deferring a much needed discussion regarding the paper’s limitations that follow from its strong assumptions.  
In the discussion, reviewers were mostly satisfied with the authors’ responses as to how they plan to address the concerns raised. Unfortunately, the authors have only stated what changes they intend to apply, and did not provide reviewers with a revised version. This makes judging these anticipated changes difficult. Nonetheless, all reviewers consider the paper and its contributions favorably; the authors are therefore strongly urged to clearly and adequately frame their paper’s results and limitations, with full integrity, and as early in the paper as possible.

confidence: Certain
Decision: Accept