Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper uses a structural causal modeling (SCM) framework to define a statistical notion of harm which the authors name counterfactual harm (CH). The statistical definition extends a philosophical one named the comparative counterfactual account, which states that an action causes harm to a person if that person would have been better off had the action not occurred. The paper introduces a harm penalized utility (HPU) objective and compares the optimal actions according to this objective with others including an unpenalized utility and an objective that formalizes risk-aversion (penalizing the second moment) across several illustrative examples. The paper argues that avoiding harm requires causal reasoning, and specifically counterfactual reasoning, and shows that optimal actions according to other objectives can lead to causing harm. It presents several definitions and theorems to generalize this argument from the examples- the theorems show that various non-CH objectives can cause harm under a distribution shift in the data. The paper also illustrates its approach by comparison on a real data example involving dose responses for a drug. 
strengths_and_weaknesses: The paper considers problems which are practically important. It is clear, interesting, and will likely generate a lot of discussion. A strength of its approach is the direct focus on things that truly matter, like utility, but a corresponding weakness is that much of the relevant quantities cannot be observed and require strong modeling assumptions, like the knowledge of an SCM. Note that this is not a specific weakness of the current paper, but one it shares with others that have been published in comparable conferences (including some of the cited related literature).
questions: Q1: Is it really necessary to focus on counterfactuals to define an adequate notion of harm? What if we replace the counterfactual distribution in formula (6) with an interventional distribution? I believe the resulting definition would still be interesting and usable for many of the examples considered. An “interventional comparative account” of harm could say that an action will cause harm to a person if that person will (probably) be better off if a different action is chosen. Since many of the examples in the paper consider actions that may actually be feasible this definition may suffice for them. By contrast, the definition of counterfactual fairness (for example) considers attributes which cannot feasibly be changed, hence it is necessarily counterfactual and could not be interventional.

Q2: Is it too narrow to operationalize harm in a (simple) SCM framework? The model focuses on specific variables, while our intuition about harm usually involves a more open and wholistic idea of the world. Consider the two thieves example in Appendix D. The model specifies that Alice can only choose to rob or not rob Bob, while in the real world Alice could choose to simply warn Bob about Eve. Why should we apply the path-specific definition of harm that ignores an important way that Alice’s action can impact Bob’s wellbeing? I don’t think this resolves the paradox, but expanding the action space can. Similarly, a policy optimized to avoid harm according to a model built on one set of variables could cause harm involving other variables which were not included in the model. I think this is not just a curious theoretical issue because in algorithmic systems the model assumptions can be become built in and the data infrastructure can grow and increase in complexity, possibly cementing in place a harmful set of assumptions. However, the proposed narrow definition of harm would not address such issues.

Q3: Are the authors certain about the claims in the paper regarding the "first" statistical definition of harm? It would require a truly massive literature search to establish this. It seems to me the contribution of the paper is interesting enough to speak for itself, without needing to be claimed as a first in kind. 
limitations: Some already highlighted under the weaknesses, like reliance on untestable assumptions and use of unobservable constructs, and most importantly the first two questions listed above.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper considers defining harm as a counterfactual quantity involving user’s utility. The harm is defined based on the expected difference in the utility following the policy and the utility following the default action. Equipped with the definition of harm, the authors proposed harm penalized utility [HPU] and relate to its implications to decision making. Authors investigated a few properties of the definition(s) and provide a proof-of-concept simulation result in the context of dose-response showing that a policy can be optimized to lower harm depending on the user specified hyper-parameter. 

strengths_and_weaknesses: I would like to thank the authors first for an intriguing notion for harm (and benefit).

Strengths
- As a causal inference researcher, seeing the definition of harm at the level of “counterfactual” not “intervention” is refreshing.

Weaknesses
- It is unclear why we should define harm and benefit separately by taking max(0, difference in utility)
- Simply the P terms in Eq 6 and 7 are not identifiable not only from factual outcome but also from any experimental data. (Line 235–236)
- For the same reason, the authors used an existing model not the data. What would be the implications of using a learned model? A learned model only reflects the factual outcome correctly and will estimate counterfactual harm incorrectly.
- It is unclear how benefit or harm are defined in medical or related research.

questions: 
- In the example in the introduction, I understand that why doctors would favor Treatment 1 vs 2. This feels so much like a trolley problem where inaction can cause 5 people die and action (switching) can cause 1 people die. What’s your explanation? In HPU, U is -5, -1 and harm is -1. So with lambda 4, there is no difference between switching lanes?
- Def 1, why should noise be mutually independent?
- Is utility fixed over the entire population? A is the agent's action but U is user's utility function. User != Agent. (Y would also be user's outcome...) 
- What's the practical importance of Y in defining harm and benefit when you have utility U? Y exists only to be marginalized out. For decision making, how would you use Y? where Y is not observed for the current user and X is given...
limitations: Please see the weaknesses especially about the non-identifiability of the measure. One may show the applicability of new definition under a learned model, etc.
=======
I raise my score which now reflects better understanding of the paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper rigorously define the concept of "counterfactual harm" using statistical analysis. Overall, the authors quantify the "benefit" & "harm" based on a common definition of counterfactual comparative account (CCA). With the mathematical formulation of counterfactual harm, the authors further derive a family of counterfactual objective functions for mitigation. Solid theoretical analysis are provided with the structural causal modeling, and an empirical example illustration is shown to demonstrate the effectiveness of the proposed quantification of counterfactual harm. 
strengths_and_weaknesses: Strengths:

- The topic of this paper is important and significant, which can largely benefit the areas such as fairness AI & ethical AI. The provided quantification method is general, so many down-stream application scenarios could refer. 

- The equations in the main body of the paper is well introduced and clarified, which is easy to follow. 

- The authors provide many contexts and background information which is very useful for readers. The provided high-level examples can help quickly extract the main idea of this paper. 

Weakness:

- The overall paper is not well structured. The section titles are kind of unclear. 

- The analysis setups are too specific, even though the harm definition is general by itself. This could be limited when applying such quantification scheme to a general ML settings. 

- The paper only provides a simple illustration for demonstration, without any real-world experiments & comparison. This makes the proposed work hard to be fully evaluated regarding to its overall performance. 
questions: 1. I'm wondering how we can apply this definition (harm quantification) to a general ML setting, so as to mitigate some potential biases learned in models? Is the SCM a necessary setting for incorporating the proposed family of objective functions? 

2. In a more specific setting (e.g., fairness domain), can we equate or connect the harm quantification to some existing measurements (e.g., counterfactual fairness - Kusner, Matt J., et al. "Counterfactual fairness." NeurIPS'17)? If yes, then what is the key difference for the proposed harm quantification? 

3. Is it possible to design some validation experiments on real-world data for this work?
limitations: 1. The major theoretical analysis & results are purely based on the statistical causal analysis, which may not fit in the settings of standard ML scenarios. Personally, I think the proposed work could be limited since it is hard to directly employ such harm quantification to mitigate the "harmful" patterns existing in commonly used models, such as neural nets. 

2. The effectiveness of this work on wider domains still needs further validation, especially with the experiments on real-world datasets. Current illustration part could be limited to fully evaluate the proposed harm measurements. 

3. The authors should conduct a more detailed discussion between this work and other existing frameworks for quantifying "harmful" concepts (e.g., fairness, privacy). It seems like the authors try to make the definition from a higher level, but we readers/researchers also need to know what is the generality brought by this paper. In the current version, this part is quite limited in my view.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper argues that harm, evasion of which is key for AI to be safe and ethical when deployed, is intrinsically a counterfactual quantity. Further, that conventional ML is guaranteed to end up with harmful policies if unable to perform counterfactual reasoning. As contribution, it claims to be the first to produce a statistical definition of harm, and have derived a set of counterfactual objective functions to robustly mitigate harm. Finally, using a statistical model for indentifying optimal drug doses, the paper demonstrates its framework that integrates harm into algorithmic decisions can dentify doses that are significantly less harmful while not sacrificing efficacy whilst standard algorithms yield significant harm.
strengths_and_weaknesses: The paper is very well written. The social significance of the research in general, together with its arguments, research questions and formal elucidations, are well positioned and articulated.

However, inherent in formal frameworks and models is to abstract or simplify potentially required strong ties to more complex real-world contexts whilst claim generalizability. While a generalized additive model was used to illustrate the counterfactual framework for dose-response predictions, how to scale up to a rather complex domain (e.g., hazardous environment in which human and AI teamp up to navigate the situation) is worth discussing. Further, it is unclear how the causal constructs elucidated here accounted for or will account for human feedback, e.g., Alice's preferences, clinician's sense of values or human demonstrations in general (objective functions, which for example provably beneficial AI methods account for).  While it is not a question for me that this is the first to  provide a statistical defintion of harm and render  it a counterfactual framework, the discussion on related works seems to miss the built up on why the current literature has not addressed it until this time, whether previous works have sought or alluded this reasearch direction and might be falling short, and how close (or far) current deep learning frameworks on counterfactual reasoning relate to this work. 

 


questions: If indeed tha paper elucidated or proved that the proposed counterfactual-based harm-aversed framework can account for more complex harmful situations, it would be good for the authors to make this explicit. The paper also showed how outcome-dependent optimal policy learning happens within the framework. How about the framework being able to learn amid contextual, P(x; M), changes? In otherwords, when distribution shift is not just on the outcome distribution, but also on the input (e.g., covariate shifts) or to both (concept drift)?  Lastly, some support (proof) to qualify certain claims (assumptions), as follows:
1. The  claim that agents trained to maximize factual objective functions are guaranteed to proceed with harmful policies, albeit they are allowed to retrain, amid distributional shifts (L332~L334). For one, Scholkopf et al. [1] argues that change in data distribution "may lead to (arbitrarily) in accurate predictions". Thus, this claim should be situated on the premise that the characterized objective function is pertinent only to domains where there is vulnerability to harm. Secondly, however, why would this claim hold even if the agent can retrain itself?
2. Why was the requirement that agents have some fixed harm aversion relaxed, and tapered to requiring only that the agents do not take needlessly harmful actions?
limitations: On the two brief mentions on limitation, namely, (1) that it remains "an open question as to how counterfactual reasoning can be achieved with current implementations", and (2) "there is growing interest in counterfactual reasoning with deep learning models", it would help to discuss this further in a section on related works.
review_title: null

Meta-review: metareview: All reviewers agreed that this paper should be accepted because of the strong author response during the rebuttal phase. Specifically the reviewers appreciated the motivation of the paper, its clarity, and the author clarification of the method, its assumptions, and scope during the rebuttal. Authors: please carefully revise the manuscript based on the suggestions by the reviewers: they made many careful suggestions to improve the work and stressed that the paper should only be accepted once these changes are implemented. Once these are done the paper will be a nice addition to the conference!
confidence: Certain
Decision: Accept