Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper introduces a formal definition of harm that builds upon the modified Halpern-Pearl definition of actual causation. The main differences between the definition and previous definitions in the literature are the new concepts introduced in the *causal utility model*, which adds 1) a utility function $\mathbf u : \mathcal R(O) \to [0, 1]$ for some $O \in \mathcal V$ and 2) a "default utility" $d \in [0, 1]$ to the standard causal model. The authors introduce 7 examples of situations where an agent is or is not caused harm that are prominent in the literature. They argue that their definition of harm either correctly identifies the presence or absence of harm for each of these examples, or correctly identifies the precise source of ambiguity regarding the presence or absence of harm.
strengths_and_weaknesses: **Originality**: The paper's main idea of introducing a causal definition of harm follows (Bontly, 2016). The novelty of the method lies in the fact that the definition introduced is formally stated using Halpern and Pearl's language of causal models. ~However, the novel components that the new definition introduces – default utility + the observation that preventing a worse outcome does not constitute harm – do not seem original, as they seem to be subsumed by the extended causal model for the purpose of determining whether or not harm was caused. (Halpern and Hitchcock, 2011 - https://arxiv.org/pdf/1309.1226.pdf). The concepts of the default world and utility function could be naturally included in the partial preorder if the previous definition of extended causal models were used. This would mean that harm may be defined as modified Halpern-Pearl causation under the extended model, using a particular partial preorder. Perhaps the authors could comment on and clarify this point, in case I am missing any subtlety here.~ (retracted, see "Reply to reviewer's first response", parts 1 and 2 below)

**Quality**: ~The solution to the proposed research question ignores or misses prior work, as discussed above, and while~ While the solution to the proposed research question resolves an issue posed to a previous attempt in the literature made at defining harm using causation, the proposed definition lacks sufficient justification. ~Why would we need this specific definition instead of using the already existent one? Further, there~ There are no theoretical results introduced that demonstrate that counterexamples do not exist, and neither extensive human evaluation of the definition. Some potential counterexamples to the proposed definition are provided in the limitations section of this review.

**Clarity**: Overall, the paper is clearly written. The motivation of the paper is clear, as is the proposed definition, the example formulation, and the comparison of the definition's evaluations on the examples to those of the prior work mentioned in the paper.

**Significance**: Formalizations of harm become increasingly important as automated systems grow in ubiquity. Thus, the problem addressed by the paper and the idea of using actual causation to formalize the problem are important to the responsible development of large-scale AI systems. Overall, the motivation behind the paper is greatly appreciated. 
questions: Questions:
1. Why is the causal utility model defined and used, rather than Halpern's extended causal model in the definition of harm? It appears to me that harm as defined in the paper is equivalent to causation under an extended causal model with a particular partial preorder.
2. Is it possible for harm to occur without causation? If not, why is this? Is it possible to introduce a result demonstrating that harm should occur due to causation?
3. It seems the notion of harm and blame should be related; do you have thoughts about this? Shouldn’t the definition of harm incorporate the mental model of the agent taking an action that may have harmed the agent being harmed and that of the agent being harmed?
4. The authors emphasize that the definition of harm proposed in the paper is "qualitative", while RBT's definition is quantitative. I am not sure this is clear, can you explain this distinction and provide further elaborations on how to think about it?
5. If the definition of harm is qualitative, why does the causal utility model use a utility function that maps to a number in $[0, 1]$, rather than one that induces an ordering (or partial preorder) over $O$? As the authors note, it is unclear how to combine the numerical utilities of multiple agents. Is there a reason a mapping to $[0, 1]$ must be used rather than an ordering?
6. This definition of harm seems quite interesting, and I am a bit intrigued to know if counterexamples to the definition already exist (similar to the ones I showed in the limitations section). 
limitations: I do not see potential for negative social impact of the work, as the work is aimed at formally defining harm - research which may help autonomous systems reason about minimizing harm in the future. Thus, I list some limitations of the work I observed below.

## Insufficient evaluation

The positive examples provided do not provide any guarantees that there are no counterexamples to the definition, and there are only 7 positive examples provided with no extensive human evaluation (by laypeople or by experts). I list two possible counterexamples below and would be curious to hear the authors’ thoughts.

## The relationship between harm and responsibility of agents is not considered by the work

This criticism is aimed at the fact that the definition proposed in the work does not consider the intention of the agents.

Consider the following example:

> A driver honks her car horn at the driver in front of her, startling an apartment resident dangling his legs off of his balcony railing. The resident falls off the balcony in fright, becoming gravely injured upon hitting the ground.

It seems strange to claim that the driver harmed the resident, because 1) she could not have known that the resident was in danger of falling 2) even if she did know, the resident is partially at fault for willfully placing himself in a precarious situation. To make the example even more clear-cut:

> An apartment resident decides that he will jump off his balcony upon the first car honk he hears. A driver honks her horn at the driver in front of her. Hearing this horn honk, the resident jumps off the balcony, becoming injured upon hitting the ground.

Now, it seems even stranger to claim that the driver harmed the resident, because the resident took an action intended to harm himself, while the driver did not.

This issue could be resolved by using Halpern's concept of blame introduced in (Chockler and Halpern, 2014 - https://www.aaai.org/Papers/JAIR/Vol22/JAIR-2204.pdf). The extended causal model could also solve this problem.

The idea of blame is also touched upon in Example 7. If Victoria knew she was able to rescue Betty without breaking her arm ($P = 2$) and chose to break her arm while rescuing her ($P = 1$), she caused Betty harm, because this goes against social norms. However, if she did not know she'd be able to do so and accidentally broke her arm ($P = 2$ was not an option), it is hard to argue that she caused Betty harm, as attempting to rescue Betty is well within social norms, and Betty's utility increased overall.

## Causing a decrease in utility in a socially unacceptable way (violating a default) is sufficient but not necessary for harm to occur

This criticism is aimed at the use of the "default utility" to represent all norms.

The work does not allow for situations where harm is done but reparations are made, and the agent in question is overall better off. For example: 

> A thief assaults and robs a passerby but later, out of a guilty conscience, gives the money back, adding some additional money to more-than compensate for any physical and emotional distress caused. The passerby is happier than they would have been otherwise, because they needed the money.

While the passerby is better off by the end of the story, the thief did cause harm to the passerby in the intermediate step of assaulting and robbing them. While this issue *could* be addressed by stating that the default world is the world where the thief does not assault/rob and gives the passerby money for no reason, this seems to be a difficult argument to make, as the thief is expected by social norms only to not commit any crimes. Setting to default world to the world where the thief gives money to the passerby would mean that *everyone* is harming the passerby by not giving money to them.

An even more clear-cut version of this example follows:

> A thief assaults and robs a passerby. After the local media reports on the crimes committed, many people donate money to the passerby in sympathy. The passerby agrees that he is better off now than he would have been had he not been robbed and assaulted.

It seems that the thief's actions, contrary to his expectations, helped the passerby overall. Is it still true that the thief did not cause harm to the passerby?

The issue could also be fixed by using the extended causal model to, regardless of utility, label any world where the thief commits a crime as less normal than a world where the thief does not commit a crime.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: [REMARK: This is an emergency review, that is, the reviewer did neither select autonomously into this paper out of interest/expertise nor was the reviewer granted with sufficient time (with respect to standard time frames) to appropriately provide a review. Please take the review with caution, the confidence level will be adjusted accordingly.]

The paper presents a formalization of the concept of "harm" (to harm someone) using structural causal models (SCM) as found in the counterfactual theory of causality vocally discussed by Judea Pearl. Specifically, the authors extend SCMs to cover a "default utility" (referred to as causal utility model) to then propose 3 conditions (H1 to H3) which captures (defines) whether an even $\overset{\rightarrow}{x}$ causes harm to an agent $\mathbf{ag}$. The authors provide a motivation, a comparison to other definitions (in terms of their presented formalism) and finally discuss several philosophical key examples displaying the advantages of their proposed definition.
strengths_and_weaknesses: **Strengths**

Discussing and overall researching into harm and how to capture/measure it is certainly timely in the advent of discussions around AGI alignment etc. The same holds for the intention of deriving a causal account since causal models allow for an explicable, interpretable modelling scheme. As the authors suggest, one can only debate about assumptions if said modelling assumption are recognizable. The presented formalization copes well with the discussed examples, which cover key philosophical dilemmas. Capturing formally, with thought through notions, otherwise intangible concepts like harm is an important step towards accountable intelligent systems.

**Weaknesses**

While one can question on a more general note whether discussing this topic within NeurIPS (especially in the presented form) is sensible, I believe the importance on a societal scale for such a work is undeniable and NeurIPS with both its breadth and influence would fit as a great platform. To comment on the paper itself more importantly, the presentation sets its focus on the formal account of harm, and while it does motivate generally and discuss specifically, the presentation (of otherwise very visual/intuitive ideas) is arguably dry. That is, to convey the ideas or spread the formalized notion (which is likely the key interest of the authors) alternate means of presentation using illustrations with figures but also visualizations of possibly synthesized versions of the examples would help in doing so. Regarding the mathematical account, the lack of analysis for induced properties (when is the computation feasible, what models do satisfy the condition in the first place, etc.) is a weakness (albeit a likely difficult one to fix since it is difficult enough already to capture a philosophical core problem that has existed for centuries mathematically).
questions: The questions are derived mostly from aspects mentioned in the "Weaknesses" section. I'd invite the authors to answer these questions to overcome the observed weaknesses for improving the paper to ultimately increase the chances of both contribution and visibility to the community. The list of questions is unordered and they range in quality and speed of expected answer (some are minor, quickly answerable whereas others are more fundamental, crucial, maybe difficult to answer):
* How can we cope with temporal notions?
* How can we overcome intractability of harm computation, in the face of our necessity to compute since otherwise a formalized notion would have no practical implications?
* Under which circumstances can our causal model guarantee that our harm computation will be approximately correct?
limitations: No contradictions or any sort of relevant mistake have been detected in the paper. The overall clarity of the paper is a clear advantage. Existing bodies of work are being referenced accordingly throughout the paper.

There is nothing to be reproduced, therefore, naturally, there is also no code. Societal impact is being discussed throughout.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors proposed the first qualitative causal framework that is used to define the harm and conceptually show their framework can solve various controversial real-life examples in terms of minimizing the harm. 
strengths_and_weaknesses: Strength:
It is by far the first literature, as far as I know, that attempts to discuss forming the framework for evaluating harm. It also discussed how to apply this framework to several real-life problems. 

Weakness:

Though I understand the difficulty of quantitatively defining the utility function, the lack of discussion of it could discourage its application to reality, I will discuss it in detail in the questions session. 
questions: The authors applied the framework of harm to several real-life scenarios. However, one thing in common is that all the scenarios consider defining harm for a single agent. However, in reality, the harm is usually multi-agents based. Without a clear quantitative suggestion for the utility function, it is hard to minimize the total harm. Even without considering the dependency among the agents, simply using the framework proposed by the author to make the decision could be challenging (Consider the Trolley problem). I suggest the authors consider how to address this issue or explicitly state this limitation.
 
limitations: The utility function u which measures the goodness of the decision is a vague term. In reality, the way of defining such a utility function could invoke controversy and fairness issues, which have also been discussed by the author.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a way to define **harm** in a causal way. The work in more on of a philosophical level, trying to criticize the fact that the definition of harm in philosophical studies is generally devoid of causality and putting forward the notion that the use of causality theory can define a more general definition of harm. Several examples are presented throughout the paper that show how the causal analysis and a new definition of harm can handle several standard examples from literature and can thus be used for effective reasoning in various domains and situations.
strengths_and_weaknesses: Strengths:

1. The paper tackles a very important and sensitive problem of defining harm. Since this can have multiple interpretations and can be conceived by different people in unique ways, trying to  come up with a causal-standard definition is indeed a nice effort.

2. The paper is written beautifully and the examples are crystal clear. In my opinion, even a person with little knowledge of causality will be able to follow the paper and get an overall idea about the main message of this work.

3. The definition of **harm** in section 3 seems well thought out and well sketched.

Weaknesses:

1. The method seems to have a very heavy dependence on the default utility. 

2. I do not agree with the Batman and Robin example, where the definition concludes that Robin was harmed. Just because Robin receives a new set of clubs at start of every season, it still does not make him entitled to receive it the next year as well. There can be the effects of some hidden confounders that are being ignored here.

3. The default utility of receiving or not receiving a tip in **Example 4** seems incomplete. For example, what about the cases where the exact amount of tip is also important? Say the waiter is better off if he/she receives 15% of the total bill as a tip instead of 5% in the US. 

Overall, this is an interesting take combining philosophical arguments with causality in a more explicit way. I would be eager to see the author's response to the raised points.
questions: In addition to some important points raised in the weaknesses section, I have a couple of specific questions:

1. How will the definition of harm be extended/modified if we have dependent exogeneous variables in the causal graph?

2. How are confounders handled in the causal definition of harm?
limitations: This is tricky. Although I do not see any specific negative societal impact, defining and constraining harm to a single definition might be a little bit risky. As I have mentioned in my review before, each can have his/her own interpretation with respect to the definition.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The work proposes a qualitative definition of harm based on causal models in the context of dealing with autonomous AI systems. The authors state that the notion of harm is ill-defined and motivate their qualitative approach on the observation that much of the discussion of harm is qualitative. After reviewing causal models and describing the introduced definition of harm, the work applies and discusses it (including its limitations) on various examples.
strengths_and_weaknesses: **Strengths**

- S1: The definition is inspired by previous work (RBT, Bontly) and extends them to a causal definition.

- S2: Besides some minor issues (see following weaknesses), the proposed definition is well described and understandable. 

- S3: The extensive examples, including the discussions contained in the main text and the supplement, give a great overview and intuition of the proposed definition.

**Weaknesses**
- W1: While building on Y. Halpern [11] and Y. Halpern and J. Pearl. [13] working with the concept of 'actual causality', the paper might be improved by briefly discussing differences to the common approach of probabilistic structural causal models [X1, X2], where variables are considered to be sampled from probability distributions with structural equations including random noise terms. Especially recursive deterministic computation of variables as described in the paper must assume that no noise terms are included in the system. The authors might consider to state this assumption explicitly.

- W2: Furthermore, the relations between exogenous and endogenous variables (line 112) seems to be more strict than the standard definition using 'Pearlian causality'. E.g. compare to [X2] p.23 “Endogeneous variables are those that the modeler tries to understand, while exogenous ones are determined by factors outside the model, and are taken as given.” While similar in the intention, Peters et al. give no definition of how to treat the relationship between endogenous and exogenous variables. Assuming endogenous variables to be fully determined by exogenous factors is a strong assumption that, again, only holds under the assumption that all exogenous variables are known and observed and no additional random noise terms are influencing the system.


[X1] Pearl, J. (2009). Causality (Cambridge university press)

[X2] Peters, J., Janzing, D., and Schölkopf, B. (2017). Elements of causal inference: foundations and learning algorithms (The MIT Press)


**Minor comments**

- Typo line 104 "that that"
- Typo line 166 "beem"
questions: See W1 and W2 above.
limitations: The authors adequately addressed the limitations.
review_title: null

Meta-review: metareview: The paper aims at formally defining a qualitative notion of harm based on "actual causality". This is an important problem tackled, also or in even in particular for ML research. Many ethical issues raised about AI circle around a notion of harm, such as discrimination induced by classifiers, decision of autonomous agents, and even just harmful content of training sets. From this perspective, I really congratulate the authors. However, it is a downside that existing approaches to related approaches within the ML community are not discussed.  It is good tradition and practice to provide related work. Indeed, there is no actual causality model of harm yet (as far as I can say) but there are e.g. deontological approaches that could be mentioned or even briefly discussed such as 

Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny Liang, Oren Etzioni, Maarten Sap, Yejin Choi: Delphi: Towards Machine Ethics and Norms. CoRR abs/2110.07574 (2021)

Patrick Schramowski, Christopher Tauchmann, Kristian Kersting: Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content? In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (2022)

Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin A. Rothkopf, Kristian Kersting. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence 4(3): 258-268 (2022)

Moreover, the authors should discuss "we define an event to cause harm whenever it causes the utility of the outcome to be lower than the default utility" in more detail. Why is this a good definition? Let us for a moment equate utility with money then even too much money could be harmful due to network effects. In other words, it would be great if the authors could discuss a bit more the assumption made and their implications. This is even more important given the closeness to Bountly's notion of harm, replacing “state of affairs” by “outcomes”, and associating with each outcome a utility. The main downside, however, is the missing illustration of how the presented formalization of harm actually will help to tackle some of the ethical issues of AI. Is this really the right way? But then, who knows, and this paper is indeed taking a very different step than the average NeurIPS paper. In my opinion, the reviewers raise some salient arguments about the suitability of this paper for NeruIPS. For instance, there are indeed positive examples presented only. However, then the negative examples in the review want to illustrate that actual causality is not the right tool either. Still, the discussion within the NeurIPS community would then help already. Moreover, not knowing to break the arm is still causing harm. Anyhow, it is exactly this type of discussion that makes the paper in my humble opinion interesting to the NeurIPS community. The discussions with the authors showed that it is a topic that will provoke a lot of discussions. And since the overall sentiment of the reviewers is positive, I recommend accepting the paper. 
confidence: Certain
Decision: Accept