Moral Responsibility for AI Systems
Authors: Sander Beckers
Venue: NeurIPS 2023 poster (2023)
Keywords: responsibility
causation
causal models
TL;DR: I develop a formal definition of moral responsibility using causal models that can be applied to (and used by) an AI system.

Abstract: As more and more decisions that have a significant ethical dimension are being outsourced to AI systems, it is important to have a definition of _moral responsibility_ that can be applied to AI systems. Moral responsibility for an outcome of an agent who performs some action is commonly taken to involve both a _causal condition_ and an _epistemic condition_: the action should cause the outcome, and the agent should have been aware - in some form or other - of the possible moral consequences of their action. This paper presents a formal definition of both conditions within the framework of causal models. I compare my approach to the existing approaches of Braham and van Hees (BvH) and of Halpern and Kleiman-Weiner (HK). I then generalize my definition into a _degree of responsibility_.

Moral Responsibility for AI Systems
Sander Beckers University of Amsterdam sanderbeckers.com srekcebrednaas@maal. com
Abstract
As more and more decisions that have a significant ethical dimension are being outsourced to AI systems, it is important to have a definition of moral responsibility that can be applied to AI systems. Moral responsibility for an outcome of an agent who performs some action is commonly taken to involve both causal condition and an epistemic condition: the action should cause the outcome, and the agent should have been aware in some form other of the possible moral consequences of their action. This paper presents formal definition conditions within the framework of causal models. I compare my approach to the existing approaches of Braham and van Hees (BvH) and of Halpern and Kleiman- Weiner (HK). I then generalize my definition into degree of responsibility.
both
Introduction
As more and more decisions that have significant ethical dimension are being outsourced to AI systems, it is important to have a definition responsibility that can be applied to the decisions of Al systems, and that can be used by AI systems in the process of its decision-making (8). To meet the first condition, such a definition should require only a minimal notion of agency and instead focus on those aspects of responsibility that readily applicable to (current) Al systems. To meet the second condition, such a definition should be formulated in a language that can be implemented into an AI system, that it can integrate judgments of responsibility into its decision decision-making. This paper sets out to propose such a definition using the well-established framework of causal models [22 23).
There exist different notions of moral responsibility that one might be interested in, and here we restrict attention to just one of them, namely responsibility for consequences, meaning the responsibility one has for particular outcome that is the result of performing particular action. This can be expressed more clearly by saying that the action caused the outcome, and therefore the first condition of concern here is the causal condition responsibility 12 26 21 5] The past two decades have seen immense progress on offering formal definitions actual causation by way of using causal models, and the definition here developed takes maximal advantage of this progress by comparing some recent proposals and choosing the one that correctly handles several complicated cases to be considered 3 [6 2].
Our actions can cause kinds of outcomes for which we are clearly not morally responsible: if train crashes into car that illegally crosses the railroad then the train conductor is not responsible for the car driver's death, if you turn on light switch a hotel room then you are not responsible if a short-circuit follows, etc.. The standard intuition that have in such cases is that the agent "could not have known" that their action would cause the outcome, This is why definitions of responsibility also invoke epistemic condition, stating roughly that the agent should have been able to foresee that they are performing an action which could result in them being responsible for the outcome 27 255
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

In addition to the causal and the epistemic conditions, it is standard to demand that responsibility also requires the fulfilment of control condition (sometimes also called freedom condition), which expresses the fact that the agent had the right sort of control whilst performing their action [25]. Due to its close connection to issues of free will and determinism, this condition is heavily dehated within philosophy. Within the context of (current) Al systems, however, the control condition can take on a more mundane form: any action that was a result of the correct operation of its program can be viewed as being under the Al's control. Therefore I simply take there to be specific action variable that ranges over a set of possible actions, and assume that whenever the AI system is running successfully it has control over the value that this variable takes.
My approach proceeds along the same lines as that of Braham and van Hees (BvH) [5]. They offer the most influential formalization of moral responsibility that incorporates both the causal and the epistemic conditions, and therefore their work forms an appropriate point of comparison. Although I agree with the spirit of their approach, I disagree with its formulation. First, their causal condition defines causation as being a Necessary Element of Sufficient Set (NESS). However, their use of game-theory instead of causal models results in an overly simplistic view of NESS-causation that cannot handle indirect causation. Therefore first formulate their definition using causal models, and then show how to modify it so that it can overcome this limitation. Second, I disagree with the particulars of both their causal and their epistemic conditions. argue for replacing the NESS definition of causation with my recently developed Counterfactual NESS (CNESS) definition (2). Their epistemic condition states that the agent should minimize the probability of causation. I argue for giving that condition secondary role: minimizing the probability of causing the outcome is subservient to minimizing the probability of the outcome simpliciter. I analyze several examples to illustrate the superiority of my conditions.
More recently, Halpern & Kleiman- Weiner (HK) [17 used causal models to propose definitions of several concepts that are closely related to moral responsibility. Although they do not explicitly define moral responsibility. they do suggest using the modified Halpern & Pearl (HP) definition of causation for the causal condition [16]. The HP definition correctly handles most of the counterexamples to the NESS definition here presented, but I discuss two types of example for which it fails (whereas the CNESS definition does not). HK also offer definition of "degree of blameworthiness" that for all intents and purposes is very similar to an epistemic condition: it measures the extent to which the agent minimized the probability of the outcome. present case in which the epistemic conditions of ByH and HK conffict in order to argue that a more elaborate epistemic condition is required. My epistemic condition combines that of HK with that of BvH by demanding that agent minimizes the probability of the outcome, but if possible also minimizes the probability of causation. |
Here is the general schema that encompasses all definitions of responsibility that aim to consider. Responsibility Schema. An agent who performs A = a is responsible for outcome O = o if:
(Control Condition) The agent had control over A= == a.
(Causal Condition) A = causes Q = o.
(Epistemic Condition) The agent believes that they could have avoided being responsible for O= = o by performing some alternative action A=d'.
As mentioned, I simply assume that the Control Condition is always met (as do BvH, who call it the Agency Condition). For sake of brevity, I leave implicit from now on.
Formalizing the Causal Condition comes down to settling the discussion on how to formalize actual causation, which has received considerable attention over the past two decades [2 31 15
30 16
A full discussion of causation would be too ambitious for the present purposes. Instead, evaluate the suitability of several definitions of causation within the context of responsibility by presenting examples that bring across how they differ. On the basis of this evaluation I suggest adopting the CNESS definition and refer the reader to for more general motivation.
The Epistemic Condition requires settling the question: what does it take for the agent to believe that performing A = d' allows them avoid responsibility? Since this condition uses the notion of
Note that the epistemic conditions of HK and BvH not necessarily inconsistent: if one simply defines causation as an increase in the probability of the outcome occurring. they become equivalent. Except for the fact that he uses objective probabilities rather than those of the agent, this is roughly the proposal defended in (29 As exemplified by the examples to be discussed (and as exemplified by browsing the recent literature on causation) such naive probabilistic approach to causation is unable deliver sensible verdicts.

---

responsibility, our Schema is circular. There exist different ways of filling it in so that it no longer is cireular, and is this flexibility that makes filling in the condition interesting. One possible suggestion is to demand that the agent believed A = a' would not result in the outcome O another weaker suggestion is to demand that the agent believed A = a' would not cause O = o, etc..
A note of clarification is in order before we proceed. The current work does not aim to offer a complete theory of moral responsibility for AI systems, but rather zooms in on the above conditions whilst ignoring certain others. Concretely, here are some important issues that I set aside in this paper.
1.1 Some Limitations and Related Work
There exist forms of moral responsibility that do not (always) involve causation, such those that follow from certain societal norms and expectations. For example, a captain is responsible for everything that happens on their ship, a parent is responsible for the behavior of their child, etc.. More generally, assigning responsibility to AI systems should itself be seen as just one part of the wider discussion on accountability that arises from the introduction of such systems into our society 19 77.
Relatedly, responsibility is often associated with the morally stronger notions of blame and praise. I take responsibility be weaker notion that necessarily precedes judgments of blame and praise: one cannot be blameworthy for an outcome unless one is responsible for it, and similarly for praise. To develop definitions of blame and praise requires bringing into view both the absolute moral valence of the outcome O = (was it good or bad?) and its relative valence (was it better/worse than an alternative which it prevented?), as well as the costs incurred by the agent when performing an action. As the vast literature on trolley cases and other moral dilemmas illustrates, these issues make matters significantly more complex [10 20
One condition in particular that seems highly relevant to assigning blame (resp. praise) is to consider whether the outcome caused by the agent is harmful (resp. beneficial) not. Indeed, one natural way of implementing a formal definition of responsibility within Al systems is to demand that it tries to avoid becoming responsible for harmful outcomes. This is confirmed by the recent European AI Act, which categorizes the risk that an Al system poses based on how likely it is to cause harm II Beckers et. al. recently proposed causal analysis of harm that is also formalized using causal models, and thus it could easily be integrated into the present proposal 3 4 In the present paper, however, choose to focus exclusively on defining responsibility, thereby paving the way for future definitions of blame and praise.
Duijf presents formalization of moral responsibility for outcomes that is likewise inspired by, but not an endorsement of, BvH 19. Rather than defending an alternative definition of responsibility as I do, he presents a broad lanscape of completely formal conditions for responsibility that one might consider and analyzes their logical relations. As with BvH, his definition of NESS causation is formulated using game-theorpy and thus it is likewise restricted to applications of direct causation.
The next section introduces the formalism of causal models that will be used to express all candidate definitions and related notions. J the BvH and HK definitions of responsibility and their respective definitions of causation. Section discusses the Causal Condition by introducing two more definitions of causation and offers some examples to argue in favor of adopting the CNESS definition. (Some further examples are offered in the appendix.) We move on to a discussion of the Epistemic Condition in Section 5 which leads the way to my definition of moral responsibility. Since responsibility is often taken to come in degrees, in Section 6 define the degree of responsibilityy and sketch how it helps interpret recent empirical work in psychology on responsibility judgments.
2 Causal Models
This section reviews the definition of causal models understood in the structural modeling tradition started by Pearl (22,, where use the notation from Halpern [16].
Definition 1. A signature S is a tuple (U.VVRO where is a set of exogenous variables, V is a set of endogenous variables, and R a function that associates with every variable Y e L UV nonempty
21 should note that they use the de--eiinitoon of causation, which I criticize below. However, they state explicitly that their approach applies just well to other definitions of causation.
3

---

set R(Y of possible values for Y (i.e. the set of values over which Y ranges). If = (x)). R(X) denotes the crossproduct R(X,) x x.. n.)).
Exogenous variables represent unobserved factors whose causal origins are outside the scope of the causal model, such as background conditions and noise. The values of the endogenous variables, on the other hand, are causally determined by other variables within the model.
Definition 2. A causal model M is pair (S.F), where S is signature and F defines a function that associates with each endogenous variable Y structural equation Fy giving the value of Y in terms of the values of other endogenous and exogenous variables. Formally, the equation Fy maps R(U UV -YY)) to R(Y). so Fy determines the value of Y, given the values of all the other variables in M.V.
We usually write the equation for an endogenous variable as Y ((X), where X are called the parents of Y (and Y is called a child of each variable in X), and the function f is such that it only depends on the values of X. The ancestor relation is the transitive closure of the parent relation. In this paper we restrict attention acyelic models, that is, models where no variable an ancestor of itself. A (directed) path is a sequence of variables in which each element a child of the previous clement.In this manner an acyclic causal model induces a unique DAG, i.e., Directed Acyclic Graph, which is simply a graphical representation of all the ancestral relations.
An intervention has the form X + I, where X is a set of endogenous variables. Intuitively, this means that the values of the variables in X are set to the values æ. The equations define what happens in the presence of interventions. The intervention X < ø in a causal model = FS,F) results in a new causal model, denoted M M which is identical to M, except that FF is replaced by F--2; for each variable Y $ X, F&+= = Fy (i.e., the equation for Y is unchanged), while for each X' in X, the equation Fxr for X is replaced by X' = (where x' is the value in ø corresponding to X').
Given a signature S = (4, V.R., , an atomic formula is formula of the form X = I. for X V and E R(X)) A causal formula (over S) is one of the form [Y1 1/,...,, YR yk yk] where
Φ is Boolean combination of atomic formulas,
Vi....,
N.....,Yy Ys are distinct variables in V, and
y/ R(Y:) for each 1 1 i k.
Such a formula is abbreviated as [Ý + v]o. The special case where k = 0 is abbreviated as ø. Intuitively, [Y + y1...., Y2 o says that á would hold if Y, were set to yi. for i I....,k.
We call a setting â E R(U)) of values of exogenous variables a context. causal formula vb is true or false in causal setting, which is causal model given context. As usual, we write (M, ū) F d if the causal formula z is true in the causal setting (M, The relation is defined inductively. (M, û) := X = z if the variable X has value in the unique (since we are dealing with recursive models) solution to the equations in M in context ū (i.e., the unique vector of values that simultaneously satisfies all equations in M with the variables in 2 set to ū). The truth of conjunctions and negations is defined in the standard way. Finally, (A, ai) [Y
zilø if (g. u) ø.
In addition to the causal setting (M, ū) that describes both the objective causal relations and their actual realization, we also need to represent the agent's beliefs regarding what could possibly happen in order to fill in the Epistemic Condition. do so in the same manner proposed by HK: we take Pr to be a probability distribution over a set of causal settings X. so that Pr expresses the agent's s subjective probabilities before the agent performs their action. As do HK. I assume for simplicity that all the causal models appearing in X. have the same signature (i.c., the same exogenous and endogenous variables). We define epistemic state of agent to consist of pair & (Pr, K.), and define a responsibility setting ( (M, u, E) as the combination of a causal setting and an epistemic state.
3 The BvH and HK Definitions
BvH 5 work within game-theoretic framework and do not use causal models, so in order to compare their approach to mine we need to first translate into the language of causal models. I do not delve into the details but rather offer rough sketch of such translation.

---

Similar to causal models, BvH represent the agents" influence on the outcome O by way of a function. Yet instead of letting some endogenous variables Á represent the actions of agents directly, they use variables to represent the strategies that each agent can adopt to guide their actions. Aside from that, the main difference between the two formalisms is that theirs is unable to represent indirect causal relations. In general, the equations of a causal model allow for an unlimited number of intermediate ancestors between variables Á and an outeome variable O so that causal influence from an agent's action can be passed on along intermediate variables to the outcome variable. BvH's outcome function on the other hand abstracts away from any mediated form of causal influence, so that the strategies causally determine the outcome directly. As result, their games are to be interpreted as a single-equation causal model of the form O fo (A). (Since the variables are determined directly by the context ö, adopt the standard practice leaving their equations implicit.)
ByH use the famous NESS definition of causation that was proposed by Wright [32 33 and also formed the inspiration for the Halpern & Pearl (HP) definitions [22][5] which states that causes are Necessary Elements of Sufficient Set for the effect. Taking into account the previous remarks, it more accurate to speak of the direct NESS definition. I here present my recent formalization of both the direct and the indirect NESS definitions using causal models 22. First we need to define causal sufficiency. As do BvH, take it to mean that a set guarantees the effect regardless of the values of the variables outside of the set.
Definition 3 (Sufficiency). We say that X = E is sufficient for Y = W.E.t (M, if & X and for all values ă E () where 2 = V - (X (Y)), it holds that (M, F= [X < म. 2 - Y =
Direct NESS-causation is then defined by stating that:
the candidate cause and the effect actually occurred;
the candidate cause is member of sufficient set;
and it is necessary for the set be sufficient.
Definition 4 (Direct NESS). X = directly NESS-causes Y = y w.r.t. (M, if there exists W- w so that the following conditions hold:
DNI. (M, ū) X x W Y = y.
DN2. X x W ã is sufficient for Y y W.F.I. (M,
DN3, W Φ is not sufficient for Y y w.r.t (M, i).
We can now formulate the counterpart of the ByH definition using causal models by filling in their conditions into our Responsibility Schema.
Definition (BvH Responsibility). An agent who performs A a is responsible for outcome O o w.r.t. responsibility setting (.) if:
* (Causal Condition) A = directly NESS-causes O = w.r.t. (M,
(Epistemic Condition) There exists a' R(AA) so that Pr(A directly NESS-causes O >)> > Pr(A = a' directly
O = o)
Informally, the BvH definition of responsibility requires that agent's action directly NESS-caused the outcome, and that the agent believes they failed to minimize the probability of their action causing the outcome. The following example (taken from ByH) illustrates their definition.
Two assassins (Assassiny and Assassing), in place as snipers, shoot and kill Victim, with each of the bullets fatally piercing Vietim's heart at exactly the same moment. Although neither of them could have prevented the outcome, each of them is clearly responsible for Victim's 's death.
Example 1.
Let V stand for Victim's death (V = 1) or survival (V 0), and let A1, Ag stand for the actions of the two assassins, where A; 1 if and only if Assassin, shoots. We can then capture this example with the single equation V = Ai v A2, and a context ū such that A = 1 and A = 1.
3 As 1 said, this is a rough sketch. Technically, one should distinguish between games in normal formal.orm, which is the form considered by ByH, and games in extensive form, from which the normal form games have been derived. Games in extensive form do allow for indirect relations, and thus there might be way of representing indirect causal relations in game theory after all.
'Of course these probabilities have to be read as being conditioned on the corresponding action, i.e., as "the agent's probability that the action would cause the outcome if it were performed".
5

---

Does the BvH definition (Definition succeed in establishing that each of the assassins is responsible
for Victim's death? To find out, we first need to evaluate whether A = (resp. A = 1) directly
NESS-causes V (Def. 4 We can choose W Ø to get the desired result, as follows.
AC1 is fulfilled because A1=1 and V actually happened. AC2 is established by verifying that the following two claims hold: (M, + [Ai A2 1JV 1 and (M,ü) + [Ai 1, A2 0JV= Since W Ø. verifying AC3 easy: we need find a single intervention on the variables other than V such that they result in V = 0. The intervention [A] + 0, A2 0] does the job.
To evaluate the Epistemic Condition requires making some assumptions about the assassins's 's probability attributions. It sounds reasonable to assume that, without evidence to the contrary, each assassin attributed a higher probability them shooting causing the outcome than them not shooting causing the outcome. Therefore the Epistemic Condition is also fulfilled for each assassin, and thus the BvH definition arrives at the right verdict for this example.
We continue with the approach pursued by Halpern Kleiman-Weiner (HK) which uses the modified Halpemn & Pearl definition of causation (16):
Definition (HP). X = 7 HP-causes Y = w.r.t (M, ū) if there exists W = w so that the following
conditions hold:
AC1. (M,
AC2. There is a setting i' such that (M, ú) = [X+ H', W u # y.
AC3. X is minimal; there no strict subset X' of X such that X' = 1" satisfies AC2, where 1" is the restriction of i to the variables in X'.
Note that, contrary the direct NESS definition, the HP definition allows for conjunctive causes X = .i, instead of merely atomic causes X z. The minimality condition (AC3) is there to prevent irrelevant events to be added to such conjuncts. We can retrieve a definition of causation for atomic events by simply considering any conjunct X x that appears in an HP-cause X = 3 to be cause as well, which is indeed what Halpern suggests himself repeatedly [16].
The heart of the HP definition is AC2: states that the outcome Y = counterfactually depends on the cause X = given that we intervene to hold fixed a suitably chosen set of variables W at their actual values ल. To see how this definition works, let us apply it to Example
First we try substituting X = - with A₁ 1. Alas, this will not allow us to get A1 = 1 as a cause of V=1 1. We start with choosing W- = Ø, and we get that (M, ú) F [A1 0JV 1. The reason is that encodes the actual context, in which A 1, and thus also V 1. Yet what is required for AC2 would be (M, ū) [A] + V]V
The only other choice for W é would be Ag 1, and that does not work either: (M, ú) F [A] + 0, A2 + 1]V = 11.
Second we try A A A2 1. If this works, then AC3 is satisfied due to the fact that neither of the conjuncts themselves satisfied AC2. W has be Ø, since there are other variables. Thus what remains is to find counterfactual values for A₁ and Ag. As they are binary, the only option is to consider A 0 A A Clearly, for this choice AC2 satisfied, as (M, ū) [Ai 0, Ag OJV= =V/=00 0. Therefore A 1 is an HP-cause of 1.
We now formulate a definition of responsibility that is closely inspired by HK. Definition (HK Responsibility). An agent who performs A = is responsible for outcome O w.E.t. responsibility setting (M, ú. E) if:
(Causal Condition) A= HP-causes O = w.r.t. (M, ú).
(Epistemic Condition) There exists a' RRA) that Pr(O = o[[AA a)) Pr(O
o[[A ++++a d']).
In addition to disagreeing about the definition of causation, the HK definition also disagrees with the ByH definition about the epistemic condition: rather than requiring that the agent failed to minimize the probability of causing the outcome, the HK definition focuses on the agent failing to minimize the probability of the outcome simpliciter.

---

Note that both HK and BvH's epistemic condition satisfy our Responsibility Schema: an agent who believes that they failed to minimize a probability that they could have minimized, thereby also believes that they could have avoided satisfying the respective epistemic condition. Given that the epistemic condition is a necessary condition for being responsible, they also believe that they could have avoided being responsible for the actual outcome.
Let us apply the HK definition to Example[] We already established that each A; 1 is an Ha-cause of V = 1, so the Causal Condition is met. Further, as long as each assassin attributes a strictly positive probability that the other assassin may fail to shoot, we get that Pr(V 1[[ - 1]) > Pr(V = 1[[ A 0}), so that the Epistemic Condition is satisfied as well. (What if the assassins are certain the other assassin will shoot? We come back to this in Section Therefore the HK definition also arrives at the correct verdict for this example.
4 The Causal Condition
Before discussing the problems with NESS- and HP-causation, I present ENESS-causation 22 As first step, we define NESS-causation causation as the transitive closure of direct LESS-causation, which is how it was conceived by Wright [33]. In addition, we pay explicit attention to the path along which the causal influence is transmitted.
Definition 8 (NESS). X = x NESS-causes Y = y along a path p w.r.t. (M, if the values of the variables in p form path of direct-NESS causes from X x to Y y.
The Counterfactual NESS definition (CNESS) takes the NESS definition and adds a subtle coun- terfactual difference tifference-making condition: there should be counterfactual value so that it would not NESS-cause the outcome along the same path as the actual value, nor along any subpath.
Definition 9 (CNESS). X ==x CNESS-causes Y y w.r.t. (M, ū) if X x NESS-causer Y = along some path p w.r.f. ( (M, ū) and there exists a x' e R(X) such that X x'' does not NESS-cause Y y along any subpath p' p w.r.t. (Mx-z' ú).
With all the definitions of causation at hand, now motivate my choice for the CNESS definition by going over some well-chosen examples. We start with case of Late Preemption.
Example 2 (Late Preemption). We return to our two assassins, but this time Assassing is slightly faster, that their bullet kills Victim, who collapses and thereby dodges Assassing' 's bullet.
In this case Assassing obviously did not cause Victim's s death, and is thus not responsible for the outcome (despite the fact that their act itself is of course still blameworthy). BvH only allow variables for strategies and are thus unable to capture this result, since the asymmetry between both assassins is not a matter of strategy. As illustrated at length by Halpern [6). using causal models this poses no problem. The equation V BH, BH₂ expresses the fact that either bullet hitting Victim would be fatal; BH, A₁ and BH₃ A ^ -BH captures the asymmetry between both assassins: Assassing's 's bullet only hits Victim if Assassini 's bullet does not. In the context at hand, we have that A A2 BH V 1, and BH2 = O. We now go through the various definitions
verify that A₁ = 1 NESS- CNESS-causes, and HP-causes V 1, whereas A₁ = 1 does not directly NESS-cause V thereby showing that the direct NESS definition is too simplistic.
We start by verifying that A 1 does not directly NESS-cause V 1. By itself A = does not form a sufficient set for V = 1, for setting both of the B H variables to 0 guarantees that the Victim survives: (M, = [A <- 1, BH, < 0, B H2 + V)V = 0. fact, this context, any sufficient set for V 1 has to contain BH, 1, yet B H is sufficient for V = 1 all by itself. Thus A 1 is not necessary member of any sufficient set for V = 1. Still, A = 1 NESS causes V = 1 along p = {A1, BH, V). because A directly NESS- casses BHH 1 and B BH, 1 directly NESS-causes V = 1.
To establish this setting we get that A₃ 0, A₂ = 1, BH, 0. and thus BH, 1 (as well as 1). (Informally: if Assassing had not shot, then Assassing 's bullet would have hit and killed Victim.) Here A₁ directly NESS-causes BH; = 0. BH, 0 directly
requires having a look at the counterfactual setting (M ú). In
causes B BH, 1 (since it forms a sufficient set together with A = and A = does not suffice on its own), and B H2 = directly NESS-causes V = 1. Therefore A, 0 NESS-causes V = along p² (A,, BH, BH2, V). (Take note of this surprising finding. We come back in Example Since y² $ p. we get that A, 1 CNESS-ccuses 1 in the counterfactual setting, since p p²))
V 1 (whereas A₁ 0 does not CNESS-
7

---

To see that A1 1 caP-cauees V = 1, it suffices to note that (M,ū)= F BH2 0 and (M,ú) = [A1 - 0, BH2 + 0JV O. Lastly, I leave to the reader to verify that A2 = is not an HP-cause of V 1, and nor is it a direct NESS-cause of anything. Because of the latter, A2 1 is not NESS-cause or CNESS-cause of anything either.
Modifying the BvH definition so that it uses NESS-causation instead of direct NESS- is not a solution, for the NESS definition itself problematic, as the following example shows. (In the appendix discuss one more example, a so-called "Frankfurt-case", to show that BvH's reliance on strategies as opposed to events forms another source of problems.)
Example 3. We revisit the counterfactual setting of Example in which Assassing does not shoot, so that Victim is killed by Assassing' 's shot.
We already established for this scenario that A = NESS causes V = 1. Thus if we use the NESS definition, get the absurd result that Assassin, failing to shoot causes Victim to die. If we then supplement the example so that also BvH's Epistemic Condition is fulfilled, we get that Assassin comes out being responsible for Victim's death. (Imagine, for instance, that they mistakenly believe to be holding a flare gun that could sound a warning shot so that Victim ducks for cover to avoid Assassing's 's bullet.) We already established that A 0 does not CNESS-cause V 1, the reader may verify that the same holds for the HP-definition.
This leaves CNESS-causation and HP-causation as candidates for the Causal Condition. use Halpem & Pearl's own example to argue against HP-causation [15].
Example 4 (Loader). " "Suppose that a prisoner dies either if A loads B's gun and B shoots, or if C loads and shoots his gun. Taking D to represent the prisoner's 's death and making the obvious assumptions about the meaning of the variables, we have that D 1 iff (A = 1 A B = 1) v C = 1. Suppose that in the actual context û, A loady B's gun, B does not shoot, but C does load and shoot his gun, so that the prisoner dies. Clearly C = is a cause of D = 1. We would not want to say that A 1 is cause of D = 1, given that B did not shoot (i.e., given that B 0)." [emphasis added]
I agree with Halpern and Pearl. A fortiori, A is not responsible for the prisoner's s death, even if A only loaded the gun because he was convinced that B would shoot. Now consider the following variant. In the original example, C's shot is determined directly by the context. Imagine we add little twist, so that C' would only fire his gun if B did not, i.e., the equation for C' is C = B. The above reasoning regarding A still applies, and therefore believe it is mistake to all of a sudden consider A = 1 cause of D = 1. Yet A = now is an HP-cause of = 1 (as it appears in the Ha-cause 1=1aB 0), and thus A would be considered responsible for the prisoner's 's death. The CNESS definition avoids this result (as does the NESS definition): the only candidate sufficient set for D = of which = 1 could be necessary part, is {A= 1, B = 1). So the mere fact that B 0 in both versions of the example implies that A = is not a NESS cause of D = 1 in either.
I leave second counterexample to the HP definition for the appendix and refer the reader to [ for a detailed critical examination of the HP definition. The alternative definition I there presented is in fact very similar to my CNESS definition, although the precise relation is the subject of further investigation This leads me to suggest adopting the CNESS definition for the Causal Condition.
5
The Epistemic Condition
Recall that the difference between HK and BvH's Epistemic Conditions lies in whether an action minimizes the probability of the outcome occurring (HK) or of causing the outcome (BvH). Given that one cannot cause an outcome unless the outcome actually occurs, and that vice versa, in many cases the best way to make sure that an outcome occurs is by causing it, both of these conditions often go hand in hand. However, the following example illustrates, they do not always do so, and when they do not the appeal of HK's condition is stronger.
Example S. Bombing A bomb (B) is connected to three detonators (Di, Dr, and D2) by two switches (S₁ and S2). D2 is functional if only S₁ is on, D2 is functional if only S2 is on, and D3 is functional whenever S₁ is on. The equations are thus as follows: B D2 V D2 v Dz, D1 S1 A -S2. D2 S2 A SS, and D3- Sy. Assassing (reasonably) assigns probability of 0.6 to Assassini
'I tentatively conjecture that the CNESS definition implies my other definition, and not vice versa.
8

---

turning on St. He decides to turn on S2, thereby guaranteeing that the bomb will explode. Assassini decides not to turn on St. so that the bomb explodes due to the functioning of Dz.
Here we certainly would want to say that Assassing is responsible for the explosion, and the reason for this seems to be precisely that he knowingly increased the probability of the bomb going off (from 0.6 if S2 = 0 to now that S2 = 1). There is also no doubt that Assassing" 's action caused the explosion: if he had turned S2 off, the bomb would not have exploded.
However, Assassing did act so as to minimize the probability that his act would cause the explosion, Concretely, for all three
regardless of whether one chooses NESS-, HP-, definitions of causation, Assassing's ' probability that S = 1 would cause B = 1 is 0.4, whereas his probability that S2 0 would cause B = 1 is 0.6. (The details are worked out in the appendix.)
Note that in case S 1, then S₂ O would result in the outcome being overdetermined, and thus although the latter action would also be a cause of the outcome, it does nothing to contribute to the probability of the outcome occurring. This is what explains why the two conditions can come apart, and why I take the general moral of this story to be that increasing the probability of the outcome trumps increasing the probability of causing the outcome.
However, it does not follow that the probability of causation is irrelevant, but only that it should fulfill secondary role. Consider again Example [ and assume that Assassin, believes that Assassing will shoot, and thus believes that Victim is facing certain death. (If that sounds too unrealistic, imagine Assassin, is one of ten members of highly trained firing squad that is executing Victim.) Thus the action of Assassin, had no effect on the probability of the outcome, and would thus not be responsible for Victim's 's death according to HK's definition. If Assassing has a similar belief, then we end up with nobody being responsible. take this to be an unacceptable result. (Fischer & Ravizza reach the same conclusion when likewise discussing a case (Missile 2) in which an agent knows that the outcome will ensue no matter what they do, and yet the agent is still responsible for the outcome by choosing to cause it 12 p. 102].)
The lesson draw from this is that if one knowingly has the opportunity to reduce the probability of causation without thereby increasing the probability of the outcome, then an agent is responsible if she fails to do so. Therefore I propose the following definition of moral responsibility.
Definition 10 (Responsibility). An agent who performs A responsibility setting ( (M, üł 8) if:
responsible for O = o w.r.t.
(Causal Condition) A a CNESS-causes O ==00 o w.nt. (M, u))
(Epistemic Condition) There exists E RRAA) so that one of the following holds:
1. Pr(O of[ A + a)) Pr(O = of[A + d')) Pr(O of[A - Pr(O of + a')) and
Pr(A a CNESS-causes O =) > Pr(A- a' CNESS-causes O = o).
6 Degree of Responsibility
My binary definition of responsibility can be complemented with a definition of the degree of responsibility in order to capture the widely shared sense that responsibility (as well as blame and praise) is a graded notion. Both BvH's and HK's Epistemic Conditions naturally suggest such a definition, and so does my combined condition.
The obvious graded counterpart of HK's condition is to simply look at the causal effect (22] which in the context of causal strength is referred to as the Fells measure of causal strength of A a relative Sprenger 2X argues
to A d': CS, (o, a, a') = Pr(O o[A + a)) Pr(O = o - a'I) [3 28 for accepting the Eells measure as general measure of causal strength, which is in line with the priority that my Epistemic Condition attributes to it. Moreover, when restricted to positive values, this is in fact HK's definition of the degree of blameworthiness. Likewise, the obvious counterpart of BvH's condition is to look at the increase of probability in causing the outcome. Thus I also define the actual causation measure of causal sirength a
Co a, d') = Pr(A- a CNESS-causes 0 = A + -PAA = a' UNESS-causee 0 = =o[[A+ + a'D).
"Surprisingly, to my knowledge this rather obvious measure of causal strength has been overlooked so far in the literature. (For any definition of causation of course, not
not just CNESS.)

---

Taking into account that our Epistemic Condition is mixture of those of BvH and HK, I suggest the following definition, where the value of a expresses the relative importance of both measures.
Definition 11 (Degree of Responsibility), The degree of responsibility for O o of an agent who performs A = a w.r.I. responsibility setting (M, (,, E) is O in case the agent is not respon- sible, otherwise let S = Rrg Pr(O L| + a a')), and let a" = argminatss Pr(A= = a' CNESS-causes O = d[A + +a')) a'I), then d = CSe a") + a -ma(0, CS.c(o,a,a")).
Informally, this measure works as follows. Among all actions that minimize the probability of the outcome, we take one that minimizes the probability of causing the outcome, and then take weighted sum of both causal strength measures for that action (where the second measure is ignored if it is negative). This captures the idea that in order to avoid responsibility, the agent should choose an action that makes the outcome as unlikely as possible, and then further select their action so that it makes causing the outcome as unlikely as possible. The following example illustrates this definition.
Example 6. Imagine again our scenario from Example D but with the following change: Assassin, known to be a reliable assassin, whereas Assassing known to have second doubts and almost never shoots. In other words, it is reasonable for Assassing to expect that Assassin will shoot, and is reasonable for Assassin, to expect that Assossing will not shoot. On this particular occasion, both assassins shoot and kill victim.
Although both assassins are responsible according to my definition, it is easy to see that Assassin, is responsible to higher degree: the measures of actual causation are identical for both and so are their respective probabilities of the outcome occurring given that they shoot (namely 1), but Assassini 's probability of the outcome occurring given that they do not shoot is far lower, and thus
CSA.a1 (V 1, = A, = 0) > CS (V = 1, A2 = 1, A2 = 0).
Interestingly, recent studies offer empirical confirmation that the agent's epistemic state does indeed impact people's 's judgments in precisely this way: in disjunctive scenario (like ours), an agent who performs an action that is typical (for them) considered to be more responsible than an agent who acts atypically
[18. The authors contrast this disjunctive scenario, which they have trouble explaining,
with a conjunctive one in which both agents' actions are necessary for the outcome to occur, which their account explains quite well. In a conjunctive scenario (in other words, if the equation were V Ai ^ A2), an agent who performs an action that is atypical is considered to be more responsible than an agent who acts typically, flipping the judgments compared to the disjunctive scenario. That is also the verdict of my degree of blameworthiness: in this case, the atypical agent can reasonably expect the outcome to depend on them performing the action whereas the typical agent can reasonably expect that their action has little impact, which translates into a larger measure of causal strength (both CS. and (S for the former. So in contrast the account of Kirfel and Lagnado [18], my proposal applies equally to both scenarios and can thus be seen as a formal extension of their work.
7 Conclusion and Future Work
Based on a comparison with the work of ByH and HK, I have offered novel formal definition of moral responsibility that is particularly suited for AI systems by filling in the causal and the epistemic conditions. I used contrasting examples to argue in favor of the Counterfactual NESS definition of causation over the NESS and the HP definition, and in favor of a nuanced epistemic condition that combines the two conditions of ByH and HK. I connected this work to measures of causal strength to define a degree of responsibility. This quantified approach can be further enhanced by also taking into account the robustness of causation, which recent research suggests plays role in responsibility judgments that is somewhat independent of causal strength [14), as well as by considering the collective responsibility of groups of agents (6 [ Lastly, diseussed, a formal definition of responsibility is a necessary prerequisite for definitions of blame and praise. To develop definitions of the latter requires incorporating harm and benefit 3 4) and possibly also intention. Therefore the current definition can be extended in several ways, which I aim to do in future work.
7 'The superscripts Ass; indicate that we are using each agent's subjective probabilities to assess their degree of responsibility.
10

---

Acknowledgments and Disclosure of Funding
Many thanks to Hein Duijf for helpful comments on an earlier version of this paper, as well as the Neurips reviewers for their constructive criticism of the original submission. This research was made possible by funding from the Alexander von Humboldt Foundation.
References
[1] Beckers $ (2021) Causal sufficiency and actual causation. Journal of Philosophical Logic 50:1341-1374
[2] Beckers S (2021) The counterfactual NESS definition of causation. In: Proceedings of The Thirty-Fifth AAAI Conference Artificial Intelligence (AAA1-21), pp 6210-6217
[3] Beckers S, Chockler H. Halpern JY (2022) A causal analysis of harm. In: Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS-22)
[4] Beckers S, Chockler H. Halpern JY (2023) Quantifying harm. In: Proceedings of the 32 International Joint Conference on Artificial Intelligence (IICAI-23)
[5] Braham M, van Hees M (2012) An anatomy of moral responsibility. Mind 121(483):0116634
[6] Braham M. van Hees M (2018) Voids or fragmentation: Moral responsibility for collective outcomes, The Economic Journal 128(612):95-11
[7] Cooper AF, Moss E, Laufer B, Nissenbaum H (2022) Accountability in an algorithmic society: Relationality, responsibility, and robustness in machine learning. In: Proceedings of the 2022 ACM Conference Fairness, Accountability, and Transparency (FaccT-222,, pp 864-876
[8] Dastani M, Yazdanpanah V (2022) Responsibility of ai systems. Al and Society (38):843-852
[9] Duijf H (2023) A logical study of moral responsibility. Erkenntnis
[10] Edmonds D (2015) Would You Kill the Fat Man?: The Trolley Problem and What Your Answer
Tells Us about Right and Wrong. Princeton University Press
[11] European Commission (2021) Artificial
intelligence
act. URL
https://
arrfficchllnteellgnncecc... wttp://ccaactttteeeaaaca/ecaaaee
[12] Fischer JM, Ravizza M (1998) Responsibility and Control. Cambridge University Press
[13] Fitelson B, Hitchcock C (2011) Probabilistic measures of causal strength. In: Illari RF P M, Williamson (eds) Causality in the Sciences, Oxford University Press, pp 600-627
[14] Grinfeld G, Lagnado D, Gerstenberg T, Woodward JF, Usher M (2020) Causal responsibility and robust causation. Frontiers in Psychology
[15] Halpern Pearl J (2005) Causes and explanations: A structural-model approach. part I: Causes. The British Journal for the Philosophy of Science 56(4):843-87
[16] Halpern JY (2016) Actual Causality. MIT Press
[17] Halpern JY, Kleiman -einerr M (2018) Towards formal definitions of blameworthiness, inten- tion, and moral responsibility. In: Proceedings of The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), 1853-1860
[18] Kirfel I., Lagnado D (2021) Causal judgments about atypical actions are influenced by agents' epistemic states. Cognition 212
[19] Kroll JA (2020) Accountability in computer systems. In: Markus Dubber FP, Das S (eds) Oxford Handbook of the Ethics of AI, Oxford University Press
[20] McIntyre A (2018) Doctrine of double effect. The Stanford Encyclopedia of Philosophy
[21] Moore MS (2009) Causation and Responsibility, Oxford University Press, Oxford, U.K.

---

[22] Pearl (2000) Causality: Models, Reasoning, and Inference. Cambridge University Press
[23] Peters J, Janzing D, Schölkopf B (2017) Elements of Causal Inference Fooundations and Learning Algorithms. MIT Press
[24]
Rosenberg I, Glymour C (2018) Review of joseph halpern, actual causality, BJPS Review of Books
[25] Rudy-Hiller F (2022) The epistemic condition for moral responsibility, The Stanford Encyclo- pedia of Philosophy
[26] Sartorio C (2007) Causation and responsibility. Philosophy Compass 2(5):749-765
[27] Sartorio C (2017) Ignorance, alternative possibilities, and the epistemic conditions for responsi- bility. In: Peels (ed) Perspectives on Ignorance from Moral and Social Philosophy, New York: Routledge
[28] Sprenger (2018) Foundations for probabilistic theory of causal strength. The Philosophical Review pp 371-398
[29] Vallentyne P (2008) Brute luck and responsibility. Politics, Philosophy and Economics 7:57-80
[30] Weslake B (2015) A partial theory of actual causation. The British Journal for the Philosophy of Science forthcoming
[31] Woodward (2003) Making Things Happen: A Theory of Causal Explanation. Oxford Univer- sity Press
[32] Wright RW (1985) Causation in tort law. California Law Review 73:1735-1822
[33] Wright RW (2011) The ness account of natural causation: A response criticisms. In: Goldberg (ed) Perspectives on Causation, Hart Publishing
12

---

Appendix
Frankfurt-Case
The following is an example of a so-called "Frankfurt-case" taken from HK. An enormous literature in philosophy is devoted to dealing with these kinds of examples, attempting to reconcile intuitions about responsibility with the counterfactual and causal features that these examples contain. Surprisingly. almost none of it uses causal models, and yet doing so reveals the causal structure to be entirely unproblematic.
Example (Frankfurt). Imagine Jones poisons Smith, who dies. Unbeknownst Jones, Black was observing his behavior: if Jones had not poisoned Smith, Black would have given Jones gun and manipulated him in some way or other so that Jones would shoot Smith. Black both perfect observer and manipulator of Jones's 's behavior, and is thus guaranteed to succeed in his plans. Intuitively it is clear that Jones is responsible for Smith's 's death, dexpite the fact that he could not have prevented it. (Typical Frankfurt cases focus on responsibility for an action, as opposed to responsibility for the consequence of an action, and therefore scenarios are normally formulated such that Black manipulates Jones to perform the same action. Except for the from the action to the consequence though, those cases are structurally isomorphic. [S]
The Epistemic Condition of both BvH and HK is obviously fulfilleddd for Jones believes that Smith's s death is completely dependent on his poisoning. We consider the following equations to assess the causal condition: SD = JPv v JS to capture the fact that Smith dies (SD) if either Jones shoots (JS) or poisons (JP) him. JS BM to capture that Jones shoots only when Black hands him a gun and manipulates him (BM), and finally BM = =J to capture that Black's 's action depends on Jones's s failure to poison.
Regardless of whether we apply the NESS definition, the CNESS definition, or the HP definition, J 1 comes out as a cause of SD L, and thus the Causal Condition is satisfied. (This is easy to see by observing that the structure of this example is standard case of Early Preemption.)
ByH claim that their account can handle Frankfurt- cases like this, but that is a mistake. Recall that their variables represent the agents' strategies rather than their actions, and that we are limited to using single equation. The outcome function they use when discussing Frankfurt-case is equivalent to the equation SD = JPV v B. where B represents Black adopting his preferred strategy. Therefore on their account both Jones and Black come out as causes of Smith's 's death, which is not sensible result. ByH admit that their NESS definition is unable to handle conditional strategies like that of Black, but contend that since we are here focussing on Jones this is not problem. Obviously simply stating that one should focus on the sensible results of one's 's theory is not satisfactory way of defending it... (This example also highlights a more philosophical problem with their approach: is not at all clear what it means for a strategy to be cause. The broad consensus is that causal relata are either events/omissions or properties of events, whereas conditional strategies are neither.)
Counterexample the HP-definition
We here consider a second counterexample to the HP definition that was suggested in (24. The example is of particular interest as it was presented precisely within the context of the relation between causation and moral responsibility.
Example 8. We have equations = D and X D, and we consider a context such that D = 1. This looks very much like standard case of overdetermination in which X 1 and D 1 are both overdetermining causes. Yel X 1 is not an HP-cause of = 1 (and it is a CNESS-cause).. The reason for this is that Y 1 depends counterfuctually on D = by üsself, whereas it does not depend on X = by itself and nor does it when we take D 1 our witness W w. Rosenberg & Glymour (24) argue that this result shows the HP definition cannot offer a basis for moral responsibility, by offering the following scenario to go along with these equations:
*This analysis can just as easily be applied to these more typical Frankfurt cases. Still, those who are sceptical that proponents of Frankfurt cases are equally comfortable as am with moving from actions to consequences, 1 point out that Fischer & Ravizza apply this shift in exactly the same manner as I do when discussing responsibility for consequences [2 ch. 4].
13

---

An obedient gang is ordered by its leader to join him in murdering someone, and does so, all of them shooting the victim the same time, or all of them together pushing the plunger connected to homb. The action of any one of the gung would suffice for the victim's 's death. If responsibility implies causality, whom among them is responsible? Halpern's is theory says the gang leader and only the gang leader is a cause of the victim's 's death. This is a morally intolerable result; absent a plausible general principle severing responsibility from causation, any theory that yields such a result should be rejected.
Bombing
We now go through the details for the Bombing example. (Ex. 6 We need to consider the following four scenarios:
1. S2 and S 0
2. S2 = and S 1
3. S2 = and S 0
4. S2 = and S
We first go through the details for CNESS-causation.
In scenario 1 we have that S D1 D3 and S2 D22 B 1. Here (S₂ 1, S1 = 03 is
sufficient for D2. whereas {S] = =0} is not. Therefore S2 = directly NESS-causes D2 = 1. Clearly also D2 1 directly NESS-causes
and thus S₂ 1 NESS-
B along (S2, Dz, B).
What about the counterfactual setting (Ms,-o, u)? That corresponds to scenario 3. There the bomb doesn't 't even explode (so B 0), and thus there are no causes of B We conclude that in scenario
1 1.2 1 SINESS-causes B 1.
In scenario 2 we have that S S² D3 B ==1 and Di = D2 In this scenario B is directly NESS-caused only by D3 = 1. Since Sn 1 does not directly NESS-cause D3 = 1, it is not a NESS-cause of B 1.
In scenario 4 we have that S = D₃ = D3 = B = 1 and S2 = D2 = 0. Here (S₂ 0, = 1} is
sufficient for D1. whereas {S1 = 1} is not. Therefore S2 directly NESS-causes D1 1. Clearly also D1 1 directly NESS-causes What about the counterfactual setting (Ms,-1, ? That corresponds scenario 2, in which S2 does not NESS-cause
1, and thus S2 0 NESS-causes B 1 along (S2, D2, B).
1. So S2 0 CNESS-caused
in scenario 4.
As result, if Assassing chooses S2 1, the probability of CNESS-causing B = is the probability that S = 0, which is 0.4. By contrast, if Assassin; chooses S₂ = 0, the probability of CNESS- causing B = 1 is the probability that S 1, which is 0.6.
NESS-causation for each scenario is already discussed in the above, so we move on to consider HP-causation. In scenario 1 we have counterfactual dependence of B 1 on Sy 1, and it is well-known that this suffices for HP-causation (as well as for ENNSS-caausatoon by the way 22).
In scenario 2, note that D3 suffices for = 1, and thus satisfying AC2 is possible only when either D3 = 1 or S1 = 1 is also part of the candidate cause X ž. However, B = 1 counterfactually depends on D3 1, meaning that D3 is cause all by itself. Thus {S2 = 1, D3 1} is not minimal, and because of AC3 this means that it is not cause. That leaves {S2 = S1 = 1}. But this is not minimal either, for S = is a cause all by itself: one can take W = { as a witness to get B = 0 when Si set to Therefore S2 is not part of any cause of B 1.
Since B = 0 in scenario 3, S2 = does not HP- HP-cause B = there either, leaving scenario 4. As with scenario 2, the candidate cause will have to include D3 1 or S = 1. Contrary scenario though, D3 = 1 is no longer a cause by itself, since D holds, and will remain to hold also when we set D3 to 0. Since B counterfactually depends (S2 D3 1}, we get that each of them HP-causes B = 1.
14


Reviews:
[{"review_id": "1opSfS7oI1", "reviewer": "Reviewer_7Pui", "summary": "The paper aims to link causal and moral responsibility through formal modeling using structural causal models. Engaging with existing literature on causality, the paper argues for the benefits of certain definitions over others in introducing a 3-part _responsibility schema_ based on a control condition (that an agent had control over the causal action), a causal condition (that the controlled action caused the outcome), and an epistemic condition (the agent believed that their control choice minimized the probability of an alternative, less preferred outcome, thereby avoiding responsibility for the actual outcome). In support of this model, the paper shows several example scenarios that differentiate existing definitions that could be used to operationalize this schema.", "questions": "* Where do the authors see their contribution and how do they set off alternative approaches in other literatures? What of other concepts such as accountability, liability, or answerability for outcomes?\n* Is moral responsibility equivalent to causal responsibility in a modeled context? Why or why not?\n* How does the offered model manage situations where causes aren't instrumental but rather structural and systemic? What of multiple-causation, both in general and in these contexts?", "limitations": "The work should overall be better contextualized around the full richness of the problem used to motivate it.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "+ The linking of causal and moral responsibility is useful and important.\n+ Connecting reasoning about values to formal models is difficult but a promising research direction.\n+ The paper engages deeply with existing definitions of causality and its relationship to agents' beliefs.\n+ The paper is well organized in support of its contribution, the responsibility schema.", "weaknesses": "- The paper's view of moral responsibility is not presented as substantially distinct from causal responsibility and the connection to human values or moral philosophy is extremely weak. This is disappointing given that there is a large literature on linking causal responsibility to the value of accountability _within the context of AI systems_ that has been developing for many decades and now constitutes one of the main lines of thought in two major venues (ACM FAccT and ACM/AAAI AIES), which are wholly ignored in the discussion and citations (see, e.g., various works on accountability of, e.g., Kroll and also various works on liability of, e.g., Selbst). I think this context should be acknowledged, at a minimum. If it's not relevant, that's acceptable but the paper should say why and mention it as an alternative approach.\n- Although the paper aims to link causal and moral responsibility, it ignores several classical lines of thought attacking this question. For example, a classical question in law going back hundreds of years is the distinction between causal or moral responsibility and _liability_, the question of which agent the law should assign responsibility to (a theory in which many long-discussed cases directly contravene examples in the paper). Relating these ideas to computing systems, nearly 30 years ago, Nissenbaum's classic \"Accountability in a Computerized Society\" addressed the question of when causal and moral responsibility could be separated and why they often can't be _in computing systems specifically_. There is even a recent re-establishment of this argument by Cooper, Moss, Laufer, and Nissenbaum revisiting the framework in light of data-derived decision rules. Again, the fact of an alternative approach should inform the location in the literature of the paper's contribution. What is the relationship between the proffered responsibility schema and values such as accountability, regimes such as liability, or concepts such as moral responsibility distinct from causal responsibility?\n- I think the paper, by sticking too closely to the details of definitions in the literatures of formal causal calculi and structural causal modeling, falls victim to some of the standard criticisms of this literature and specifically of tying its results to reasoning about human values. If the paper wishes to claim such a tie as a contribution, this context must also be reckoned with. For example, authors such as Hu, Kohler-Hausmann, Miller, and Kasirzadeh and Smart have in the last few years attempted to build theories that deal with multiple-causation and \"structural causation\" (here, structural in the sense that the cause is inherent to the shape of an entire system, rather than in the sense that the structure of the graph model informs which variables are or are not causally relevant). Such approaches resist reducing causation to even a set of variables or to sufficient conditions, and have proven useful in understanding things which are otherwise difficult, such as describing the plausibility of different counterfactuals. Again, there is an alternative approach in the literature and the contribution would be strengthened by acknowledging it and setting off the contribution in light of existing work. Not everything tangentially relevant has to be cited, of course, but the paper should make clear where its contributions lie and why other work with similar goals is different.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed engagement with our paper, but we fail to understand why they recommend rejection. The core contribution of our paper is an analysis that is an improvement over two important existing analyses. As they do not point to any concrete flaws in our analysis, it appears the reviewer’s judgment is based mostly on the absence of a more detailed integration into the broader literature. Although we are unsure how we could do justice to such an integration in a short conference paper, we will add more context and references to better situate our contribution.\n\nQuestions:\n\nQ1: \nThe reviewer is completely right that there is more to responsibility than what is discussed in our paper. Moreover, they are also right in pointing out the importance of many related concepts. The aim of our paper is therefore fairly modest, in the sense that it focusses _only_ on responsibility for an outcome in virtue of performing an action. Furthermore, we point out that our notion of responsibility in and of itself leaves open entirely whether the agent is thereby blameworthy or praiseworthy: instead the definition here presented is a necessary condition for such further judgments, but certainly not a sufficient one. Therefore our definition is in no way meant to offer an _exhaustive_ theory of all kinds of responsibility for AI systems, let alone a theory of accountability and other related notions. Rather, we see our work as filling in just one element of such a theory, which we believe can be integrated into many existing theories out there. (See below for more details.) We will be more explicit on this point.\n\nQ2: \nIf causal responsibility means “being an actual cause”, then in our view — and for the notion of responsibility that we are focussing on! — such causal responsibility is a necessary condition for moral responsibility, for it is captured by our Causal Condition. We take up this view from the philosophy literature, and it is also well-supported by the law. Given that, on our view, moral responsibility also requires meeting the other two conditions, it is therefore not equivalent to causal responsibility. \n\nQ3: \nOur definition exclusively focuses on actual (aka token) causation, and exclusively on variables which represent the actions of an agent. Thus, at first sight, any responsibility that is owed due to there being structural or systemic causes (which are, we take it, not actual, and are not actions), is not covered by our definition. If the reviewer has in mind a particular kind of example where this seems problematic, we would be very much open to discussing it. \nRegarding multiple causation: since the only causes that are relevant to our definition are the actions of agents, issues of multiple causation only show up in case of multiple actions. Our Assassins example in which both shoot can be seen as an instantiation of multiple causation. In the current paper we only judge their responsibility individually.\n\nRemarks:\n\nWe do not see the connection to moral philosophy as being very weak, given that we are starting out from a responsibility schema that is derived directly from the standard literature in moral philosophy. We therefore disagree with the reviewer that this schema captures the essence of our contribution: our contribution lies in _filling in_ this schema in a manner that we argue is superior to existing accounts. Furthermore, we do not consider this to be an _alternative_ to the important work that the reviewer mentions: it is merely one piece of the full picture that should inform the moral evaluation of AI systems. \n\nWe are unclear as to what the reviewer means when they state that there exist classical lines of attack on the view that moral responsibility is closely connected to causation. (To reiterate, we make no claim to exhaustivity: we acknowledge that there exist notions of responsibility that are not so closely tied to causation.) They refer to work on liability, which is not the topic of our paper, and they refer to work by Cooper et al that seems to support the existence of a close connection. In fact, we looked into the article mentioned, and we believe our work could be very beneficial for the issues there identified. For example, they state that \"Blame, defined in terms of causation and faultiness, is assigned to moral agents for harms they have caused due to faulty actions\", which is entirely in line with our view (see also our reply to reviewer ir7c regarding harm). Later they say that \"in practice, if a trained model causes harms, it can be extremely challenging to tease out particular actors who should answer for them.\" Our work is meant precisely to tease out those actors, and thus we see it as fitting perfectly within the challenges identified by Cooper et al. The reviewer also points to the work by Kroll, and there too we see a connection, as he explicitly focusses both on causal responsibility as well as on the agent’s ability to perform alternative, preferable, actions, as offering one important way to understand accountability. So we fail to see how our work is in conflict with the literature on accountability that the reviewer is referring to. \n\nLastly, our reliance on causal models is a consequence of their widespread use in the study of causation, both in AI as well as in philosophy. Despite their limitations, causal models are still the most popular (and successful, we might add) formalism for addressing causal issues. We would like to learn more as to which criticisms the reviewer has in mind in particular, because as far as we know, these criticisms mainly target counterfactuals and causation that involve causally suspicious variables such as race or gender, and do so in causal models that partly aim to capture the causal structure of society. Our approach does no such thing: we focus on mundane cases of actual causation (rather than structural or type causation), in which the actions of agents cause specific outcomes.\n"}, {"review_id": "1sBxlQz4Ag", "reviewer": "Reviewer_ir7c", "summary": "This paper presents a novel formal definition of moral responsibility tailored for AI systems, filling in both causal and epistemic conditions. The work effectively draws comparisons to BvH and HK's works, favoring the Counterfactual NESS definition of causation and a nuanced epistemic condition. ", "questions": "I would like to see more discussions on the application of the proposed definition in real-world scenarios.", "limitations": " The authors have acknowledged that future work could enhance the quantified approach by considering collective responsibility and degree of causation. However, practical application and testing of the proposed definition are yet to be performed.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": " The proposed formal definition of moral responsibility for AI systems is a novel contribution. The paper contrasts several existing theories of causation and epistemic conditions to make its case, demonstrating a thorough understanding of the existing literature. The submission is clearly written and well organized. The authors articulate complex concepts with clarity and a lot of interesting examples.", "weaknesses": "There is a need for further elaboration on how this proposed concept can be practically applied and measured in real-world AI systems. Also, the work would be more convincing if it incorporated a broader range of philosophical perspectives on moral responsibility. Lastly, while the paper talks about future work related to defining blame and praise, these aspects are not explored in the current submission, leaving an incomplete picture of the potential implications of their framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments, and their positive evaluation of our paper!\n\nRegarding their question about real-world scenarios: we actually completely agree! We would also like to see more discussion of our definition to real-world scenarios, and we very much would like to offer this in future work. In particular, there is now a very large empirical literature in experimental philosophy and in psychology on people's actual judgments of responsibility and causation in many real-life scenarios. Although there is an abundance of empirical results, there is much disagreement on their interpretation. Concretely, there is disagreement on whether -- and how -- causal judgments can be separated from judgments of responsibility, on what the role is of the epistemic states of the agent (see in particular the recent work by Kirfel and Lagnado that we discuss in the supplementary material), what the role is of  probabilities and background conditions, and on what the right view is on actual causation. In the supplementary material we only briefly sketch how our work could shed some new light on all of this. We would very much like to write a follow-up paper in which we apply our definition to this vast literature, because we are optimistic that it could offer a more principled interpretation than what is currently on offer.\n\nOf course there also exist other kinds of real-world scenarios that we should look into, namely the kind of scenarios in which an AI system might realistically be deployed to make the kind of decisions to which our definition would apply. Autonomous vehicles and the oft-discussed trolley cases come to mind (although these would require extending our definition to include a focus on multiple outcomes), as well as the use of autonomous weapons and the moral requirements that are implied by principles of just war, to name just a few.\n\nMore generally, recently Beckers, Chockler, and Halpern have developed \"A Causal Analysis of Harm\" (NeurIPS, 2022), which could be integrated into our framework as well (except that we would choose to replace their definition of causation with the CNESS definition). At first thought, one might speculate that an agent who performs action A is blameworthy for some outcome O  whenever: \n- the agent in question is responsible for O in virtue of performing A (which is the focus of our analysis), and\n- A caused harm to another agent, where the harm caused is due to A causing O (which is the focus of Beckers, Chockler, and Halpern). \n\nLastly, as we have outlined in our response to reviewer 7Pui, we completely agree that our definition focuses only on one kind of responsibility and should be placed within a broader landscape that discusses important related (but distinct) notions such as blame, accountability, liability, and others. We will be more explicit in the paper regarding the relation of our work to this broader landscape. \n\n\n\n\n\n"}, {"review_id": "PcJYTtXOMB", "reviewer": "Reviewer_UYHL", "summary": "This paper formalizes the notion of responsibility of an agent by using\ncontrastive necessary element of a sufficient set causation (CNESS) for the\ncausal condition and Halpern and Kleiman-Weiner's notion epistemic\nresponsibility.  The resulting formulation of responsibility benefits from the\nstrong alignment of CNESS with intuitions of causation/responsibility vis-a-vis\nsimple NESS and Halpern and Pearl-causation.  The paper extends responsibility\nfrom a binary variable to real-valued degree of responsibility.\n", "questions": "How is formulation of responsibility particularly well-suited for AI?  What\nimpacts could it hav?\n\n## Comments\n- `Example 2` The paragraphs of italicized text are a bit distracting.\n", "limitations": "If arguing for a definition of moral responsibility, it is important to at\nleast briefly touch on the societal impacts of the work vis-a-vis the\ncontributions.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "## Originality\n## Quality\n- `[major]` Incorporates the strengths of multiple components to formulate an effective formalization of responsibility.\n- `[minor]` Formulation allows for degrees of responsibility.\n## Clarity\n- `[minor]` Examples illustrate the differences between theories of causation and epistemic responsibility.\n## Significance\n", "weaknesses": "Although the formulation of moral responsibility is sound on philosophical\ngrounds, the paper does not make a strong case for how this applies to AI.\nThus, this paper seems out-of-scope for NeurIPS.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We must say that we were very surprised to read this review. The reviewer recommends a strong rejection, and yet they fail to point out a single flaw in our paper. (For completeness, we should point out that their summary does contain a flaw, as we do not use HK's epistemic condition but rather present a novel condition of our own.) Moreover, the entire basis for their verdict lies in their comment that they fail to see how responsibility is relevant to AI, and thus they judge our paper out of scope for NeurIPS. \n\nIs the reviewer not convinced that the ethics of AI is a crucially important topic to do research on, and thus deserves a place at the most important conference on AI? Because if so, then we must firmly disagree (and as reviewer 7Pui outlines in their review, there is a wealth of literature within AI to back this up): we believe that addressing the ethical challenges posed by the use of AI are of vital importance and require much more rather than less research, given that the stakes could not be higher. Moreover, we are confident that this is also the predominant view nowadays within the AI community, and we are therefore at pains to understand the reviewer's succinct and dismissive judgment of our paper. \n\nWe develop a notion of responsibility that is formulated using nothing but causal models and probabilities, both of which are well-established formalisms within AI, and one that only requires a very minimal notion of agency to be applicable, thus making it well-suited for the kinds of artificial agency that one could attribute to even rather limited AI systems. We see this as well within the scope of NeurIPS, and we therefore kindly ask the reviewer to explain why they believe otherwise."}, {"review_id": "Nm06kQ2Y9r", "reviewer": "Reviewer_ZjtP", "summary": "In this paper the authors put forward a formalised definition of moral responsibility that can be applied to AI systems. To do so, they compare and contrast how existing definitions fare with regard to a causal and an epistemic condition for responsibility. Using these contrasting cases, the authors argue for a form of counterfactual ‘Necessary Element of a Sufficient Set’ definition of causation, and combine two existing definitions of the epistemic condition.", "questions": "My questions are general conceptual questions, not ones that necessarily need to be addressed.\n\nAt the end, the authors indicate that the quantified approach taken could be applied to collective responsibility. How do the authors anticipate the approach dealing with the epistemic condition regarding collective responsibility, in particular? What about institutional agents and responsibility?\n\nBlame and praise are inherently most value-infused than a causal notion of responsibility. How would the authors proceed to capture these aspects, and is it necessary?", "limitations": "n/a", "rating": 7, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I should note that I am less able to assess the technical aspects of the paper, and so I focus on the conceptual aspects.\n\nOriginality: The paper advances a novel formalisation of responsibility, which is contrasted clearly and well with existing definitions of the causal and epistemic conditions for responsibility.\nQuality: As I’m not less able to assess the technical aspects of the paper, I do not believe that I am well-placed to comment on the technical quality. Conceptually, the paper is strong.\nClarity: The paper is well-written. (Very minor, but after each example large sections of text are left in italics. This makes it difficult to read and to distinguish between the example and the discussion of the example.)\nSignificance: Both the causal and epistemic conditions for responsibility are debated in the wider literature, where a challenge is to capture important nuances in formalisations. If successful, the contribution is significant.", "weaknesses": "I do not feel competent to comment on the technical substance of the paper.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive and encouraging feedback! \n\nThe reviewer raises a good point that the epistemic condition in particular poses some challenges once we generalize to a notion of collective responsibility. This issue is not novel though, as research in social epistemology (and in social choice theory) also focusses on collective epistemic states, so in first instance we could consult this literature for inspiration. One option would be to integrate communication between the agents into our framework, which definitely seems realistic within the context of AI systems (just think of autonomous vehicles, or swarms of drones, for example). Thus, the various agents can coordinate their actions, and in this sense be treated more or less as if they form a single agent, somewhat similar to how the law treats corporations. But these are merely initial thoughts on the subject, we are definitely open to suggestions on this topic.\n\nRegarding blame and praise: as we mention in our paper, we take our definition to at least form a necessary condition for blame and praise. We offer one line of thought on how to construct a definition of blame in our reply to reviewer ir7c, when discussing how the recent work \"A Causal Analysis of Harm\" could be combined with our approach to form a definition of blame. (We add a bit more detail in our reply to reviewer ir7c.) A natural further suggestion would be to do the same for the notion of benefit, giving a definition of praise. We would need to integrate further conditions than merely harm and benefit though, because in and of themselves those do not deal with the issue of how to compare and weigh off multiple outcomes (think of trolley cases, the doctrine of double effect, etc.), or on how to integrate the \"cost\" that an agent incurs by performing an action, nor does it fully capture what duties and obligations an agent might have. In other words, generalizing to blame and praise would be an exciting but far from trivial avenue to pursue.\n\nLastly, we apologize for the long sections of italics, as we now realize they are not very readable, we will correct this in the final version.\n\n"}]

Author rebuttal:

Meta-review: This paper offers a fresh perspective on the moral responsibility of AI systems, providing an interesting and clearly delineated new framework. Besides, it could benefit from a deeper exploration of practical applications and a more diverse philosophical backdrop. It has a good foundation, but branching out could really make it stand out. Overall, the meta-reviewer's recommendation is acceptance as a poster.
Decision: Accept (poster)