Responsibility in the Age ofAutonomous Machines
IEEE ARSO 2016
July 8, 2016
Shanghai

Minao Kukita
Graduate School of Information Science
Nagoya University

A fatal car accident by Tesla Model S
while the driver used autopilot.

According to Tesla• It is the first fatal accident after 130 million miles with autopilot
activated, compared to the US average of one fatal accident
every 94 million miles.
• Neither the driver (of Model S) nor the car didn’t notice thewhite body of the trailer against the brightly lit sky.
• Tesla’s autopilot system is “public beta phase” and theyrequire the driver to maintain control and responsibility forthe vehicle while the autopilot is activated.

Who is responsible?
• Tesla? who is selling a car still in “beta phase.”• The driver? who was informed of the imperfection of
the system.

• The question would be harder and far more serious if
the victim were pedestrian or other drivers, not thedriver of the autonomous car.

The concept of “responsibility”• “You are responsible” may mean either– that you are obliged to make sure that some good thing will
happen, or to prevent some bad thing will not,– that you are among the major factors which caused some(bad) thing,– or that you are to blame.

The concept of “responsibility”• Traditionally, only human adult can be held
responsible who is able to predict results of one’saction, to make autonomous decisions, and to control
the course of one’s action.

Responsibility gap• As for a highly complex and autonomous system, it isdifficult to identify who is responsible for its
behaviour, because…
– It is difficult for anyone to predict the behaviours of thesystem.
– Too many people are involved in its development anddeployment.
– Accidents will occur without any obvious fault of anyone.
– There will be no one who is controlling it.

The function of “responsibility”• The responsibility is embedded in our practice in whichpeople are obliged to do something, and if one fails
the obligation and does some harm to others, one willbe blamed for that.

The function of “responsibility”• Having this concept, we encourage ourselves to dogood to the society and discourage ourselves fromdoing harm.
• The concept of responsibility seems to havedeveloped because of this function.
• The responsibility gap will make it hard for theconcept to fulfill the function.

Emotions• Strong emotions, e.g. anger or guilt, accompany theclaim “You are responsible”.
• Recent results in psychology reveal that theseemotions are key to our morality; they are importantfactors driving people to moral behaviours.
• However in some situations, emotions get in the way
to rational, really beneficial behaviours.
Cf. Joshua Greene, Moral Tribes

Necessity for revision of the concept• It can be useless to follow our emotions and try to
identify the culprit that is responsible for the accident
caused by a complex, autonomous system.
• Given the original function of the notion, we have torevise it so that it will be useful in a society where
humans and autonomous machines coexist.
Concept Analysis

Concept Engineering

Suggestions• Put more emphasis on social decision making
concerning an autonomous system than on individual
persons involved in its development or its usage.
• Ethical and democratic risk analysis involving
various stakeholders is necessary in the process ofthe decision making.

Suggestions• Do not invest too much in manhunt, and instead
do our best to improve the system, though it will bedifficult not to follow our deeply wired emotions• Transparency and data-sharing is crucial. Do notallow huge or hasty companies to monopolise thedata.
• The duty of compensation should be distributed tobeneficiaries, including users, manufacturers, and thesociety itself.

Thank you!