# Are Intelligent Agents More Ethical?

This post is a response to a claim by Scott Sumner in his conversation
at [LessOnline](https://less.online/) with Nate Soares, about how
ethical we should expect AI's to be.

Sumner sees a pattern of increasing intelligence causing agents to be
increasingly ethical, and sounds cautiously optimistic that such a trend
will continue when AIs become smarter than humans. I'm guessing that
he's mainly extrapolating from human trends, but extrapolating from
trends in the animal kingdom should produce similar results (e.g. the
cooperation between single-celled organisms that gave the world
multicellular organisms).

I doubt that my response is very novel, but I haven't seen clear enough
articulation of the ideas in this post.

To help clarify why I'm not reassured much by the ethical trend, I'll
start by breaking it down into two subsidiary claims:

1. The world will be dominated by entities who cooperate, in part
because they use an ethical system that is at least as advanced as ours.

2. Humans will be included in the set of entities with whom those
dominant entities cooperate.

Claim 1 seems well supported by trends that economists such as Sumner
often focus on. I doubt that Nate was trying to argue against this
claim. I'll give it a 90+% chance of turning out to be correct.
Sumner's point sounds somewhat strong because it focuses on an
important, and somewhat neglected, truth.

Claim 2 is where I want to focus most of our concern. The trends here
are a bit less reassuring.

There's been a clear trend of our moral circle expanding in the
direction that we currently think it should expand. How much of that
should we classify as objective improvements versus cultural fads? Claim
1 is often measured by fairly objective criteria (GDP, life expectancy,
etc.). In contrast, we measure expansion of our moral circle by the
criteria of our current moral standards, giving us trends that look
about as good if they're chasing fads as they do if the trends will
stand the test of time.

Gwern provides [extensive pushback](https://gwern.net/narrowing-circle)
against strong claims that moral circle expansion is a consistent trend.

I'll add one example: children have increasingly had their freedom to
wander restricted during my lifetime (see the free range parenting
movement). It's almost as if they're considered to be like zoo
animals, with their caretakers optimizing for safety at the expense of
happiness. I don't find it hard to imagine a future where AI treats us
like that.

The obvious versions of the moral circle expansion hypothesis suggest
that we should expect human societies to soon grant moral patienthood to
animals.

Blindly maximizing the size of our moral circle would be [more
problematic](https://experiencemachines.substack.com/p/moral-circle-calibration)
than maximizing cooperation. It's pretty unlikely that we will want to
expand our moral circle to include pet rocks. It sure looks to me like
moral circle expansion has been driven largely by pragmatic evaluations
of costs and benefits, with only a modest influence from increasingly
principled altruism.

Given this uncertainty about how closely our moral circle expansion
approximates attaining an objective moral truth about who should be a
moral patient, we ought to be more uncertain about it than we are about
the continuation of economic progress.

I expect that whether humans remain in the moral circle depends on
hard-to-predict factors, such as the costs associated with interacting
with humans, or whether AIs have relevant altruistic goals.

I recommend looking at examples such as human interactions with cats and
chickens as possible analogies for how beings with very different
cognitive abilities might interact. My impression is that increasing
human intelligence does not make those interactions more ethical, but
increasing human wealth weakly tends to make the interactions more
ethical. Humans seem mostly aware of ethical concerns regarding factory
farmed chickens, yet their reactions seem mostly influenced by cost
considerations rather than improved ethical insights.

So I'm only weakly reassured by the trend that Sumner sees.