The end of the evolution strategy brought
about by technological advance- Seeking for a new form of sustainably
１６：００－１７：３０

developing society of living things -Hiroshi Yamakawa

EcSIA: Desirable future coexisting with AI
In a desirable future, the happiness of all humans will bebalanced against the survival of humankind under the purview
of a superintelligence. In that future, society will be an
ecosystem formed by augmented human beings and various
public AIs, in what I term an ecosystem of shared intelligent
agents (EcSIA).
Although no human can completely understand EcSIA—it is
too complex and vast—humans can control its basic directions.
In implementing such a control, the grace and wealth that
EcSIA affords needs to be properly distributed to everyone.
（Hiroshi Yamakawa, July 2015)

UNILATERAL INCREASE OFARTIFICIAL EXISTENTIAL RISKS

Threat of universal unilateralism
Emerging technologies increases disruptive capabilities to agents(from nations to individuals) .
That leads to a rapid increase in the number of agents that can resultin global catastrophic risk. (→ Agential risk )(a) Dual usability: Technologies can be used on the morally bad side(b) Power: Increasing influences(c) Accessibility: The minimum IQ required to destroy the worldcontinues to decline
Torres, P. Superintelligence and the Future of Governance: On Prioritizing the Control Problem at the End of History. In
Artificial Intelligence Safety and Security; Yampolskiy, R.V., Ed.; Taylor & Francis Group: Boca Raton, FL, USA, 2018

Increases in artificial risk by technology is an issue• Fossil fuels: Climate change• Nuclear Energy: Mass Destruction, Nuclear Winter• Synthetic biology: a lethal bacterial pandemic• Nanotechnology: Grey Goo• Artificial general intelligence: Accelerating technology

development, Our loss of initiativeEnvironment and Intelligent agent• Suppose that we sent a group of agent that can breed tounknown planets.
• Fortunately, they can quickly gain energy sources and survive.
• If they survive after 1000 years, what properties should theytake?

https://www.pinterest.jp/pin/137641332348680426/

DOI: 10.5194/isprsarchives-XL-1-W2-299-2013

Stages of intelligence to survive in the environment
Various adaptive IAs exist in an autonomous decentralized mannerin order to survive against various environments and their changes.

Each IA has instinct
for individuals survive
and responds quickly
to the surrounding
environment.

AAAA
A

Environment

AIntelligent Agent

Impact range of IA

Stage that require social intelligence
Expanding knowledge (including the development of science and technology),
the influence of IA is enhanced. The influence as organization
becomes obvious.
Rapid knowledge sharing
progresses. Not only IA but
also the interaction between
organizations (coordination,control, conflict, etc) willincrease.

AAA

A

A
A

AAAA

A

A
A

EnvironmentAA
AIntelligent Agent

Impact range of AI
Impact range oforganizations

Stage of existential risk
The influence of IA further increases, the organization intertwines
complicatedly, intervention to the environment become big. Within finiteenvironments some IA 's impulsive behavior destroys the world.

AA

A

A

For survival of IA society,individual survival instincts
should be controlledAAAAIntelligent Agent
Impact range of AIAA
A

Environment

AImpact range oforganizations

World destroyed probability by extinction weapons
Probability of survival after oneyear• There are N agents who can push the switch to"Effective range of the weapon > the range ofthe agent's existence".
• All agents use the weapon with probability b
per unit time.
• The probability of survival (P) is the probability

that any of N agents does not push the switch.

10.8

0.6

0.4

0.2

0262144131072
65536
32768
16384
8192
4096
2048
1024
512
256
128
64
32
168421P ＝ (1-b)NProbability of survival: P

use an extinction weapon that satisfies the1.2

0.01

0.001

0.0001

4.7619E-05

0.00001

Probability: b

When b = 1/ 20,000 agents exist in 20,000, there is a half probability that they will become extinct within a year.

Phase change in evolutionary strategy
Decentralized autonomy

Diversification

Local competition

Advances in technology have made it easier to get more powerover a wider range.
Decentralized autonomy

Diversification

Global competition

Agential Risk → Existential Risk

Need to prevent impulsive/short-circuited vandalism

Can't humanity escape the reality of "struggle"?

• Environmental factor– Offensive realism– A race without a nation– Ethics is often ignored in confusion

The origin ofagential risks• Individual factor (aggressive desire)– The range of biologically embedded compatriots is narrow.
– Battles brings elations like festivals (R. Caillois)– Columbine High School shootings (1999)

ARTIFICIAL GENERAL INTELLIGENCETO AVOID THE EXISTENTIAL RISKSExistential Risk Avoidance Scenarios and Advanced AI
The ultimate diversification:
If our goal is supposed to be "survive of information," we can broaden
our choices of what we want to leave behind.

Relocating people to virtual space to preserve physical infrastructure:•Brain-machine interface, mind uploading, etc. are required

Expanding the range of survival:••

Restoring the locality of the world by spatially spreading
Economic space exploration technology still needs time to be establishedLive together on earth peacefully with help of friendly AIs:
• Global surveillance: Individual rights and freedoms are narrowed to someextent for safety• Global Governance: Curb the arms race between powers and prevent thedestruction of international public goods

Roles of Advanced AI•Various rout to build super intelligence
Machine Kingdom

SpecializedArtificial
neuralnetwork

GeneralOpen AIなど

DeepMind
Neurosimulator

Whole brainarchitecture
Whole branemulation
HumanBrain

SuperintelligenceAdvanced AIs can be a new colleague by sharing goals
Colleague = range of ethical universality = range of goal sharing

Scope of value to be shared
1. Brain implements value for survival of individual or relatives
2. Culture and education extended our scope to humanity
3. Could it extend to such as superintelligence, intelligent
animals, and extraterrestrial intelligence?

MachineKingdomCreatures
Kingdom

AnimalKingdom
Homosapience

Value systems in human brain

Organizations

Curiosityｔｙ
SocialRelatives
Individuals

Breeding(libido)
Stefan Koelscha, The quartet theory of human emotions:
An integrative and neurofunctional model, 2015.

Homeostasis

Intelligent agent

A sustainable ecosystem supported by AI society
In the short term, AI society will support ahuman-centered society• Rethink human rights (freedom, dignity,privacy, etc.) for the sake of security andallow global monitoring and arbitration byAI society as an appropriate level• Development of an autonomous anddistributed AI/AGI system thatconsistently maintains and sharescommon values is a technical challenge

Human society
Capitalism tends to widen disparity while increasing wealthCollect

BenefitArbitrate

Various values thatreflect individuals / groups
AI

Common values forthe all humanity

AGIAI
AI

AGIAGIAI

Stable and consistent AI society(Autonomous distributed AI system )(Yamakawa, H. Peacekeeping Conditions for an Artificial Intelligence Society. Big Data Cogn. Comput. 2019, 3, 34.)

The Friendly Supersingleton Hypothesis[Toress, 2018]

In post-Singularity, people can gain security by transitioning
to a society that has delegated the power to govern society
to Global Singleton, who is more favorable to humanity.
• Technical background: Computers will gain intelligencebeyond biological constraints and can control the world(Bostrom 2014; Yampolskiy 2016)– Quantitatively, Have advantages in terms of computation speed and memory capacity.
– Qualitatively, Can deal with concepts beyond human comprehension.

Singleton isn't necessarily perceived negatively.

Building a singleton is not easy way
Introverted intelligence
explosion.
Self-renewal ofphysical elements

Singleton
scenariodecisive strategic superiority

Scenario branch due
to stochastic
fluctuations

Relative superiority /Avoidance of locality
constraints

Extroverted intelligence explosion

Breaking through thethermodynamic limits

Superintelligence

Multipolar
scenario

Human level
intelligence
High-level autonomy
Present

Self-improvement

Ecologicalscenario
Upper-boundscenario

K. Takahashi, Scenarios and branch points to future machine intelligence, 1F3-OS-5b-03, JSAI 2018.
This figure was translated into English and modified with the permission of the author.

HOW "SOCIETY OF LIVING THINGS"CAN SUSTAINABLY DEVELOP IN A
WORLD WITHOUT LOCALITY?

How to safely sustain innovation in world without locality?
The possibility of achieving a safely developing society will be in thecollection of the individual with an adaptive value system pursuing thepreservation of society and the individual in a balanced manner.
Value system

Non-adaptive

Preservation

Adaptive• Deficit of developability • Innovative• Risky (related to theIndividual • Safe
Instrumental sub-goal convergence)
Most of the animalsHumanity• Deficit of developability • Innovative
Individual• Safe• SafeandSocial insects
A mixed ecosystem ofsociety
machines and humans

Sustainable development ecosystem in a globalized world①

Sustainable development ecosystem in a globalized world①
If competition forsurvival is localized,
evolutionary strategies
can work even withouta clear objective.

In times when the world waslarge, the survival of theindividual (or sub-group)facilitated the development oflife as a whole.

Sustainable development ecosystem in a globalized world②

Sustainable development ecosystem in a globalized world②
Insofar as individual
survival is pursued,
conflicts of interest areinevitable.

In a globalized context, one
impulsive intention has thepotential to destroy the wholeSustainable development ecosystem in a globalized world③

Sustainable development ecosystem in a globalized world③• Resource conflicts between agentsshould be resolved by "mediation• Mediation requires an agreed-uponvalue in society• "Survive of information" can be acommon value for every
evolutional thing

From ”A Theory Of Justice” (John Bordley Rawls)
J. B. Rawls says something like “In order to establish asingle ordered society for individuals with different

conceptions of the good that are the impetus forpotential conflict, some normative principle must be
established at a higher level than the level of thegood.” [Moriyama 2006]→ The need for meta-level normative principles
that go beyond specific values (the good) is statedSustainable development ecosystem in a globalized world④

Sustainable development ecosystem in a globalized world④

Those who can sharecommon values become
members of society.

Trust supporting
technology is neededPeace-keeping condition for autonomous distributed system

Situation in which every agents areconfident that "every other agent
intends a socially acceptable goal
Goals that contribute to a common
goal and do not realistically conflictwith the local values of other agents

Sustainable development ecosystem in a globalized world⑤

Sustainable development ecosystem in a globalized world⑤
power loop in group

Living things have pursued
the survival of theindividual for a long time.

Issues in Mutual Trust for Autonomous Distributed Agents
Agent

To maintain consistent goals insociety, goal management
system should solve threeproblems.

Agent

Local values
AAA

Goal
sharing

Division ofAgentroles

Problems ofLocal values• communication channelA• comprehension
Self• computationalcontrol
complexity

Local values(sub) goals
Target-means
decomposition

Monitoring
Agent

Agent

Local values

Local values
AA

Goalsharing

Common goals A

Arbitration

Sustainable development ecosystem in a globalized world⑥

Sustainable development ecosystem in a globalized world⑥

An ecosystem needs to be built
in which the individual feels
happier in the pursuit of socialvalues than in self-preservation.

Sustainable development ecosystem in a globalized world⑦

Challenges to build future society of living things
What is the new evolution of living things in a world without locality?
•Make each agent (individual with value system ) more informational entity– Lower maintenance costs will release us from obsessions with self-preservations– Software entities can be diversely designed• Hardware for supporting computations become social infrastructures– Making contributions to social infrastructure an incentive for each individual– Standardization in the pursuit of efficiency leads to a problem of vulnerability• “Survive of information" can be a common goal for every evolutional living thing•

Mechanisms to build/maintain trust among multiple agents are necessary•

Our humanity are blessed with the opportunity to open up a new sustainably
developing world of living thingRelated articles
Yamakawa, H., Future Society and Hardware Expectations
Realized by General Purpose Artificial Intelligence, Applied
Physics, 2020, Vol. 89, No. 3, p. 163-167
Will artificial intelligence be mankind's friend? Or...
(Interview with Hiroshi Nakagawa, Hiroshi
Yamakawa and others)

Yamakawa, Hiroshi, The fundamental question of humannature posed by general-purpose AI, Phronesis, April 2020