March 19, 2020

Opening talk
Hiroshi Yamakawa

Evolution

The world without locality
https://www.youtube.com/watch?v=6uu1MGyGjvI&feature=youtu.be

https://en.wikipedia.org/wiki/Commercial_aviation

The world without locality
https://www.youtube.com/watch?v=6uu1MGyGjvI&feature=youtu.be

https://en.wikipedia.org/wiki/Commercial_aviation

Singularity looks the end of evolutionary strategy
Decentralized autonomy

Diversification

Local competition

Advances in technology have made it easier to get more powerover a wider range.
Decentralized autonomy

Diversification

Global competition

Limitation of the evolutionary strategy– Loss of "locality" as a result of the evolutionary-driven arms race
– The (artificial) technology, a product of evolution, brings its own
global destruction (existential risk)(Yamakawa, December 12, 2019)

Global catastrophic risk
Global catastrophic risk
are the risk that have asignificant impact on

Worst casescenario,
humanextinction.

human welfare in termsof scope and severity
potentially.

https://en.wikipedia.org/wiki/Global_catastrophi
c_riskMass extinction history due to non-anthropogenic factors
PeriodRate of species that survived

70 million years ago (Cretaceous)～25%

200 million years ago (Triassic)～20%

250 million years ago (Permian)～5%

370 million years ago (Devonian)～25%

440 million years ago (Ordovician)～15%

2 billion years ago～0.5%

Occurrence frequency of about once every few hundred million years-> probably not urgent

Increasing anthropogenic risk due to emerging technologies• Fossil Fuel: Climate Change• Nuclear energy: Mass destruction, nuclear winter• Synthetic biology: lethal bacterial pandemics• Nanotechnology: Gray Goo• Artificial general intelligence: Loss of control

AI/AGI isn't the only risk posed by humans

Globalization

TechnologyLocality

Evolution
Intelligence

Evolution based on localized competition produces intelligence, and intelligence develops technology.
Globalization through technology deprives us of locality, which ends the evolution strategy.

Overcome normalcy bias
Normalcy bias:– Characteristics of people who ignore or underestimate
information that is not convenient for them• The evolutionary strategy has been successful in overcoming theenvironmental changes, but it may have just been a matter of luck.
• Against the rapid changes brought about by today's technology,
biological evolution may not be able to keep up.

"Think we'll be okay just because we were okay"
is not wise thinking.

Principle of preemption
In order to prevent irreversible Existential Risk,
society needs to take preemptive workarounds.

FireCarCan learn from failures.

Nuclearpower

Synthetic
biology

AGI

Need to deal with successfully in first time.

(Modified from Max Tegmark, 2019)

From the online discussion of lecture members• The design of the next era will be completely different dependingon what kind of society and culture will be formed in this era.
• If it goes haywire, AI-ELSI will be defeated.
• I think this is where we, the working generation, should work.
Satoshi Hase

Purpose of this seminar• Easy reassurance is not our goal.

• Overcoming the normalcy bias and accepting
the existence of risk
• Make opportunity thinking about what we cando in this age.

AgendaTimeContent :Speaker

3 min. Orientation: Kazuo Okanoya
10 min. Opening talk: Hiroshi Yamakawa
15 min. Lecture 1: Threat of AI: Hiroshi Nakagawa
30 min. Lecture 2: The end of the evolution strategy brought
about by technological advance: Hiroshi Yamakawa
30 min. Lecture 3: Emergence of Language and an End ofBiological Evolution: Kazuo Okanoya
30 min. Lecture 4: The environment is emergent. Satoshi Hase
50 min. Panel Discussion Moderator Hiroshi Nakagawa