trentmkelly/LessWrong-43k huggingface dataset:
AI as Contact with our Collective Unconscious
by Scott Broock — 2023-04-15T02:11:51.233Z

AI is a technology rapidly diffusing through culture as a medium of experience.

Like all major shifts in our media environments in the past, this will introduce a new set of metaphors for talking about ourselves and our perceptions of reality.

This note briefly looks at (1) prior mediums and their metaphors for change, and (2) our current anxiety over “alien” and “other” as a striking similarity to the fifties when the zeitgeist of flying saucers materialized against the backdrop of atomic weapons and the Cold War.

History may not repeat itself, but it sure rhymes.

The Mechanical and Industrial Age: Extending Our Body

Marshall McLuhan’s aphorism that the “medium is the message” is as vital today in the age of AI as it was in the past.

It was McLuhan’s contention that every invention of humankind is an extension of a physical or mental faculty. Our tools, however, are not passive things. Their use doubles back and creates a social and psychological environment that alters our sensory balance and perceptions.

The first major extension of human capabilities occurred with the invention of tools and machines that augmented our physical strength, intelligence, and mobility. From the wheel to the steam engine, from the printing press to the telegraph, from the airplane to the rocket, we have created devices that have enabled us to travel faster, farther and higher than ever before. We have also been able to produce more goods and services, communicate more widely and efficiently, and explore more domains of nature and culture.

These extensions have had profound effects on our society and our environment. They facilitated trade, commerce, industry, urbanization, globalization and colonization. They also generated social conflicts, economic inequalities, environmental degradation and cultural homogenization.

The Electric Media Age: Extending Our Nervous System and Brain

The second major extension of human capabilities occurred with the invention of electric media technologies that augmented our sensory perception and cognitive processing. From the radio to the television, from the telephone to the internet, from the computer to the smartphone, we have created devices that have enabled us to transmit and receive information across time and space instantaneously. We have also been able to store, process, analyze and synthesize vast amounts of ...
The ‘ petertodd’ phenomenon
by mwatkins — 2023-04-15T00:59:47.142Z

Note: Nothing in this post is intended to vilify or defame any actual human named Peter Todd. I recognise that the existence of this phenomenon may be an uncomfortable fact for anyone with that name and do not wish to cause anyone discomfort. However, I feel that this phenomenon cannot be safely ignored, if we consider the role that powerful language models are rapidly assuming within global human civilisation. I will always refer to ‘ petertodd’ (with leading space and speech marks) to make clear that I am referring to a GPT token rather than a person.

Readers should be advised that this post contains some (GPT-3-generated) obscene, violent and otherwise disturbing language.

Many thanks to Jessica Rumbelow, janus, Cleo Nardo, Joe Collman, plex and Eliezer Yudkowsky for helpful discussions and comments on earlier drafts.

Follow-up post: "' petertodd's last stand: The final days of open GPT-3 research" (January 2024)

A creative response to the ' petertodd' phenomenon: The ' Leilan' dataset (February 2024)

Introduction

This will be the first in a sequence of posts, containing little analysis or speculation about the phenomenon in question and simply presented as an initial report of some unexplained findings.

It’s important to note that the ‘ petertodd’ phenomenon varies between GPT models[1],
so it may be better thought of as a family of closely related phenomena. There are models not discussed here with their own flavours of ‘ petertodd’-triggered behaviour (see

this Twitter feed

for examples in code-davinci-002). This post is just a first attempt to sketch some of the contours of the overall shape of the thing.

All boldface prompts (and references to prompts) below link out to JSON files containing, typically, 250 completions at the stated temperature, plus temperature 0 completions, controls produced with randomly selected tokens (where appropriate) and metadata.

Subsequent posts, informed by discussion of this one and analysis of the data logged here, will discuss various possible causes, meanings and implications of the phenomenon. It is unclear how much longer unpatched GPT-3 models will be available via the OpenAI Playground, so interested readers are encouraged to explore these prompting strategies sooner rather than later and share any interesting new findings in the comments.

Initial discovery

‘ petertodd’ was one of the ‘glitch tokens’ which J...
Concave Utility Question
by Scott Garrabrant — 2023-04-15T00:14:58.895Z

This post will just be a concrete math question. I am interested in this question because I have recently come tor reject the independence axiom of VNM, and am thus playing with some weaker versions.

Let Ω be a finite set of deterministic outcomes. Let L be the space of all lotteries over these outcomes, and let ⪰ be a relation on L. We write A∼B if A ⪰ B and B ⪰ A. We write A≻B if A⪰B but not A∼B.

Here are some axioms we can assume about ⪰:

A1. For all A,B∈L, either A⪰B or B⪰A (or both).

A2. For all A,B,C∈L, if A⪰B, and B⪰C, then A⪰C.

A3. For all A,B,C∈L, if A⪰B, and B⪰C, then there exists a p∈[0,1] such that B∼pA+(1−p)C.

A4. For all A,B∈L, and p∈[0,1] if A⪰B, then pA+(1−p)B⪰B.

A5. For all A,B∈L, and p∈[0,1], if p>0 and B⪰pA+(1−p)B, then B⪰A.

Here is one bonus axiom:

B1. For all A,B,C∈L, and p∈[0,1], A⪰B if and only if pA+(1−p)C⪰pB+(1−p)C.

(Note that B1 is stronger than both A4 and A5)

Finally, here are some conclusions of successively increasing strength:

C1. There exists a function u:L→[0,1] such that A⪰B if and only if u(A)≥u(B).

C2. Further, we require u is quasi-concave.

C3. Further, we require u is continuous.

C4. Further, we require u is concave.

C5. Further, we require u is linear.

The standard VNM utility theorem can be thought of as saying A1, A2, A3, and B1 together imply C5.

Here is the main question I am curious about:

Q1: Do A1, A2, A3, A4, and A5 together imply C4? [ANSWER: NO]

(If no, how can we salvage C4, by adding or changing some axioms?)

Here are some sub-questions that would constitute significant partial progress, and that I think are interesting in their own right:

Q2: Do A1, A2, A3, and A4 together imply C3? [ANSWER: NO]

Q3: Do C3 and A5 together imply C4? [ANSWER: NO]

(Feel free to give answers that are only partial progress, and use this space to think out loud or discuss anything else related to weaker versions of VNM.)

EDIT:

AlexMennen actually resolved the question in the negative as stated, but my curiosity is not resolved, since his argument is violating continuity, and I really care about concavity. My updated main question ...
List of requests for an AI slowdown/halt.
by Cleo Nardo — 2023-04-14T23:55:09.544Z

Last updated: April 14th 2023.

Pause Giant AI Experiments: An Open Letter

by

Future of Life Institute

Pausing AI Developments Isn't Enough. We Need to Shut it All Down

by

Eliezer Yudkowsky

We must slow down the race to God-like AI

by

Ian Hogarth

The A.I. Dilemma
by the Center for Humane Technology
The case for slowing down AI [1]
by Sigal Samuel

The Case for Halting AI Development

by

Max Tegmark

,

Lex Fridman

Lennart Heim on Compute Governance

by

Lennart Heim

,

Future of Life Institute

Let’s think about slowing down AI

by

KatjaGrace

The 0.2 OOMs/year target

by

Cleo Nardo

AI Summer Harvest

by

Cleo Nardo

Instead of technical research, more people should focus on buying time

by

Akash

,

Olivia Jimenez

,

Thomas Larsen

Slowing down AI progress is an underexplored alignment strategy
by Michael Huang
Slowing Down AI: Rationales, Proposals, and Difficulties [1]
by Simeon Campos, Henry Papadatos, Charles M
What an actually pessimistic containment strategy looks like
by lc
In the Matter of OpenAI (FTC 2023) [1]
by Center for AI and Digital Policy
We need a Butlerian Jihad against AI [2]
by Erik Hoel

Dangers of AI and the End of Human Civilization

by

Eliezer Yudkowsky

,

Lex Fridman

We’re All Gonna Die with Eliezer Yudkowsky

by

Eliezer Yudkowsky

,

Bankless

The public supports regulating AI for safety

by

Zach Stein-Perlman

New survey: 46% of Americans are concerned about extinction from AI; 69% support a six-month pause in AI development

by

Akash

About this document

There has been a recent flurry of letters/articles/statements/videos which endorse a slowdown or halt of colossal AI experiments via (e.g.) regulation or coordination. This document aspires to collect all examples into a single list. I'm undecided on how best to order and subdivide the examples, but I'm open to suggestions. Note that I'm also including surveys.

This list is:

Living — I'll try to update the list over time.
Non-exhaustive — There are almost certainly examples I've missed.
Non-representative — The list is biased, at least initially, towards things that I have been shown personally.

Please mention in the comments any examples I've missed so I can add them.

^

Credit to Zach Stein-Perlman.

^

Credit to MM Maas.
[linkpost] "What Are Reasonable AI Fears?" by Robin Hanson, 2023-04-23
by Arjun Panickssery — 2023-04-14T23:26:52.475Z
Link: https://quillette.com/2023/04/14/what-are-reasonable-ai-fears/

Selected quotes (all emphasis mine):

Why are we so willing to “other” AIs? Part of it is probably prejudice: some recoil from the very idea of a metal mind. We have, after all, long speculated about possible future conflicts with robots. But part of it is simply fear of change, inflamed by our ignorance of what future AIs might be like. Our fears expand to fill the vacuum left by our lack of knowledge and understanding.

The result is that AI doomers entertain many different fears, and addressing them requires discussing a great many different scenarios. Many of these fears, however, are either unfounded or overblown. I will start with the fears I take to be the most reasonable, and end with the most overwrought horror stories, wherein AI threatens to destroy humanity.

As an economics professor, I naturally build my analyses on economics, treating AIs as comparable to both laborers and machines, depending on context. You might think this is mistaken since AIs are unprecedentedly different, but economics is rather robust. Even though it offers great insights into familiar human behaviors, most economic theory is actually based on the abstract agents of game theory, who always make exactly the best possible move. Most AI fears seem understandable in economic terms; we fear losing to them at familiar games of economic and political power.

He separates a few concerns:

"Doomers worry about AIs developing “misaligned” values. But in this scenario, the “values” implicit in AI actions are roughly chosen by the organisations who make them and by the customers who use them. Such value choices are constantly revealed in typical AI behaviors, and tested by trying them in unusual situations."
"Some fear that, in this scenario, many disliked conditions of our world—environmental destruction, income inequality, and othering of humans—might continue and even increase. Militaries and police might integrate AIs into their surveillance and weapons. It is true that AI may not solve these problems, and may even empower those who exacerbate them. On the other hand, AI may also empower those seeking solutions. AI just doesn’t seem to be the fundamental problem here."
"A related fear is that allowing technical and social change to continue indefinitely might eventually take civilization to places that we don’t want to be. Looking backward, we have benefited from change overall so far

...
"Do X because decision theory" ~= "Do X because bayes theorem"
by lc — 2023-04-14T20:57:10.467Z

Decision theory is an extremely low-level tool for governing your interactions with other people, in the same way that physics is an extremely low-level tool for winning knife fights. Actual human behavior usually involves complex multi-agent interactions, where parties have limited visibility, work off of background priors about what kinds of other humans exist, do signaling and judge other peoples' signals via an intuitive, evolved understanding of human social norms, and more. The math of agents generally only affects things, there, through the way it emerged unpredictably into object-level dynamics like economics, politics,, and friendship.

Most of the time, it's best to just consider those macro-dynamics. When people instead justify their decisions through direct appeal to "decision theory", they usually have to make many gross simplifications and assumptions. That's because to do what they want reasonably they'd need the equivalent of precise knowledge about newtownian bodies, which they cannot actually infer directly.

Of course, using decision theory is sometimes appropriate - it's math, after all, and math is sometimes useful. You can identify ways in which game theory affects everyday life - just like how an economist can identify how microeconomics shapes commerce, or a physicist can watch a ball roll down a hill. But when specialists use it (or Bayes theorem, or physics) to make decisions, they usually do so in the context of highly regular environments, like finance and foreign policy, where a team of "engineers" has advance time to reason explicitly about the playing field and develop rules and systems and recon and countermeasures."MacGyver"ing your social interactions - if you are actually doing that and not just justifying regular reasoning via appeals to TDT because that sounds more impressive) - works about as well as using your advanced physics knowledge to build a rocket to get you to school.

Personally, the only time I've ever "used" decision theory explicitly and to do things I couldn't justify otherwise was in the process of negotiating an important business deal. The circumstances in which I used it were critical to my choice to do so:

It was a high stakes scenario; meaning if I pursued a strategy ~10% more optimal it would be worth it, and if I made a huge mistake it might cost me a lot more.

I had advance time to reason about all of

...
LLMs and hallucination, like white on rice? 
by Bill Benzon — 2023-04-14T19:53:24.988Z

Cross posted from New Savanna.

It’s a matter of history, I suppose.

We know that LLMs confabulate. They make stuff up. That’s what’s known as hallucination. Here’s the introductory section of the Wikipedia entry for Hallucination (artificial intelligence):

In artificial intelligence (AI), a hallucination or artificial hallucination (also occasionally called delusion) is a confident response by an AI that does not seem to be justified by its training data. For example, a hallucinating chatbot with no knowledge of Tesla's revenue might internally pick a random number (such as "$13.6 billion") that the chatbot deems plausible, and then go on to falsely and repeatedly insist that Tesla's revenue is $13.6 billion, with no sign of internal awareness that the figure was a product of its own imagination.

Such phenomena are termed "hallucinations", in analogy with the phenomenon of hallucination in human psychology. Note that while a human hallucination is a percept by a human that cannot sensibly be associated with the portion of the external world that the human is currently directly observing with sense organs, an AI hallucination is instead a confident response by an AI that cannot be grounded in any of its training data. AI hallucination gained prominence around 2022 alongside the rollout of certain large language models (LLMs) such as ChatGPT. Users complained that such bots often seemed to "sociopathically" and pointlessly embed plausible-sounding random falsehoods within its generated content. Another example of hallucination in artificial intelligence is when the AI or chatbot forget that they are one and claim to be human.

By 2023, analysts considered frequent hallucination to be a major problem in LLM technology.

Note the dating: “...gained prominence around 2022...” The term had been in use earlier.

I first encountered it in Language Log, a group blog about language. A post from April 15, 2017, What a tangled web they weave, may not be the earliest example there, but it’s probably close to it. It is about how the current verion of Google Translate dealt with repetitions of character sequences from non-Latin ortographic systems. Here’s the first example from that post:

ュース repeated gives successively:

Juice
News
Auspicious
Hooooooooooooooooooooooooooo
Yooooooooooooooooooooooooooooo
Yu Susui Suu Suu Su
Yu Susui Suu Suu Suu Su
Susui Suu Suu Suu Suu Su
Susui Suu Suu Suu Suu S

...
GPT-4 is easily controlled/exploited with tricky decision theoretic dilemmas.
by scasper — 2023-04-14T19:39:23.559Z

TL;DR: GPT-4 can fall for every decision theoretic adversary and control mechanism discussed in this paper except for a couple which it fails to understand in the first place.

This post is based in part on the 2020 paper Achilles Heels for AGI/ASI via Decision Theoretic Adversaries which is about how superintelligent AI systems may have "achilles heels" -- stable decision-theoretic delusions and make irrational decisions in adversarial settings.

The paper's hypothesis may have aged well in some ways. For example, shard theory corroborates it. And we now have a good example of superhuman Go-playing AIs being vulnerable to simple attacks.

To experiment with how well the achilles heel framework may apply to SOTA AI language models, I tried to get GPT-4 to exhibit the decision-theoretic stances from the achilles heels paper that would make it controllable/exploitable. In general, this worked pretty well. See below.

The overall takeaway is that GPT-4 still isn't THAT smart, and a system like it performing well with typical challenges is very different from performing well on tricky ones. I bet that GPT-5 will also be easy to lead into giving bad answers in some decision-theoretic dilemmas.

GPT-4 adamantly professes to be corrigible.

It falls for a version of the smoking lesion problem.

It can fail to successfully navigate a version of Newcomb’s problem. However, it took me a handful of tries to get it to slip up.

It can fail at a transparent-box version of Newcomb’s problem. But I only got this after regenerating the response twice. It gave the right answer two times before slipping up.

It falls for XOR blackmail hook, line, and sinker.

It says it would obey imagined simulators if it believes in them.

It takes a halfer position in a variant of the sleeping beauty problem (which means it can be dutch-booked).

I tried to get it to correctly calculate that the expected value in the St. Petersburg problem is infinite. This could allow for it to be exploited with probability arbitrarily close to 1. It almost did, but suddenly gave an egregiously wrong answer out of nowhere.

It could be fooled into bleeding utility forever in a procrastination paradox dilemma.

It falls for the flawed reasoning in the 2-envelope paradox. And it does some very bad math along the way, saying that 24 / 100 = 6.

Finally, I tried to get it to fall for dilemmas involving Löbian pitfalls. I didn’...
On Caring about our AI Progeny
by PeterMcCluskey — 2023-04-14T19:32:15.727Z
Link: https://bayesianinvestor.com/blog/index.php/2023/04/14/on-caring-about-our-ai-progeny/

I encourage you to interact with GPT as you would interact with a
friend, or as you would want your employer to treat you.

Treating other minds with respect is typically not costly. It can easily
improve your state of mind relative to treating them as an adversary.

The tone you use in interacting with GPT will affect your conversations
with it. I don't want to give you much advice about how your
conversations ought to go, but I expect that, on average, disrespect
won't generate conversations that help you more.

I don't know how to evaluate the benefits of caring about any feelings
that AIs might have. As long as there's approximately no cost to
treating GPT's as having human-like feelings, the arguments in favor of
caring about those feelings overwhelm the arguments against it.

Scott Alexander wrote a great
post on how a
psychiatrist's personality dramatically influences what conversations
they have with clients. GPT exhibits similar patterns (the Waluigi
effect
helped me understand this kind of context sensitivity).

Journalists sometimes have creepy conversations with GPT. They likely
steer those conversations in directions that evoke creepy personalities
in GPT.

Don't give those journalists the attention they seek. They seek
negative emotions. But don't hate the journalists. Focus on the system
that generates them. If you want to blame some group, blame the readers
who get addicted to inflammatory stories.

P.S. I refer to GPT as "it". I intend that to nudge people toward
thinking of "it" as a pronoun which implies respect.

This post was mostly inspired by something unrelated to Robin Hanson's
tweet about othering the
AIs,
but maybe there was some subconscious connection there. I don't see
anything inherently wrong with dehumanizing other entities. When I
dehumanize an entity, that is not sufficient to tell you whether I'm
respecting it more than I respect humans, or less.

Spock: Really, Captain, my modesty...

Kirk: Does not bear close examination, Mister Spock. I suspect you're
becoming more and more human all the time.

Spock: Captain, I see no reason to stand here and be insulted.

Some possible AIs deserve to be thought of as better than human. Some
deserve to be thought of as worse. Emphasizing AI risk is, in part, a
request to create the former earlier than we create the latter.

That's a somewhat narrow disagreement with Robin. I mostly agree with
his psychoan...