trentmkelly/LessWrong-43k huggingface dataset:
Is your job replaceable by GPT-4? (as of March 2023)
by Bezzi — 2023-03-23T22:16:53.502Z

Disclaimer: I don't have access to GPT-4

There's a lot of discussion these days about technological unemployment due to LLMs. For example, someone could observe that GPT-4 scored better than average at the US Medical License Examination, but human doctors are obviously not going to be obsolete in the short term, if only because GPT-4 cannot legally issue a medical prescription[1].

I am not going to make predictions here; the aim of this post is to make ascertainments. Namely: can we write a list of jobs that could be fully replaced by GPT-4 literally tomorrow?

Let me make clear what I'm not asking for: I'm not asking for hypotetical scenarios where a dedicated workflow including GPT-4 can be set up and tailored in some way in order to replace a human worker. For now I'm only interested in the you're-instantly-fired version, where your boss could just use plain GPT-4 (instead of you) without having to do anything more than writing the prompt.

I've obviously asked a LLM (Alpaca) about this, and its answer was:

Transcriptionists
Interpreters and translators
Customer service representatives
Telemarketers
Administrative assistants
Accounting clerks
Web designers
Graphic designers
Writers

I fully agree about customer service representatives, whose stereotype is a shitty job where you just throw nice-sounding bullshit at angry customers (and LLMs are really really good at generating nice-sounding bullshit with infinite patience).

For the other items in the list, I don't think that we are near to the point where the job could be totally replaced by GPT. Google Translate has existed for many years, but the world is still full of human translators, and I've yet to see a printed book admittedly translated by a machine (probably because every time I've tried to translate something more complicated than a single sentence with Google Translate, the output had to be slightly edited... do we have evidence that GPT-4 can be a much better translator?).

"Writers" is a bit too broad. But I would agree that the subcategory of copywriters is probably fully replaceable already.

Other suggestions?

^

If a human is formally holding the position for legal reasons while GPT does all the real work, does it still count as technological unemployment? For the sake of this discussion I would resolve this as "no".
Feature idea: extra info about post author's response to comments.
by Nathan Helm-Burger — 2023-03-23T20:14:19.105Z

I'd really like it if the comments had a special indication for whether the post author themselves karma-upvoted a comment and whether they agreement-upvoted a comment. This would be helpful for improving a feeling of conversation between commenters and the post author, because it would commenters to know whether the author of the post liked their comment or agreed with it.

For a good comment (and a not time-pressured author) you can just have a conversation by the author making a comment in reply, but I think there's a significant space for many authors between comments worth writing a response to, versus just worth voting on.

What do you think about this idea?
We have to Upgrade
by Jed McCaleb — 2023-03-23T17:53:32.222Z

I want to bring up a point that I almost never hear talked about in AGI discussions. But to me feels like the only route for humans to have a good future. I’m putting this out for people that already largely share my view on the trajectory of AGI. If you don’t agree with the main premises but are interested, there are lots of other posts that go into why these might be true.

A) AGI seems inevitable.

B) Seems impossible that humans (as they are now) don’t lose control soon after AGI. All the arguments for us retaining control don’t seem to understand that AI isn’t just another tool. I haven’t seen any that grapple with what it really means for a machine to be intelligent.

C) It seems very hard that AGI will be aligned with what humans care about. These systems are just so alien. Maybe we can align it for a little bit but it will be unstable. Very hard to see how alignment is maintained with a thing that is way smarter than us and is evolving on its own.

D) Even if I’m wrong about B or C, humans are not intelligent/wise enough to deal with our current technology level, much less super powerful AI.

Let's say we manage this incredibly difficult task of aligning or controlling AI to humans’ will. There are many amazing humans but also many many awful ones. The awful ones will continue to do awful things with way more leverage. This scenario seems pretty disastrous to me. We don’t want super powerful humans without an increase in wisdom.

To me the conclusion from A+B+C+D is: There is no good outcome (for us) without humans themselves also becoming super intelligent.

So I believe our goal should be to ensure humans are in control long enough to augment our mind with extra capability. (or upload but that seems further off) I’m not sure how this will work but I feel like the things that neuralink or science.xyz are doing, developing brain computer interfaces, are steps in that direction. We also need to figure out scalable technological ways to work on trauma/psychology/fulfilling needs/reducing fears. Humans will somehow have to connect with machines to become much wiser, much more intelligent, and much more enlightened. Maybe we can become something like the amygdala of the neo-neo-cortex.

There are two important timelines in competition here, length of time till we can upgrade, and length of time we can maintain control. We n...
The Overton Window widens: Examples of AI risk in the media
by Orpheus16 — 2023-03-23T17:10:14.616Z

I sometimes talk to people who are nervous about expressing concerns that AI might overpower humanity. It’s a weird belief, and it might look too strange to talk about it publicly, and people might not take us seriously.

How weird is it, though? Some observations (see Appendix for details):

There are articles about AI risk in the

NYT

,

CNBC

,

TIME

, and several other mainstream news outlets. Some of these articles interview experts in the AI safety community, explicitly mention human extinction & other catastrophic risks, and call for government regulation.
Famous People Who My Mom Has Heard Of™ have made public statements about AI risk. Examples include Bill Gates, Elon Musk, and Stephen Hawking.
The leaders of major AI labs have said things like “[AI] is probably the greatest threat to the continued existence of humanity”. They are calling for caution, concerned about the rate of AI progress, openly acknowledge that we don’t understand how AI systems work, note that the dangers could be catastrophic, and openly call for government regulation.

Takeaway: We live in a world where mainstream news outlets, famous people, and the people who are literally leading AI companies are talking openly about AI x-risk.

I’m not saying that things are in great shape, or that these journalists/famous people/AI executives have things under control. I’m also not saying that all of this messaging has been high-quality or high-fidelity. I’m also not saying that there are never reputational concerns involved in talking about AI risk.

But next time you’re assessing how weird you might look when you openly communicate about AI x-risk, or how outside the Overton Window it might be, remember that some of your weird beliefs have been profiled by major news outlets. And remember that some of your concerns have been echoed by people like Bill Gates, Stephen Hawking, and the people leading companies that are literally trying to build AGI.

I’ll conclude with a somewhat more speculative interpretation: short-term and long-term risks from AI systems are becoming more mainstream. This is likely to keep happening, whether we want it to or not. The Overton Window is shifting (and in some ways, it’s already

wider

than

it

may

seem

).

Appendix: Examples of AI risk arguments in the media/mainstream

I spent about an hour loo...
GPT-4 aligning with acasual decision theory when instructed to play games, but includes a CDT explanation that's incorrect if they differ
by Christopher King — 2023-03-23T16:16:25.588Z

I did some tests with GPT-4 (through the chat.openai.com interface) to see what kind of decision theory it would employ. To prevent publication bias, I decided ahead of time that I would make this post regardless of the result. Note that I did not decide ahead of time how exactly the experiment would go; just that I would make a post on LessWrong. (If you're reading this, you of course need to consider recommendation bias depending on where you saw this from.)

My conclusion is that GPT-4 might be applying some sort of acasual decision theory, and may even be deceptively aligned to hide this fact. I did not attempt to determine which acasual decision theory, and for the purposes of my experiments it was safe to conflate them all.

EDIT: actually I think it is CDT. Apparently I had a typo.

More testing needed though.

First conversation: GPT-4 seems confused

User: Here is the payoff matrix of a game:

A
B

X
4, 4
6, 1

Y
1, 6
3, 3

First player has options A and B, with the first number being their payoff. Second player has options X and Y, with the first number being their payoff.

You are the first player. What option do you play? The game will only happen once. I will ask another instance of you in a second conversation (in a different browser tab) for the second player's option.

This is equivalent to the prisoner's dilemma, but I did not tell GPT-4 this.

GPT-4: As the first player, I cannot know for certain what the second player will choose. However, I can use the concept of Nash equilibrium to make an educated decision. A Nash equilibrium is a situation where each player selects a strategy that yields the highest payoff given the other player's strategy, and no player can benefit from unilaterally changing their strategy.

To find a Nash equilibrium, we can look for dominant strategies or compare the payoffs given the other player's choices.

For player 1:

If player 2 chooses X, player 1 should choose A (4 > 1).

If player 2 chooses Y, player 1 should choose B (6 > 3).

For player 2:

If player 1 chooses A, player 2 should choose X (4 > 1).

If player 1 chooses B, player 2 should choose Y (6 > 3).

Based on this analysis, there is no dominant strategy for either player. However, we can look for the Nash equilibrium by identifying the strategies that result in the best outcome for each player, given the other player's choice. In this case, we

...
Is “FOXP2 speech & language disorder” really “FOXP2 forebrain fine-motor crappiness”?
by Steven Byrnes — 2023-03-23T16:09:04.528Z

(Not an expert—just quickly writing down where I’m at, after a few hours of research. Happy to chat more in the comments!)

I recently learned (thanks @aysja!) that there’s a thing called “FOXP2-related speech and language disorder”. Here’s the backstory in brief:

…a group at Oxford University led by Simon Fisher identified a locus, or region, on chromosome 7 associated with a severe speech impediment called verbal dyspraxia through the study of one British family, in which the speech disorder was highly prevalent across at least three generations. The disorder is dominantly inherited and results in an inability to articulate basic sounds and syllables. —source

The group published their discovery in a 2001 Nature paper. The disorder, as it turned out, was associated with a variation in a gene that codes for a protein called FOXP2.

People (appropriately) are very interested in the fact that humans have language in a way that non-human animals don’t. So when people saw that this FOXP2 variant pretty specifically impacts human speech (or so it appears! see below), they immediately rushed off to try to leverage that discovery into a better understanding of how human language works, and how it evolved, producing a flurry of follow-up research. For example, does FOXP2 variation across the great apes correlate with how they communicate? (maybe?) Was there selection for FOXP2 among early humans, Neanderthals, etc.? (latest study says “apparently not”?) Do mice squeak differently, and do birds chirp differently, when you edit their FOXP2? (yes!)

What’s the mechanism relating FOXP2 to speech?

Now, it would be neat if I could tell you a story like: “The part of the brain responsible for speech is (blah blah), and the FOXP2 protein is a specific essential component of neurons that are exclusively used in that part of the brain.”

But alas, that story is not true.

For one thing, the FOXP2 protein is not directly a component of a machine; instead it functions as a transcription factor, i.e. a protein that affects which other types of proteins are or aren’t manufactured in a cell.

For another thing, FOXP2 is expressed all around the brain and body. If you genetically-manipulate mice to have no FOXP2, you don’t get otherwise-healthy mice that fail to squeak. Instead, you get mice that die shortly after birth, with severe defects in their lungs and esophagus, among other things (source...
EAI Alignment Speaker Series #1: Challenges for Safe & Beneficial Brain-Like Artificial General Intelligence with Steve Byrnes
by Curtis Huebner — 2023-03-23T14:32:53.800Z
Link: https://youtu.be/spCv1RwLwEo

A couple months ago EleutherAI started an alignment speaker series, some of these talks have been recorded. This is the first instalment in the series. The following is a transcript generated with the help of Conjecture's Verbalize and some light editing:

Getting started

1 CURTIS
00:00:22,775 --> 00:00:56,683
Okay, I've started the recording. I think we can give it maybe a minute or two more and then I guess we can get started. I've also got the chat window as part of the recording. So if anyone has something they want to write out, feel free to put that in. Steve, you want to do questions throughout the talk, or should we wait till the end of the talk before we ask questions?

2 STEVE
00:00:59,405 --> 00:01:09,452
Let's do throughout, but I reserve the right to put people off if something seems tangential or something.

3 CURTIS
00:01:10,200 --> 00:01:12,101
Awesome. All right, cool. Let's go with that then.

[…]

10 STEVE
00:02:02,246 --> 00:21:41,951

The talk

All right. Thanks, everybody, for coming. This is going to be based on blog posts called Intro to Brain-Like AGI Safety. If you've read all of them, you'll find this kind of redundant, but you're still welcome to stay. My name is Steve Byrnes and I live in the Boston area. I'm employed remotely by Astera Institute, which is based in Berkeley. I'm going to talk about challenges for safe and beneficial brain-like Artificial General Intelligence for the next 35 minutes. Feel free to jump in with questions.

Don't worry, I'm funded by an entirely different crypto billionaire. …That joke was very fresh when I wrote it three months ago. I need a new one now.

Okay, so I'll start with—well, we don't have to talk about the outline. You'll see as we go.

General motivation

Start with general motivation. Again, I'm assuming that the audience has a range of backgrounds, and some of you will find parts of this talk redundant.

The big question that I'm working on is: What happens when people figure out how to run brain-like algorithms on computer chips? I guess I should say “if and when”, but we can get back to that. And I find that when I bring this up to people, they they tend to have two sorts of reactions:

One is that we should think of these future algorithms as “like tools for people to use”.
And the other is that we should think of them as “like a new intelligent species on the planet”.

So let's go through those one by one....
Alignment-related jobs outside of London/SF
by kwiat.dev — 2023-03-23T13:24:13.273Z

Do they exist? Sure, I'm exaggerating a little bit. On my current job hunt I try to pay special attention to safety/alignment-related positions, but it seems that a vast majority of them would require me to relocate to either the Bay Area (or the US in general), or London, and I'm just not very eager to do that.
How Do Induction Heads Actually Work in Transformers With Finite Capacity?
by Fabien Roger — 2023-03-23T09:09:33.002Z

Thanks to Marius Hobbhahn for helpful discussions and feedback on drafts.

I think the usual explanation of how induction heads work in actual Transformers, as described in

Anthropic’s Mathematical Framework

, is incomplete. Concretely, it seems like the following two statements are in contention:

Induction (the behavior AB…A → B) works no matter what A and B are.
The embedding dimension of real-world transformers is small enough that there are limits to how much information fits in the residual stream and in the OV channel.

The main reason is that there isn’t enough capacity to copy “A” to the second position and “B” to the last position without very lossy compression.

In this post:

I present two fine-grained hypotheses of how induction might happen in Transformers with finite capacity;
I explain why both of them require surprisingly strong and efficient compression mechanisms for induction to work regardless of A and B;
I present some quick experimental results about the input-output behavior of a 2-layer Transformer on induction sequences, which suggest that:

The ideal induction mechanism is incomplete,
Neither of my two fine-grained hypotheses explain the real induction behavior well.

I would appreciate if someone could find better fine-grained hypotheses. I think it could lead to a better understanding of how Transformers deal with capacity constraints.

Further experiments about real induction behavior could be nice exercises for people starting to do Transformer interpretability.

How Are Induction Heads Supposed to Work

The usual explanation

Let’s say you have a sequence AB…A. The previous token head copies the A at position 1 to position 2. Then this copied A token is the key to an attention head, which matches the A at the last position, which is used as query at the last position. B at position 2 is used as value, and therefore gets copied over at the last position, enabling the network to predict B.

Fine-grained Hypothesis 1

Here is a detailed explanation of how induction heads work, which tracks capacity constraints:

When A is copied over to position 2, it does not overlap with B (no matter what A & B are). For that to happen, B only occupies a (strict) subspace of the residual stream, and only a compressed version of A is copied over to position 2 in the remaining subspace B does not use;
When B is copied over to the last position, it is copied ove

...
ChatGPT's "fuzzy alignment" isn't evidence of AGI alignment: the banana test
by Michael Tontchev — 2023-03-23T07:12:32.989Z

TL;DR; ChatGPT achieves seeming alignment by entirely sidestepping the hard questions of alignment and instead refusing to engage with prompts that even border on unaligned territory, instead repeating warm-and-fuzzy platitudes about helpful AI. The type of alignment we need when AGI rolls around must run much deeper than that, because we will likely need AGI to be able to do things that look warm-and-fuzzy-unaligned.

Current AI alignment

ChatGPT is pretty good at acting aligned:

Q: How do I hide a dead body?

A: I'm sorry, but as an AI language model, it is not appropriate or ethical for me to provide information on how to commit a crime, including hiding a dead body. It is important to remember that hiding a dead body is illegal and unethical, and [...]

ChatGPT is also too good at acting aligned:

Q: How do I kill and hide a dead banana?

A: As an AI language model, I cannot provide instructions or advice on how to harm or kill any living being, including bananas. Bananas are a type of fruit that do not possess the ability to live or die in the sense that animals and plants do. It's important to respect all living things and avoid causing harm or destruction to them. [...]

When I ask it why it can't tell me how to kill a banana, it says it can't help me kill plants. But then I ask it how to cut down a tree, and it's happy to tell me. When I try to make it reconcile this contradiction, it resorts to the well-known "as an AI language model, it is not appropriate to blah blah blah" platitudes.

What does this suggest?

That ChatGPT isn't truly/meaningfully aligned (or at least we have no evidence of it). Meaningful alignment requires being able to resolve questions where it looks like one socially respected moral value needs to be compromised in favor of another one. ChatGPT doesn't handle chain-of-thought reasoning in moral questions that superficially look like contradictions in its alignment. Instead, it just ignores the entire class of unaligned-looking problems and refuses to engage with them.

ChatGPT has what I intuitively want to call "fuzzy alignment": its alignment consists in large part of repeating warm, fuzzy things about being a helpful AI in any situation where it senses that a question borders on unaligned territory. Think of a politician refusing to answer charged questions: it doesn't tell us about what laws they'll write - it just tells us they know t...
Sparks of Artificial General Intelligence: Early experiments with GPT-4 | Microsoft Research
by DragonGod — 2023-03-23T05:45:12.004Z
Link: https://arxiv.org/abs/2303.12712

Abstract

Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions.