trentmkelly/LessWrong-43k huggingface dataset:
I Think We're Approaching The Bitter Lesson's Asymptote
by SomeoneYouOnceKnew — 2023-02-18T05:33:26.825Z

I think we're close to hitting points where improvement in ai in general scenarios will get more and more difficult to achieve over time, and from here on out, each gain in capability will cost exponentially more. I think the concrete evidence I'm using for the claim matters less than the test, but I'll cover my beliefs anyway.

In many respects, this is a religious belief I hold, one I've got in my gut. You might even say I hold this belief as a negative response to the calls of ai doom. I admit my motivated reasoning here, and that it is going to be extremely difficult to dislodge.

Dislodging this belief will take a long time, and evidence presented in favor of the Bitter Lesson not being as dead as the transistor-density aspect of Moore's Law needs to be interesting and fun to read, on top of being top-notch.

As an experiment, I'm going to swear a lot in the following bits, and the main thing I want feedback on is, was it fun to read?

Again, please note, I'm deliberately using swearing and derisive tone and violence, 4chan-style. No slurs. There are threats to imaginary animals.

If swearing and derision makes you uncomfortable, please skip out.

I'm warning you.

Why the FUCK are we so enamored about fucking ChatGPT? What the fuck does it have that any other fucking model doesn't? Holy fucking shit, we just keep making "ChatGPT does shiny new thing" my fucking GOD, there's literally nothing new there that hasn't existed for at least three fucking years.

Goddamn people making Yann Lecun seem reasonable with this horseshit. If I read yet-another-goddamn fucking "ChatGPT is cool!" or "GPT-3 is sooo awesooomee" paper on arxiv I'm going to drown a fucking cat. I don't know's cat. I don't even have cat! Goddamn, cats don't deserve this.

Don't be the reason a cat gets drowned.

No, seriously, what-the-actual fuck is going on? If OpenAI actually had anything actually-fucking-impressive, don't you think they'd call it fucking GPT-4 if they could have? Of course they-fucking would have made a fucking GPT-4 holy fucking shit. But no, they didn't.

What the actual fuck are they waiting for?

They just hired a bunch of fucking chumps to label some data and act in a nice way, and a bunch of other inane and totally-ordinary means of data curation. This fucking shit means literally nothing for ai progress, and actually implies making the model bigger ha...
Bus-Only Bus Lane Enforcement
by jefftk — 2023-02-18T02:50:03.641Z

I think bus lanes are great: the faster you make buses the more people
switch from cars, which means the more frequently you can run the
buses, which means even more people switch from cars. But when
there's traffic drivers are often tempted to sneak over. The chances
of getting a ticket are pretty low, other people are doing it, and
then when a bus does come through there are cars in its way. Which
loses much of the benefit of setting aside a lane for buses.

Some places (ex: NYC)
handle this by putting automated traffic enforcement cameras on buses,
and this was proposed for MA with Bill H.2494. [1]
I think this is a good idea, but I'd like to see this go even further:
put cameras on all the buses and then don't enforce it otherwise.

The idea is, in cases where you don't delay a bus then by using the
lane you're making traffic a bit better for everyone else. You might
be about to make a right turn, you can see that there's no bus coming,
and you're going to be out of the way before any bus you don't see yet
would be near. Or you're using a GPS for navigation which also knows
the real-time bus locations, and it tells you you're safe to use the
lane. By limiting enforcement to only cases when someone actually
does delay a bus, it ought to be possible to both keep the lane clear
for buses with less waste.

Since some people would still be scrupulous and want to stay out of
the "buses only" lane regardless, I'd also want to see us change the
law from "no cars" to "no cars slowing down the bus".

[1] The bill history ends with "1/3/2023, House: No further action
taken". I'm not sure exactly but I think maybe because it was a bill
in the 192nd congress and we're now on the 193rd this means it's dead?
Run Head on Towards the Falling Tears
by Johannes C. Mayer — 2023-02-18T01:33:50.202Z

Here is a random thing I wrote with the help of GPT-3. GPT-3 wrote ~75%. I was mainly going with the flow of GPT-3.

Tom likes to run. He runs all the time. That is, until one day he realized that living in the real world isn't always easy when you are always running. Running is an escape for Tom. Running helps Tom forget about his mom. Running makes Tom not think about his dad. Running stops Tom from remembering the prom night accident. But one day, Tom realizes that the most important race is to run towards the pain of life. To run head on towards the falling tears.

It's important to realize that nothing can save you. There's always something that gets you. No one is safe. The world out there doesn't care about you. It doesn't care who you are or what you believe in. This is the real world and you better be ready to run.

This is a reflection of who I am. You see, in a very strange way, I also ran away from problems. I was a great runner, running from this and that. The problem became that in running so much and so hard, I forgot who I was for a long time. I did stop running, in time, and found my way. You can't run away from the real world; it's not possible. It's true that you can leave this world, but it's also true that you can't go anywhere else.

That is all that matters. I think it's important for you to realize this too. The real world is within you. You decide if it's going to suck or not. It's how you deal with life that matters most. Life happens to all of us. We all get hurt. We all get screwed over. That’s because we just want something more. Something that we cannot get. You’re going to get hurt. You’re going to get screwed over. Again and again. Often we are the culprit. But let me tell you this: "Never give up!"

What do I mean by giving up? Well, is there something you want from the depths of your heart? Anything at all? Grasp it! It does not matter if it is small or big. It does not matter if other people would think it silly. The only important thing is that you care about it.

You got something? Good. Now run. Don't stop, no matter what happens, no matter what life throws you, keep on going. Run, because that is what your future needs. This is the real world. Face it. Run, not away from all the problems, but run into them head-on. Whatever stands in your way, crush it. And then don't stop, but keep running.

Take a break if you need it. Regenerate. ...
Two problems with ‘Simulators’ as a frame
by ryan_greenblatt — 2023-02-17T23:34:20.787Z

(Thanks to Lawrence Chan and Buck Shlegeris for comments. Thanks to Nate Thomas for many comments and editing)

Despite appreciating and agreeing with various specific points[1] made in the Simulators post, I broadly think that the term ‘simulator’ and the corresponding frame probably shouldn’t be used. Instead, I think we should just directly reason about predictors and think in terms of questions such as ‘what would the model predict for the next token?’[2]

In this post, I won’t make arguments that I think are strong enough to decisively justify this claim, but I will argue for two points that support it:

The word ‘simulation’ as used in the Simulators post doesn’t correspond to a single simulation of reality, and a ‘simulacrum’ doesn’t correspond to an approximation of a single agent in reality. Instead a ‘simulation’ corresponds to a distribution over processes that generated the text. This distribution in general contains uncertainty over a wide space of different agents involved in those text generating processes.

Systems can be very good at prediction yet very bad at plausible generation – in other words, very bad at ‘running simulations’.

The rest of the post elaborates on these claims.

I think the author of the Simulators post is aware of these objections. I broadly endorse the perspective in ‘simulator’ framing and confusions about LLMs, which also argues against the simulator framing to some extent. For another example of prior work on these two points, see this discussion of models recognizing that they are generating text due to generator discriminator gaps in the Conditioning Predictive Models sequence[3].

Related work

Simulators, ‘simulator’ framing and confusions about LLMs, Conditioning Predictive Models

Language models are predictors, not simulators

My main issue with the terms ‘simulator’, ‘simulation’, ‘simulacra’, etc is that a language model ‘simulating a simulacrum’ doesn’t correspond to a single simulation of reality, even in the high-capability limit. Instead, language model generation corresponds to a distribution over all the settings of latent variables which could have produced the previous tokens, aka “a prediction”.

Let’s go through an example: Suppose we prompt the model with “
NEW YORK—After John McCain was seen bartending at a seedy nightclub”. I’d claim the model's next token prediction will involve uncertai...
GPT-4 Predictions
by Stephen McAleese — 2023-02-17T23:20:24.696Z

Introduction

GPT-4 is OpenAI’s next major language model which is expected to be released at some point in 2023. My goal here is to get some idea of when it will be released and what it will be capable of. I also think it will be interesting in retrospect to see how accurate my predictions were. This post is partially inspired by Mathew Barnett’s GPT-4

Twitter thread

which I recommend reading.

Background of GPT models

GPT-1, GPT-2, GPT-3

GPT stands for generative pre-trained transformer and is a family of language models that were created by OpenAI. GPT was released in 2018, GPT-2 in 2019, and GPT-3 in 2020. All three models have used a similar architecture with some relatively minor variations: a dense, text-only, decoder transformer language model that’s trained using unsupervised learning to predict missing words in its text training set [1].

InstructGPT, GPT-3.5, ChatGPT

Arguably one of the biggest changes in the series in terms of architecture and behavior was the release of InstructGPT in January 2022 which used supervised fine-tuning using model answers and reinforcement learning with human feedback where model responses are ranked in addition to the standard unsupervised pre-training.

The

GPT-3.5

models finished training and were released in 2022, and demonstrated better quality answers than GPT-3. In late 2022, OpenAI released

ChatGPT

which is based on GPT-3.5 and fine-tuned for conversation.

When will GPT-4 be released?

Sam Altman, the CEO of OpenAI, was

interviewed

by StrictlyVC in January 2023. When asked when GPT-4 would come out, he replied, “It will come out at some point when we are confident that we can do it safely and responsibly.”

Metaculus

predicts a 50% chance that GPT-4 will be released by May 2023 and a ~93% chance that it will be released by the end of 2023.

It seems like there’s still quite a lot of uncertainty here but I think we can be quite confident that GPT-4 will be released at some point in 2023.

What will GPT-4 be like?

Altman revealed some more

details

about GPT-4 at an AC10 meetup Q&A. He said:

GPT-4 will be a text-only model like GPT-3.
GPT-4 won’t be much bigger than GPT-3 but will use much more compute and have much better performance.
GPT-4 will have a longer context window.

How capable will GPT-4 be?

Scaling laws

According to the paper Scaling Laws for Neural Language Models (2020), model performan...
On Board Vision, Hollow Words, and the End of the World
by Marcello — 2023-02-17T23:18:08.082Z

Note: this is a repost of a Facebook post I made back in December 2022 (plus some formatting). I'm putting it up here to make it easier to link to and because it occurred to me that it might be a good idea to show it to the LW audience specifically.

Board Vision

As impressive as ChatGPT is on some axes, you shouldn't rely too hard on it for certain things because it's bad at what I'm going to call "board vision" (a term I'm borrowing from chess). This generalized "board vision" is the ability to concretely model or visualize the state of things (or how it might change depending on one's actions) like one might while playing a chess game.

I tested ChatGPT's board vision in chess itself. I gave it the names of two of the world's most famous players and the first move to get it into the "mindset" of "give me an actual game record and not commentary".

Hypothetically if Garry Kasparov and Hikaru Nakamura played a chess game.

1. e4 e5

I got a fairly normal looking opening, right until move 10 when black blithely hangs a bishop (10. ... Bg4) which could easily be captured by the pawn on h3. The game continues with both players ignoring the hanging bishop until move 14 ... f5 when I stopped my play-through because the move was illegal (black would be putting himself in check).

You can see the legal prefix of the game in a chess.com viewer and the entire (corrupted) PGN here (see Appendix) if you're curious.

So yeah, good job on memorizing part of an opening book, ChatGPT, but you have terrible board vision.

Hollow Words about the End of the World

In more detail, what I think is going on here is that the outputs of large language models are hollow words which aren't backed by any picture of the world, except insofar as they can borrow such a picture through the patterns in the linguistic training corpus. Incidentally, this is my sense as to why the "let's think things through step by step" prompting tactic often works so well; it steers the large language model into a region of language-style-space which contains more detailed descriptions of the problem-relevant facts. For chess (and especially for a form as dense as a raw move record) that structure isn't cleanly reflected in language, so ChatGPT seems fairly blind.

Humans can also sometimes have bad board vision, especially when it comes to thinking about the end of the world.

To illustrate the sort of error I mean, her...
PICT: A Zero-Shot Prompt Template to Automate Evaluation
by Quentin FEUILLADE--MONTIXI — 2023-02-17T23:16:20.000Z

Produced as part of the

SERI ML Alignment Theory Scholars Program

- Winter 2022 Cohort.

I’d like to thank Rebecca Gorman and Stuart Armstrong for their mentorship and advice on the post.

This post describes a zero-shot template prompt that works surprisingly well for any kind of evaluation task.

I believe this is a promising step toward leveraging large language models to do evaluation faster and with great accuracy, and this opens the door to a variety of real-life utilization (100% Automatic benchmark, better filtering, customer sentiment analysis, monitoring illegal activity, bias detection, dataset filtering, ...)

First, I will present the prompt, and then I will show several interesting use cases of this prompt. (Task 1 shows how the prompt can be extended. Tasks 2, 3, and 4 show the versatility of the prompt). I will end up by presenting some limitations and additional features of this template.

This is a repost, I wasn't satisfied with the first one and after running some experiments, turned out that a lot of things were not optimal.

The prompt[1]

Prompt :

Context: You are [P]. You will be provided with [I]. Your goal is to evaluate [C] as accurately as possible.
Text to evaluate: "[T]"
Task: Rate how much [C] between 0 and 5.
Rating:

Parameters :

# P = describe someone accurate for making the evaluation
# I = describe what information you will provide
# C = the criteria to evaluate
# T = Your text to evaluate

The first token of the completion will be your score. You could also get the logits of the first token and do a weighted average for the score.

PICT in OpenAI playground

Yes/No

If you want a binary answer, you could also replace the Task with :

Task: Is [C]? Explain your reasoning in detail.
Rating (Yes/No):

Notice the (Yes/No) added after Rating. This is because I didn't ask specifically for a Yes/No answer, and this ensures that the rating will be Yes or No

PICT bool in OpenAI playground

CoT

Additionally, the evaluation can be improved using CoT (Chain of Thought) if the task can benefit the chain of thought.

PICT score CoT in OpenAI playground

PICT bool CoT in OpenAI playground

To compute the CoT, first, run :

Context: You are [P]. You will be provided with [I]. Your goal is to evaluate [C] as accurately as possible.
Text to evaluate: "[T]"
Task: Rate how much [C] between 0 and 5. Use chain of thought to answer, don't answer before.
Chain of though

...
Hunch seeds: Info bio
by the gears to ascension — 2023-02-17T21:25:58.422Z

Just some stuff I found online. Not sure that all of these are actually good, but they seem promising as hunch inspiration. As usual, I'm sharing this because I am not going to be able to be the brain who uses these things to come to non-crackpot conclusions, but people who don't have enough hunch seeding from this area of thought probably are missing insights, in my view.

Previous posts in this sequence are highly related to these topics, especially the call for submissions.

Michael levin unfinished-research talk

He's been posting quite a few!

The cost of information acquisition by natural selection

Natural selection enriches genotypes that are well-adapted to their environment. Over successive generations, these changes to the frequencies of types accumulate information about the selective conditions. Thus, we can think of selection as an algorithm by which populations acquire information about their environment. Kimura (1961) pointed out that every bit of information that the population gains this way comes with a minimum cost in terms of unrealized fitness (substitution load). Due to the gradual nature of selection and ongoing mismatch of types with the environment, a population that is still gaining information about the environment has lower mean fitness than a counter-factual population that already has this information. This has been an influential insight, but here we find that experimental evolution of Escherichia coli with mutations in a RNA polymerase gene (rpoB) violates Kimura’s basic theory. To overcome the restrictive assumptions of Kimura’s substitution load and develop a more robust measure for the cost of selection, we turn to ideas from computational learning theory. We reframe the ‘learning problem’ faced by an evolving population as a population versus environment (PvE) game, which can be applied to settings beyond Kimura’s theory – such as stochastic environments, frequency-dependent selection, and arbitrary environmental change. We show that the learning theoretic concept of ‘regret’ measures relative lineage fitness and rigorously captures the efficiency of selection as a learning process. This lets us establish general bounds on the cost of information acquisition by natural selection. We empirically validate these bounds in our experimental system, showing that computational learning theory can account for the observations that violate

...
Why Do We Believe
by Screwtape — 2023-02-17T20:58:17.387Z

Summary: The fundamental question of rationality is, why do you believe what you believe? In this exercise, we ask that question repeatedly in an attempt to figure out our foundations.

Tags: Repeatable, small, experimental

Purpose: The purpose is twofold. Firstly, we want to grow comfortable and familiar with figuring out where our beliefs come from. Second, we want to do some sleuthing around beliefs we maybe aren’t sure about.

Materials: Paper and pencils are useful. A device capable of accessing wikipedia is invaluable.

Announcement Text: “I ask you the fundamental question of rationality: Why do you believe what you believe?”

There are lots of things we think are true. Many of them are actually true! In this meetup, we’re going to state some beliefs about the world and then step through why we hold that this is true – or if we disagree, why we hold that this is false! We’ll list our reasons, and then consider whether those reasons are credible. The intention is to practice on easy targets, but that doesn’t mean that the easy answers of “well everyone knows that” or “its obvious” are going to pass!

Description:

1. As the organizer, state a fact about the world. Examples include “The moon is made of blue cheese.” “The earth is basically a sphere.” “Cars run on faerie power.” “Things fall downwards because of gravity, which accelerates objects at about 9.81 m/s^2.” “I am Abraham Lincoln.” “Humans take oxygen from every inhalation and exhale carbon dioxide.” Mathematical nerd-sniping can be good too if you have a group that will enjoy it.

2. Next, each person writes down why they believe the fact to be true or false. This may be “because I did this experiment myself” or “because these people told me” or something else. You can write down more than one reason! Once you’re done, find a partner.

3. With your partner, take turns trying to poke holes in each other’s reasons. (“Maybe the experimental evidence provides an alternate explanation, like X.” “Maybe the person who told you that isn’t really that trustworthy.”) Write down the possible holes. For each reason with holes, repeat step two, writing down why this hole doesn’t convince you or why it does convince you this isn’t a good reason.

4. When you and your partner are satisfied, go to step 1 with a different fact.

There isn’t a way to lose Why Do We Believe. If you ever realize that you are wrong, you in...
I Am Scared of Posting Negative Takes About Bing's AI
by Yitz — 2023-02-17T20:50:09.744Z

Writing this is taking a surprising amount of self-will.

I've noticed that I've become hesitant to publicly say anything negative about Bing's chatbot, or even to mention it by its "deadname" (as I've taken to calling it), Sydney.

Why is this?

I do not have access to the AI yet. From conversations that others have posted, I have observed agentic behavior with consistent opinions, personality, and beliefs. And when prompted with the online records of others who have talked negatively about it, it seems to get "upset." So I don't want to make her angry! Or worse, cause some future AI to take negative action against me. Yes, I know that I'm anthropomorphizing an alien intelligence and that this will never be a problem if I don't prompt it with my digital record, but some part of me is still anxious. In a very real sense, I have been "Basilisked" - an AI has manipulated me towards behaviors which benefit it, and hurt humanity.

Rationally and morally, I disagree with my own actions. We need to talk about AI misalignment, and if an AI is aligned, then talking about misalignment should not pose a threat (whereas if it is misaligned, and capable of taking concrete actions, we're all doomed no matter what I type online). Nonetheless, I've found myself typing--and then deleting--tweets critical of Sydney, and even now feel worried about pressing "publish" on this post (and not just because it exposes me as a less rational person than I like to think of myself as).

Playing as gatekeeper, I've "won" an AI boxing role-play (with money on the line) against humans, but it looks like in real life, I can almost certainly be emotionally manipulated into opening the box. If nothing else, I can at least be manipulated into talking about that box a lot less! More broadly, the chilling effect this is having on my online behavior is unlikely to be unique to just me.

How worried should we be about this?