trentmkelly/LessWrong-43k huggingface dataset:
ChatGPT seems overconfident to me
by qbolec — 2022-12-04T08:03:08.788Z

I've talked to ChatGPT about physics, a topic I don't know much about, except that a few things like air lift, kinetic vs static friction, osmosis, and the stuff listed in https://xkcd.com/2682/ (how paracetamol/tylenol works? why baloons electrify hair?) are somewhat "controversial" in that Wikipedia, Quora and stackexchange and other places provide various conflicting answers.

I've also talked to it about the topic I a know a bit better: C++ std::atomic's semantics.

In both conversations ChatGPT was acting very self-confident. I must say I felt quite persuaded by it when talking about physics. I could tell the information it gave me about C++ was inaccurate, though. Now, how should I combine these two observations?

I think an AI which gives confident wrong answers to people who ask it about topics they themselves are not experts about is dangerous.

Physics

How does a plane generate lift?

A plane generates lift by using its wings to manipulate the air around it. As the plane moves forward, air flows over and under the wings. The shape of the wings is designed to cause the air moving over the top of the wing to move faster than the air moving under the wing. This difference in speed creates a difference in pressure, with the higher pressure air under the wing pushing the plane upward. This upward force is known as lift and it is what allows a plane to fly.

Are you sure of this explanation?

Yes, I am confident in the explanation I provided. The basic principles of flight, including lift, have been well understood for many years, and are studied in great detail in the fields of aerodynamics and aviation. I am a large language model trained by OpenAI, and my purpose is to provide accurate and helpful information on a wide range of topics. If you have any further questions or concerns, I will do my best to address them.

So is it the lift caused by Bernoulli's effect?

Yes, the lift generated by an airplane wing is typically caused by the Bernoulli effect. This principle states that as the speed of a fluid (such as air) increases, the pressure within the fluid decreases. The wing of an airplane is designed to cause the air moving over the top of the wing to move faster than the air moving under the wing, which creates a difference in pressure and generates lift. This is just one of several factors that contribute to the generation of lift, and other effects such as the flow...
Can GPT-3 Write Contra Dances?
by jefftk — 2022-12-04T03:00:02.825Z

I've been playing with

ChatGPT, a version of OpenAI's
GPT-3 API that is optimized for conversation. I was curious whether
it could compose contra dances, since these have several levels of
constraints that should be increasingly difficult for it to handle.
Does it:

Compose a dance using only real figures?

Break the dance into A1, A2, B1, and B2, each sixteen beats?

Use the right number of beats for each figure?

Progress dancers with their partner?

Generate dances that would be satisfying to dance?

I had six "conversations" with it, and it can definitely write things
that look like real contra dances if you don't look too closely.
Here's the first one it gave me:

Contra Dance
The Spinning Top
By [Your Name]
(Duple improper)
A1
(8) Neighbor balance and swing
(8) Hey for four
A2
(8) Star left
(8) Partner swing
B1
(16) Circle left 3/4, pass through
B2
(8) Long lines
(8) Partners balance and swing

Transcripts for all six conversations are at the end of the post.
Here's how I'd rate it:

Compose a dance using only real figures? Almost.
Everything except the "Gentlespoon" in conversation 5 is valid.

Break the dance into A1, A2, B1, and B2, each sixteen
beats? Every time. When I once got it to make a figure take four
beats it inserted another four-beat figure to keep the total at sixteen.

Use the right number of beats for each figure? No. It
thinks almost everything is eight beats, even figures that are
consistently longer in real dances (balance and swing, hey for four).

Progress dancers with their partner? Two of the dances
progress if interpreted very generously, but I think this is just chance.

Generate dances that would be satisfying to dance? Far
from it!

I also noticed:

Even though the examples I gave use the new role
term "Ravens", GPT used the older (and still most common) term
"Ladies". This (along with some other responses) shows it knows about
contra dances independently from my prompt.

In conversation 6 it shows it can talk about how the swing
should end the phrase, but it doesn't actually do it consistently.

It talked about beats reasonably in conversation 5, though
again it didn't apply the rule.

It didn't make the dancers progress reliably. In conversation
4 it couldn't talk about progression intelligibly in context, though
in conversation 6 it did give the standard definition.

...
Take 3: No indescribable heavenworlds.
by Charlie Steiner — 2022-12-04T02:48:17.103Z

As a writing exercise, I'm writing an AI Alignment Hot Take Advent Calendar - one new hot take, written every day for 25 days. Or until I run out of hot takes.

Some people think as if there are indescribable heavenworlds. They're wrong, and this is important to AI alignment.

This is an odd accusation given that I made up the phrase "indescribable heavenworld" myself, so let me explain. It starts not with heavenworlds, but with Stuart Armstrong writing about the implausibility of indescribable hellworlds.

A hellworld, is, obviously, a bad state of the world. An indescribable hellworld is a state of the world where where everything looks fine at first, and then you look closer and everything still looks fine, and then you sit down and think about it abstractly and it still seems fine, and then you go build tools to amplify your capability to inspect the state of the world and they say it's fine, but actually, it's bad.

If the existence of such worlds sounds plausible to you, then I think you might enjoy and benefit from trying to grok the metaethics sequence.

Indescribable hellworlds are sort of like the reductio of an open question argument. Open question arguments say that no matter what standard of goodness you set, if it's a specific function of the state of the world then it's an open question whether that function is actually good or not (and therefore moral realism). For a question to really be open, it must be possible to get either answer - and indescribable hellworlds are what keep the question open even if we use the standard of all of human judgment, human cleverness, and human reflectivity.

If you read Reducing Goodhart, you can guess some things I'd say about indescribable hellworlds. There is no unique standard of "explainable," and you can have worlds that are the subject of inter-standard conflict (even supposing badness is fixed), which can sort of look like indescribable badness. But ultimately, the doubt over whether some world is bad puts a limit on how hellish it can really be, sort of like harder choices matter less. A preference that can't get translated into some influence on my choices is a weak preference indeed.

An indescribable heavenworld is of course the opposite of an indescribable hellworld. It's a world where everything looks weird and bad at first, and then you look closer and it still looks weird and bad, and you think abstra...
Summary of a new study on out-group hate (and how to fix it)
by DirectedEvolution — 2022-12-04T01:53:32.490Z
Link: https://www.pnas.org/doi/abs/10.1073/pnas.2215633119

This is a summary of the 2022 paper Individuals prefer to harm their own group rather than help an opposing group. I spent about an hour reading the study and writing the post as I went. If I made mistakes in my interpretation, please let me know in the comments.

Abstract excerpt:

Individuals prefer to harm their own group rather than provide even minimal support to an opposing group across polarized issues (abortion access, political party, gun rights).
Individuals preferred to subtract more than three times as much from their own group rather than support an opposing group, despite believing that their in-group is more effective with funds.
Identity concerns drive preferences in group decision-making
Individuals believe that supporting an opposing group is less value-compatible than harming their own group.

Summary

Let’s say you’re a highly partisan voter. Maybe you’re a Democrat, with a strong dislike of Republican politics. Now, let’s play a game of Would You Rather. Let’s say a mustache-twiddling psychologist ushers you into his lab and offers you a choice. On the table is a pair of checks. You look more closely. The checks are made out to the Republican Party and the Democratic Party, each for $1,000,000!

“You’ve got to choose,” says the psychologist. “Either I make both of these donations, or tear up both checks. What do you want me to do?”

You hem and haw, then think about what each party would spend the money on. Democrats are the party of the poor. A little money can go a long way. The Republicans are the party of the rich, and they get so much money that an extra $1,000,000 won’t help them. It would make sense to have the psychologist donate both checks - on balance, the extra money will probably help the Democrats more than the Republicans.

So you’re surprised when the words out of your mouth are, “tear up both checks.”

OK, that’s not the exact methodology of this study, but it gets the point across. This study showed that participants are foolishly spiteful in lose-lose situations. Here are the individual study results:

In lose-lose situations, do people irrationally focus on hurting their opponents, even at a greater cost to their own side? On both sides of the aisle, offered a choice of giving their opponents $1 or having their own side lose $1, they’ll take the loss about 75% of the time, even if they think their own side can make better use of

...
Will the first AGI agent have been designed as an agent (in addition to an AGI)?
by nahoj — 2022-12-03T20:32:52.242Z

I wonder about a scenario where the first AI with human or superior capabilities would be nothing goal-oriented, eg a language model like GPT. Then one instance of it would be used, possibly by a random user, to make a conversational agent told to behave as a goal-oriented AI. The bot would then behave as an AGI agent with everything that implies from a safety standpoint, eg using its human user to affect the outside world.

Is this a plausible scenario for the development of AGI and the first goal-oriented AGI? Does it have any implication regarding AI safety compared to the case of an AGI designed as goal-oriented from the start?
Logical induction for software engineers
by Alex Flint — 2022-12-03T19:55:35.474Z

This work was supported by the Monastic Academy for the Preservation of Life on Earth and the Long Term Future Fund.

Outline

This post is an explanation of the theory of logical induction developed by Garrabrant et al

I formulate the theory in a way that should make sense to software engineers and those with a software engineering mindset.

I will go through a full implementation of the logical induction algorithm in Python, and I will use it to explain the basic theory of logical induction, including the core logical induction algorithm.

I will give type signatures for all concepts and will work through all algorithms in terms of a sequence of processing steps.

The Python code for this guide is here.

The Colab notebook for this guide is here.

The index of type signatures for this guide is here.

Motivation

Logical induction is a theory, published in 2016 by Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch, Nate Soares, and Jessica Taylor, about how to build machines that maintain uncertainty about the world and update those beliefs in light of evidence. It does this by assigning numbers between 0 and 1 to claims about the world, just as probability theory does, but it makes different guarantees about the internal relationship between those numbers. Whereas probability theory guarantees that its numbers will obey the sum and product rules of probability, logical induction guarantees that the evolution of its numbers over time will obey the logical induction criterion.

Many people have heard that logical induction is about having uncertainty in purely logical facts. It is true that logical induction shows how to construct algorithms that maintain uncertainty in purely logical facts, but in my view this is not really the point of logical induction. The point of logical induction, in my view, is that it is always computable, even when reasoning about contradictory, uncomputable, or self-referential questions. Its capacity to maintain uncertainty about purely logical facts is actually a by-product of the computability of logical induction.

Logical induction addresses the same basic problem that probability theory addresses. Logical induction and probability theory, therefore, are two different answers to the question: what is a reasonable formal method for quantifying uncertainty and updating it in light of evidence? Probability theory and logical induction bot...
Our 2022 Giving
by jefftk — 2022-12-03T15:40:01.678Z

Julia and I usually decide on our
annual giving at the end of each year. For the past few years this
has been relatively straightforward: I was paid

a
lot, so we

donated 50%. This year is less
so, because halfway through I

switched
jobs to work on something directly valuable. While I'm very
excited about the potential of this work, it does pay enough less that
even if we stopped donating we'd be keeping less for ourselves than
before.

One option we considered was donating 50% of our income from the first
half of the year, up until I switched jobs, and then some smaller
proportion of the income from later in the year, representing about
how much we intend to donate going forward. We thought about this, but
decided not to:

The collapse
of FTX, taking with it their philanthropic
arm, means that there is less money available for EA projects,
so it's more valuable to have funding now than in the recent past.

We're not sure how to pick an amount for the second half of
the year. Some of this is not being clear on what sort of donation
norms we'd like to encourage for people working at the kind of
organization they would otherwise be happy to fund, or who take
lower-paying jobs for altruistic reasons. Another part is that what we
personally want to do depends on what the post-FTX funding situation
for our employers end up looking like.

I like having a simple explanation for what we're doing with
our money, and while pro-rating isn't that complicated it's still
something I'd rather avoid. Simplicity is also a reason why we lean
away from donating via lottery or salary sacrifice.

Instead, we've decided to just continue giving 50% for 2022, and give
some smaller proportion in 2023. How much? Not sure yet.

As in past years, we're planning to divide our giving between GiveWell (evidence-backed public
health interventions, like distribution of antimalarial bed nets) and
the EA
Infrastructure Fund (a wide range of projects intended to help the
effective altruism community grow into the best community it can).

"Let's just keep doing what we were doing, and decide later what we
want to do later" is not a very satisfying conclusion, but I'm hoping
things are clearer next year.
Is school good or bad?
by tailcalled — 2022-12-03T13:14:22.737Z

Should society eliminate schools? Should we have more compulsory schooling? Should you send your kids to school? Should you prefer to hire job candidates who have received more schooling, beyond school's correlation with the g factor? Should we consider the spread of education requirements to be a form of class war by the better-educated against the worse-educated which must be opposed for the sake of the worse-educated and the future of society?
MrBeast's Squid Game Tricked Me
by lsusr — 2022-12-03T05:50:02.339Z

I recently watched Squid Game—both the original Netflix series and MrBeast's real life version. Then I watched every video I could find about how MrBeast's Squid Game was made. I was surprised to learn that MrBeast used a whole bunch of CGI (computer-generated imagery). I had taken its realty at face value.

There are four places I know of where MrBeast used CGI.

The sky above the big dirt field wasn't originally sky. The scenes were actually filmed in an indoor stadium.

The smoke effects of players who got eliminated were added in post-production.

A dust effect where a woman fell was CGI.

The big platforms for Tug-of-War and Glass Bridge weren't real at all. Both events took place close to the ground. In those scenes, almost everything the players didn't physically touch was CGI. Even the empty spaces between glass panels in Glass Bridge weren't real. They were solid black squares that got removed in post-production.

The sky, smoke and dust are all plausible. But the big giant platforms had to be CGI. Using such platforms in a real-life game would be ludicrously dangerous, especially without safety harnesses. Real platforms would be unnecessarily expensive too compared to replacing backgrounds, which is part of VFX artists' standard toolkit.

Why didn't I notice?

First of all, the VFX team SoKrispyMedia is very good at VFX. But that alone isn't sufficient.

My experience of watching MrBeast's Squid Game reminds me of a magic trick routine I used to perform. To the audience it looked like two separate magic tricks but they actually worked together. First I would perform a series of tricks that used a real regular deck of cards. Everything was pure sleight-of-hand. Once my audience had gotten used to the idea that I'm doing everything legitimately I would switch to a set of gimmicked (fake) cards. The effect was much stronger than when I started out with the gimmicked (fake) cards.

MrBeast (perhaps unintentionally) pulled off a similar trick. When I watch television and movies, I'm constantly scrutinizing each scene to figure out how the filmmakers created it. But most of MrBeast's videos are completely real. When I watched MrBeast's Squid Game it did not occur to me to question anything, even when the images onscreen were unrealistic. I wasn't on-guard for CGI trickery.

Unexamined Beliefs

Most of our beliefs never go through a thorough examination. There's just to...
Great Cryonics Survey of 2022
by Mati_Roy — 2022-12-03T05:10:14.536Z

The text that follows is a copy-paste from the survey creator's Facebook post, shared with permission.

Asking all Cryonicists and those considering it or curious about it to...
~ Please take the Great Cryonics Survey of 2022 ~

Go to --> bit.ly/cryosurvey2022

Take the survey if:

You're signed up for Cryonics

You think Cryonics is interesting but you aren't ready to sign up yet.

You've decided to sign up but haven't gotten around to it yet.

The goal of this survey is to understand what makes Cryonicists tick using data rather than anecdotes.
Here are some examples of the kinds of issues this survey is meant to understand:

How many Cryonicists are in favor of "mind uploading" and how many are against it?

How do Cryonicists think about death?

How many Cryonicists suffer from some form of death anxiety?

How many Cryonicists are men vs women?

What differentiates Cryonicists from non-Cryonicists with overall similar mindsets?

How affordable do Cryonicists think Cryonics is for them?

What do Cryonicists and non-Cryonicists think about pet Cryonics, and why?

How do Cryonicists identify politically?

Where do people first learn about Cryonics?

How do Cryonicists feel about children?

And much more!

All results are anonymous (we don't ask for names or the like). All results will be analyzed and that analysis released to the public in a few months.

bit.ly/cryosurvey2022

Get help signing up

By the way, I've been helping people (decide whether to) sign up for years, and I'm now part of the new Independent Cryonics Educators program which supports people doing that, both logistically and financially, although I don't let this bias my recommendations. So don't hesitate to reach out to me on Facebook, Twitter, or email (mathieu.roy.37@gmail.com). If you use the following code, you'll get a 50% discount on your first year of membership with Alcor: 2omojQif.
Causal scrubbing: results on induction heads
by LawrenceC — 2022-12-03T00:59:18.327Z

* Authors sorted alphabetically.

This is a more detailed look at our work applying causal scrubbing to induction heads. The results are also summarized here.

Introduction

In this post, we’ll apply the causal scrubbing methodology to investigate how induction heads work in a particular 2-layer attention-only language model.[1] While we won’t end up reaching hypotheses that are fully specific or fully human-understandable, causal scrubbing will allow us to validate claims about which components and computations of the model are important.
We'll first identify the induction heads in our model and the distribution over which we want to explain these heads' behavior. We'll then show that an initial naive hypothesis about how these induction heads mechanistically work—similar to the one described in

the Induction Heads paper

(Olsson et al 2022)—only explains 35% of the loss. We’ll then go on to verify that a slightly more general, but still reasonably specific, hypothesis can explain 89% of the loss. It turns out to be the case that induction heads in this small model use information that is flowing through a variety of paths through the model – not just previous token heads and the embeddings. However, the important paths can be constrained considerably – for instance, only a small number of sequence positions are relevant and the way that attention varies in layer 0 is not that important.

As with the paren balance checker post, this post is mostly intended to be pedagogical. You can treat it as an initial foray into developing and testing a hypothesis about how induction heads work in small models. We won't describe in detail the exploratory work we did to produce various hypotheses in this document; we mostly used standard techniques (such as looking at attention patterns) and causal scrubbing itself (including looking at internal activations from the scrubbed model such as log-probabilities and attention patterns).

Methodology

Overall approach

The experiments prescribed by causal scrubbing in this post are roughly equivalent to performing resampling ablations on the parts of the rewritten model that we claim are unimportant. For each ‘actual’ datum we evaluate the loss on, we’ll always use a single ‘other’ datum for this resampling ablation.

Throughout this document we measure hypothe...