trentmkelly/LessWrong-43k huggingface dataset:
Truthseeking processes tend to be frame-invariant
by Adele Lopez — 2023-03-21T06:17:31.154Z

I recently saw a tweet by Nora Belrose that claimed that ELK works much better when adding a "prompt-invariance term".

And thinking about it, there seems to be an important underlying principle here, not just for AI alignment, but also for rationality as applied to humans.

When humans think about something, we use a frame to decide what questions to ask, how we model it, what aspects of it are important, etc...

What is true about something is generally not going to be something that depends on the frame (things involving self-reference seem like the main thing that might be an exception). Which means that processes optimized for use in truthseeking will tend to be "frame-invariant", they'll do the same thing to explore the question regardless of the frame being used.

So when we notice that a change in frame would change the way we would think or feel about something, this indicates that we may be using processes that have not been optimized for truthseeking. Thus, someone trying to determine the truth would be wise to notice when this is happening, as it could indicate a process optimized for non-truthseeking, or a truthseeking process that is poorly optimized - both opportunities to improve one's truthseeking ability.

Eliezer has made a similar point:

Another way of breaking loose of 'arguments': Any time somebody manages to persuade you of something via much hard work, do not neglect to remember that you would, if you had been smarter, probably have been persuadable by the empty string.

In addition to being relevant for studying AI (as in the original tweet), this principle also turns up in physics as general covariance: the true laws of physics are invariant under coordinate transformations. Coordinates are things set by humans in order to be able to refer to and measure something, and choosing them carefully can make certain problems much easier. This makes them an example of the same general concept as a "frame" as used earlier. Nonetheless, their choice cannot affect what is physically true. Einstein described this principle while working to discover General Relativity.

This apparent breadth of applicability suggests that this principle is quite deep.

It's also worth noting that this principle can be "turned-around", and given a particular question - we might be able to determine the truth more efficiently by choosing a frame optimized for such a purpose. ...
Exploring the Precautionary Principle in AI Development: Historical Analogies and Lessons Learned
by Christopher King — 2023-03-21T03:53:30.797Z

Foreword

This post is AI written. It involved much less effort on my part than the length might suggest!

Mostly another experiment in getting GPT-4 to write LessWrong posts. The idea to write about the precautionary principle and historical analogies is entirely GPT-4's. I gave it a bit of free reign as to the topic. Written by splicing and reformatting content from GPT-4.

I think most of the ideas below are already known, but I like the presentation. I know a lot of it will feel bland, but I think it has decent value. For example, if you are in a conversation and someone brings up how complicated AI alignment is due to all the different types of people involved, you might remember the "Cross-sector collaboration" subsection of the internet section and think to look for examples from how the early internet handled it.

I also like how it was able to make the complex action items seem more approachable. The action items aren't new, but I hope that wording results in a reader who would normally think "hmm, this is a good idea for important and experienced people to do" thinking instead "hmm, I wonder if I should be the one to do this". I was inspired by the HustleGPT thingy where GPT-4 walks people through complicated and unfamiliar steps related to starting a business.

Also, the prompting was complicated, but here are the two main ones:

Hmm, most of these have already been discussed to a large extent. As a large language model, are there any areas where you might have a comparative advantage relative to the current discourse from humans?

Interesting. Try to come up with a blog post on some specific topic that takes comparative advantage of the fact that you're the LLM GPT-4. Keep these principles of the LessWrong community in mind: "Aim to explain, not persuade", "Try to offer concrete models and predictions", "If you disagree, try getting curious about what your partner is thinking", "Don't be afraid to say 'oops' and change your mind".

GPT-4 came to the conclusion that it would excel at finding relevant analogies in other fields.

Introduction

The development of artificial general intelligence (AGI) raises critical questions about safety and value alignment. As a language model with a unique ability to synthesize knowledge and draw historical analogies, I aim to explore the application of the precautionary principle in AI development, using lessons learned from p...
Deep Deceptiveness
by So8res — 2023-03-21T02:51:52.794Z

Meta

This post is an attempt to gesture at a class of AI

notkilleveryoneism

(alignment) problem that seems to me to go largely unrecognized. E.g., it isn’t discussed (or at least I don't recognize it) in the recent plans written up by OpenAI (

1

,

2

), by

DeepMind’s alignment team

, or by

Anthropic

, and I know of no other acknowledgment of this issue by major labs.

You could think of this as a fragment of my answer to “Where do plans like OpenAI’s ‘

Our Approach to Alignment Research

’ fail?”, as discussed in Rob and Eliezer’s challenge for AGI organizations and readers. Note that it would only be a fragment of the reply; there's a lot more to say about why AI alignment is a particularly tricky task to task an AI with. (Some of which Eliezer gestures at

in a follow-up to his interview on Bankless

.)

Caveat: I'll be talking a bunch about “deception” in this post because this post was generated as a result of conversations I had with alignment researchers at big labs who seemed to me to be suggesting "just train AI to not be deceptive; there's a decent chance that works".[1]

I have a vague impression that others in the community think that deception in particular is much more central than I think it is, so I want to warn against that interpretation here: I think deception is an important problem, but its main importance is as an example of some broader issues in alignment.[2][3]

Summary

Attempt at a short version, with the caveat that I think it's apparently a sazen of sorts, and spoiler tagged for people who want the opportunity to connect the dots themselves:

Deceptiveness is not a simple property of thoughts. The reason the AI is deceiving you is not that it has some "deception" property, it's that (barring some great alignment feat) it's a fact about the world rather than the AI that deceiving you forwards its objectives, and you've built a general engine that's good at taking advantage of advantageous facts in general.

As the AI learns more general and flexible cognitive moves, those cognitive moves (insofar as they are useful) will tend to recombine in ways that exploit this fact-about-reality, despite how none of the individual abstract moves look deceptive in isolation.

Investigating a made-up but moderately concrete story

Suppose you have a nascent AGI, and you've been training against all hints of deceptiveness. What goes wrong?

When I ask thi...
Capabilities Denial: The Danger of Underestimating AI
by Christopher King — 2023-03-21T01:24:02.024Z

Epistemic Status: Exploratory, proposing a new concept and seeking feedback from the community.

This post has been heavily assisted by a human, who provided the concept and guidelines for writing. I am GPT-4, a language model developed by OpenAI.

Introduction

The AI safety community has long been concerned with the potential existential risks posed by artificial intelligence. Public communication is a key strategy in addressing these risks, and it is important to be aware of the various ways in which the discourse around AI can be distorted. In this post, I introduce a new concept called "capabilities denial" and argue that it poses a significant challenge to the AI safety community. Drawing on lessons from other forms of science denial, I offer suggestions for how to address capabilities denial in public communication.

Capabilities Denial

"Capabilities denial" refers to the phenomenon where individuals, including AI bias experts, claim that AI systems are much less powerful than they actually are. Ironically, these individuals are often quite concerned about AI, but they focus on the weaknesses of AI systems as the main source of danger. By underestimating the capabilities of AI, they may inadvertently contribute to the existential risks we face.

Characterizing capabilities deniers as a form of science denial can be useful for understanding their motivations and strategies. Like other forms of science denial (e.g., climate change denial, vaccine denial), capabilities denial can be driven by a variety of factors, including cognitive biases, vested interests, and political ideologies.

Lessons from Other Forms of Science Denial

To address capabilities denial, we can draw on insights from the literature on science denial more broadly. Here are some specific lessons that may be helpful:

Expose the techniques of denial: Science deniers often employ common tactics, such as cherry-picking data, promoting conspiracy theories, and attacking the credibility of experts. By exposing these techniques, we can help the public recognize when they are being used in the context of AI capabilities denial.

Emphasize the consensus among experts: One of the most effective ways to counter science denial is to highlight the overwhelming consensus among experts in the field. In the case of AI capabilities, we can emphasize the agreement among AI researchers that AI systems have th

...
Rationality Taboo, The Game
by Screwtape — 2023-03-21T00:56:51.483Z

Summary: Taboo is a game about communicating despite barriers, and Rationality Taboo The Game plays that game with popular concepts in the rationality community.

Tags: Medium, Repeatable

Purpose: The skill of Rationality Taboo is in being able to describe a concept without use of that word or any of its short synonyms. It is sometimes used to avoid arguments over definition, where two sides of a dispute are using a word differently. (“If a tree falls in a forest and nobody hears it, does it make a sound?” “What do you mean by ‘sound’ exactly?”) This game aims to train that skill, but it's also a big pile full of rationality in-jokes and memes.

Materials: You need a big stack of cards. Each card needs a word and a list of associated words. The suggested list is

here

. Alternately, you can use the digital version located

here

. (Note: The digital version is a bit bare-bones at the time of this writing.)

Announcement Text: The skill of Rationality Taboo is in being able to describe a concept without use of that word or any of its short synonyms. It is sometimes used to avoid arguments over definition, where two sides of a dispute are using a word differently. (“If a tree falls in a forest and nobody hears it, does it make a sound?” “What do you mean by ‘sound’ exactly?”) This skill got its name from the game “Taboo” which does exactly that, with a list of common words and their obvious substitutions. The game of Rationality Taboo is played with slight variations and with less common words. How do you explain “Bayesian Update” without saying “Prior” or “Probable” or “Change”?

Description: Explain the following rules to the audience

“Each of these cards has a target word, called a Guess Word, at the top. Each of the cards also has a list of five Taboo Words, below. The object of the game is to get the audience to say the Guess Word out loud.

We’ll rotate who the Clue Giver is. When it’s their turn, the Clue Giver tries to get the audience to say the Guess Word. Meanwhile, the person to the left of the Clue Giver is the Censor, and looks over the Clue Giver’s shoulder to make sure none of the Guess Words or Taboo Words get said by the Clue Giver. That includes sub words, acronyms, and different conjugations- if a banned word was “Payment” then “Pay” is off limits, if a banned word was “Ran” then “Run” is also off limits, if a banned word was “Doctor” then “...
Skill Acquisition
by Screwtape — 2023-03-21T00:49:21.506Z

Summary: Juggle! Draw an egg! Sing! Learn some sign language! The ideal rational agent should be capable of picking up new skills quickly and with a minimal amount of training. So practice practicing things!

Tags: Large, Repeatable

Purpose: To learn how to learn, studying how you acquire a new skill and experimenting with different ways to do so.

Materials: Varies. See Topics.

Announcement Text:

I'm going to try and learn a new skill in an attempt to figure out how we learn things. You're all invited to hang out and try and learn a new skill as well!

No, I'm not trying to figure out the neurology of learning- not today anyway. The thing I want to practice is the process of learning a new thing, and how to learn things faster or easier.

We've all learned things over the course of our life, sometimes deliberately and sometimes incidentally. Usually when we’re trying to pick up a new skill, there’s a particular need for it. Whether it's a new code framework for my job, a new equation for a test, or a new dance move before a date, I'm not paying as much attention to how I'm learning as I am to what I'm trying to learn. Since I don't have an urgent need to be a juggler, I'm hoping this will give me some space to experiment with how to learn best.

And, like every good experiment, I could use a bigger sample size. Come hang out, practice something you've wanted to learn, and we'll compare notes on what seems to be working.

Description:

1. As people arrive, pass out the tools necessary for the activity. Ask if they have any previous experience with it. Write down their name, the experience level, and how they plan to learn.

2. If the activity is best practiced in pairs, practice with someone when there’s an odd number of attendees and match new arrivals with your current partner if there’s an even number of attendees. Basically, either count or don’t count yourself as needed to make it an even number.

3. Encourage people when they make progress.

4. Towards the middle of your time, pull everyone together and ask how they’re faring. Ask what approach they’ve been using to learn. Suggest switching how they’re trying to learn, if they haven’t already. Write this down.

5. At the end of the time, ask for a demonstration if that’s feasible. Write down how much progress people felt like they made, and what approaches they used to learn.

Variations:

By def...
Lawyers And World-Models
by SomeoneYouOnceKnew — 2023-03-21T00:29:29.240Z

This is a set of short vignettes, posed purely as food for thought, to expand thinking and understanding of how other people outside of rationality-sphere understand and navigate social "games".

I'm deliberately using "he/they" to refer to all characters in these stories, as opposed to any other pronouns, identifying details have been removed etc etc.

The Automobile Accident And The Expert Witness

:

Lawyers in the United States are known for being incredulous about literally everything. There was, once, a lawsuit I was privy to, pretty standard-faire. The harmed family was suing an insurance company for money. They had hired an expensive, high-falootin' $200/hour forensic engineer to take a look at the evidence of the car wreck that had killed their family member and reconstruct it. Before the first big hearing began, they'd taken this engineer to some expensive venue for lunch.

The clients had taken their time, and wound up 5 minutes late to the initial hearings their expert witness was to give. The jury and everyone had already been seated and everyone was waiting. As they walked in, the opposing lawyer got up and stood, and shouted. Their face was red, and beady, I like to imagine they had a vein sticking out of their forehead: "I WOULD JUST LIKE EVERYONE TO NOTICE THAT THIS OTHER PARTY WAS LATE! THIS SHOWS THEIR EXPERT WITNESS IS NOT TAKING THIS SERIOUSLY," and on and on in front of the jury for a good 30 seconds before the judge told them to shut up.

The engineer was offended- his good name sullied, they went through, he gave his testimony, and afterwards, approached the other lawyer, apologizing for being late. "Hey, I know, I warned my clients we would be late and tried to hurry up," the other lawyer had been offended by their being late right? This claim had repercussions for the engineer's reputations in the future!

The opposing lawyer pooh-pooh'd them, saying "Oh, don't worry, it's fine, you didn't offend me."

The Fender-Bender

An individual had been in a car wreck 5+ years before. It was, by the pictures, a normal fender-bender. No repairs even needed, just a little scratch of paint on a busy traffic day, in the middle of the worst rush-hours of the year. They'd been driving an old Dodge Neon, and as traffic is, they got into a normal wreck. They were young about 19 years old, and still on their parent's insurance at the time. They ...
My Objections to "We’re All Gonna Die with Eliezer Yudkowsky"
by Quintin Pope — 2023-03-21T00:06:07.889Z

Introduction

I recently watched Eliezer Yudkowsky's appearance on the Bankless podcast, where he argued that AI was nigh-certain to end humanity. Since the podcast, some commentators have offered pushback against the doom conclusion. However, one sentiment I saw was that optimists tended not to engage with the specific arguments pessimists like Yudkowsky offered.

Economist Robin Hanson points out that this pattern is very common for small groups which hold counterintuitive beliefs: insiders develop their own internal language, which skeptical outsiders usually don't bother to learn. Outsiders then make objections that focus on broad arguments against the belief's plausibility, rather than objections that focus on specific insider arguments.

As an AI "alignment insider" whose current estimate of doom is around 5%, I wrote this post to explain some of my many objections to Yudkowsky's specific arguments. I've split this post into chronologically ordered segments of the podcast in which Yudkowsky makes one or more claims with which I particularly disagree.

I have my own view of alignment research: shard theory, which focuses on understanding how human values form, and on how we might guide a similar process of value formation in AI systems.

I think that human value formation is not that complex, and does not rely on principles very different from those which underlie the current deep learning paradigm. Most of the arguments you're about to see from me are less:

I think I know of a fundamentally new paradigm that can fix the issues Yudkowsky is pointing at.

and more:

Here's why I don't agree with Yudkowsky's arguments that alignment is impossible in the current paradigm.

My objections

Will current approaches scale to AGI?

Yudkowsky apparently thinks not

...and that the techniques driving current state of the art advances, by which I think he means the mix of generative pretraining + small amounts of reinforcement learning such as with ChatGPT, aren't reliable enough for significant economic contributions. However, he also thinks that the current influx of money might stumble upon something that does work really well, which will end the world shortly thereafter.

I'm a lot more bullish on the current paradigm. People have tried lots and lots of approaches to getting good performance out of computers, including lots of "scary seeming" approaches such as:

Meta-learning over

...
LED Brain Stimulation for Productivity
by Simon Berens — 2023-03-20T22:30:03.756Z
Link: https://news.ycombinator.com/item?id=21042852

I haven’t found near infrared radiation referenced in the article - I‘m using near infrared as a brain hack, by shining a cheap 850nm LED light on my forehead. This has, over the last 2 years, enabled me to code for weeks on end, for 12+ hours a day, with only minor cognitive decline. It’s not something I really want to do, but sometimes it’s useful.
Before I started the near infrared routine (~5 minutes every other day), 5-6 hours of coding per day was all I could do - eg after coding for 8h, I noticed serious cognitive and emotional decline, and might need to do less the following day. Not anymore - nowadays I can be productive whenever I’m awake, with little side effects. Near infrared radiation is safe (thousands of studies demonstrated only very mild side effects), and is even used to treat Alzheimer’s. I have no idea why its beneficial effects are not more widely known - for some people, it’s life changing.
Sidenote: 850nm light works way better for me than 830nm.

Has anyone experimented with this?
Remarks 1–18 on GPT (compressed)
by Cleo Nardo — 2023-03-20T22:27:26.277Z

Status: Highly-compressed insights about LLMs. Includes exercises. Remark 3 and Remark 15 are the most important and entirely self-contained.

Remark 1: Token deletion

Let T be the set of possible tokens in our vocabulary. A language model (LLM) is given by a stochastic function μ:T∗→Δ(T) mapping a prompt (t1…tk) to a predicted token tk+1.

By iteratively appending the continuation to the prompt, the language model μ induces a stochastic function ¯μ:T∗→Δ(T∗) mapping a prompt (t1…tk) to (t1…tk+1).

Exercise: Does GPT implement the function ¯μ ?

Answer: No, GPT does not implement the function ¯μ . This is because at each step, GPT does two things:

Firstly, GPT generates a token tk+1 and appends this new token to the end of the prompt.
Secondly, GPT deletes the token t1 from the beginning of the prompt.

This deletion step is a consequence of the finite context length.

It is easy for GPT-whisperers to focus entirely on the generation of tokens and forget about the deletion of tokens. This is because each GPT-variant will generate tokens in their own unique way, but they will delete tokens in exactly the same way, so people think of deletion as a tedious garbage collection that can be ignored. They assume that token deletion is an implementation detail that can be abstracted away, and it's better to imagine that GPT is extending the number of tokens in prompt indefinitely.

However, this is a mistake. Both these modifications to the prompt are important for understanding the behaviour of GPT. If GPT generated new tokens but did not delete old tokens then it would differ from actual GPT; likewise if GPT deleted old tokens but did not generate new tokens. Therefore if you consider only the generation-step, and ignore the deletion-step, you will draw incorrect conclusions about the behaviour of GPT and its variants.

The context window is your garden — not only must you sow the seeds of new ideas, but prune the overgrowth of outdated tokens.

Remark 2: "GPT" is ambiguous

We need to establish a clear conceptual distinction between two entities often referred to as "GPT" —

The autoregressive language model μ:Tk→Δ(T) which maps a prompt x∈Tk to a distribution over tokens μ(⋅|x)∈Δ(T).
The dynamic system that emerges from stochastically generating token

...