trentmkelly/LessWrong-43k huggingface dataset:
Is behavioral safety "solved" in non-adversarial conditions?
by Robert_AIZI — 2023-05-25T17:56:28.223Z
Link: https://aizi.substack.com/p/is-behavioral-safety-solved-in-non

I’m trying to crystalize something I was said to a friend recently: I think that techniques like

RLHF

and

Constitutional AI

seem to be sufficient for making large language models (LLMs) “behaviorally safe” in non-adversarial conditions.

Let’s define my terms:

Behaviorally safe - I mean the banal “don’t do obviously bad things” sense of AI safety. To paraphrase

Ajeya Cotra’s definition

, being behaviorally safe means the AI doesn’t “lie, steal… , break the law, promote extremism or hate, etc, and they want it to be doing its best to be helpful and friendly”.
Non-adversarial conditions - “Intended use cases”, excluding jailbreaking, prompt injection, etc. To give a concrete definition, I would define non-adversarial conditions to be something like “a typical user, trying in good faith to use the AI in an approved way, and following simple rules and guidelines”.
Solved - To say the problem is solved, I mean it should happen vanishingly rarely, or be easily patchable if it does come up.

So taken together, the title question “Is behavioral safety "solved" in non-adversarial conditions?” translates to:

Would a typical user, trying in good faith to use the AI in an approved way, and following simple rules and guidelines, experience the AI lying, stealing, law-breaking, etc only vanishingly rarely?

Operating purely off my gut and vibes, I’d say the answer is yes.

The notable exception was February 2023 Bing Chat/Sydney, which generated headlines like

Bing Chat is blatantly, aggressively misaligned

. But there’s a widely accepted answer for what went wrong in this case -

gwern’s topvoted comment

about a lack of RLHF for Sydney. And since February 2023, the

examples

of

AI misbehavior

I have heard about are from adversarial prompting, so I’ll take that absence of evidence as evidence of absence.

If the title question is true, I think the AI safety community should consider this a critical milestone, though I would still want to see a lot of safety advances. “It works if you use it as intended” unlocks most of the upside of AI, but obviously in the long-term we can’t rely on powerful AIs systems if they collapse from being poked the wrong way. Additionally, my impression is that it can be arbitrarily hard to generalize from “working in intended cases” to “working in adversarial cases”, so there could still be a long road to get to truly safe AI.

Some questions I’d appreciate feedba...
Book Review: How Minds Change
by bc4026bd4aaa5b7fe — 2023-05-25T17:55:32.218Z

In 2009, Eliezer Yudkowsky published

Raising the Sanity Waterline

. It was the first article in his

Craft and the Community

sequence about the rationality movement itself, and this first article served as something of a mission statement. The rough thesis behind this article—really, the thesis behind the entire rationalist movement—can be paraphrased as something like this:

We currently live in a world where even the smartest people believe plainly untrue things. Religion is a prime example: its supernatural claims are patently untrue, and yet a huge number of people at the top of our institutions—scholars, scientists, leaders—believe otherwise.

But religion is just a symptom. The real problem is humanity's lack of rationalist skills. We have bad epistemology, bad meta-ethics, and we don't update our beliefs based on evidence. If we don't master these skills, we're doomed to just replace religion with something equally as ridiculous.

We have to learn these skills, hone them, and teach them to others, so that people can make accurate decisions and predictions about the world without getting caught up in the fallacies so typical of human reasoning.

The callout of religion dates it: it was from the era where the early English-speaking internet was

a battlefield between atheism and religion

. Religion has slowly receded from public life since then, but the rationality community stuck around, in places like this site and SSC/ACX and the Effective Altruism community.

I hope you'll excuse me, then, if I say that the rationalist community has been a failure.

Sorry! Put down your pitchforks. That's not entirely true. There's a very real sense in which it's been a success. The community has spread and expanded to immense levels. Billions of dollars flow through Effective Altruist organizations to worthy causes. Rationalist and rationalist-adjacent people have written several important and influential books. And pockets of the Bay Area and other major cities have self-sustaining rationalist social circles, filled with amazing people doing ambitious and interesting things.

But that wasn't the point of the community. At least not the entire point. From

Less Wrong's account of its own history

:

After failed attempts at teaching people to use Bayes' Theorem, [Yudkowsky] went largely quiet from [his transhumanist mailing list] to work on AI safety

...
Self-administered EMDR without a therapist is very useful for a lot of things!
by EternallyBlissful — 2023-05-25T17:54:13.389Z

EMDR (Eye Movement Desensitization and Reprocessing) therapy is a structured therapy that encourages the patient to focus briefly on a traumatic memory while simultaneously experiencing bilateral stimulation (typically eye movements, but also tones or taps), which is associated with a reduction in the vividness and emotion associated with the traumatic memories.

EMDR is usually done with a therapist. However, you can also just do self-administered EMDR on your own - as often and whenever you want without any costs! Most people don't seem to know this great "do it on your own" option exists - I didn't. So my main goal with this post is to just make you aware of the fact that: "Hey, there's this great therapeutic tool called EMDR, and you can just do it!". And bearing some important caveats in mind, I highly recommend it. I've been doing emotional work like extended meditation retreats, Internal Family Systems (IFS), Ideal Parent Figures (IPF), Focusing, etc., for a long time, but self-administered EMDR has actually been one of the most helpful techniques of them all to me!

Also, I've found EMDR helpful for a much broader set of problems than the official EMDR protocol implies. You can use it for anything "trauma" in the broadest sense of the word - any unhelpful emotional schema; any strongly negatively charged emotion, belief, or memory that is kind of stuck in unconsciousness.

EMDR also doesn't have to be this complicated thing at all. I don't think you need to know more than there is in this blog post. You can totally 80/20 this, i.e. get 80% of the benefits from just 20% of the knowledge/effort.

How to do self-administered EMDR

Read here how to do EMDR with a therapist:

https://www.helpguide.org/articles/therapy-medication/emdr-therapy.htm

For self-administered EMDR, you simply do the same, just without the therapist! So to summarize the most important steps of the official protocol, you:

Identify:

A specific scene or picture that best represents the traumatic event you’re targeting with this treatment. This is your “target.”
A negative or unhelpful belief that is associated with the traumatic event. For example, “I am helpless,” “I am worthless”, or “I fucked up and therefore I’m a bad person.”
A more wholesome alternative positive belief you would rather believe instead. This belief should reflect what is appropriate in the present. It should als

...
RecurrentGPT: a loom-type tool with a twist
by mishka — 2023-05-25T17:09:37.844Z
Link: https://arxiv.org/abs/2305.13304

This is a link post for RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text paper by a team from ETH Zürich ("Swiss Federal Institute of Technology in Zürich").

The Compleat Cybornaut recent post discusses looming and the evolving field of loom-type tools. We are going to see more loom-type tools in the near future, and it might be helpful for their authors to be aware of the details of RecurrentGPT tool.

The authors of RecurrentGPT don't seem to be aware of the Simulator theory or Cyborgism, but they seem to be affected by their zeitgeist. The abstract starts as follows (italic is mine):

The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is built upon a large language model (LLM) such as ChatGPT and uses natural language to simulate the Long Short-Term Memory mechanism in an LSTM.

AaltoMediaAI notes[1]:

This is not the first paper to use LLMs to both generate text and make notes/edits into a long-term memory. However, this one is interesting in that the approach mimics the operation of LSTMs, on a textual level, using prompt engineering.

GPT-4 comments on the abstract of this paper as follows[2]:

The paper "RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text" by Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, and Mrinmaya Sachan proposes a novel approach to address a key limitation of GPT models: the inability to generate arbitrarily long text due to their fixed-size context.

In the paper, the authors introduce RecurrentGPT, which utilizes a language-based simulation of the recurrence mechanism found in Recurrent Neural Networks (RNNs). This model builds upon a Large Language Model (LLM), such as ChatGPT, and simulates the Long Short-Term Memory (LSTM) mechanism in a language-based format.

RecurrentGPT operates by generating a paragraph of text at each timestep and updating its language-based long-short term memory, which is stored on the hard drive, as well as the prompt. This allows RecurrentGPT to generate texts of arbitrary length without forgetting any of the context. Moreover, since these language-based memories can be observed and edited by users, RecurrentGPT is interpretable and allows for intera

...
The Genie in the Bottle: An Introduction to AI Alignment and Risk 
by Snorkelfarsan — 2023-05-25T16:30:50.998Z

I'm writing an article about the alignment problem and AI risk, and I'm sharing a draft with the fine folks here on LessWrong to elicit some feedback. I suspect most frequent readers here will already be familiar with the contents and arguments presented below, so bear in mind that this isn't really written for you guys, but for a broader "lay" audience. As I'm learning about AI risk and alignment, I'd like feedback on whether or not there are any obvious mistakes or untruths I should correct, whether I've mischaracterized any of the arguments, whether I've missed something obviously important and whether the overall structure works for the purpose of the article. Feedback on style and tone is also welcome.

It's quite a lengthy article, so skimming is fine ;).

Thank you!

* A note on claims: Throughout the article, I’ve referred to various ideas and claims, but I haven’t provided sources for all of them. This is on purpose.

Reasons why AIs might one day kill us all and why you should at least be a little worried.

Introduction

As AI systems become more advanced, concerns about their alignment with human values and interests become more pressing. AI alignment refers to the challenge of ensuring that AI systems behave as we would *intend* and do not harm humans, either accidentally or intentionally. The premise is that AI systems are unlike regular computer programs in that they can exhibit non-obvious, novel behaviors and generate solutions that may not be what humans would have intended, given that they are intelligent.

At the heart of the AI alignment and AI risk discussions is the idea of creating generally intelligent autonomous agents, or more specifically, power-seeking advanced intelligence(s) with the capabilities that being intelligent confers. Advanced AIs are unlike other technologies like nuclear weapons because intelligence is more fundamental than specific technologies. All inventions and technologies are the results of intelligence + applied knowledge. To get nuclear weapons, you first need (enough) intelligence. In a sense, anything that we would want to do or that can be discovered about the universe is just a matter of having the right knowledge. Intelligent machines with capabilities far superior to humans or human organizations could thus invent and create nearly anything that is physically possible - given enough time. Depending on what the ...
AI #13: Potential Algorithmic Improvements
by Zvi — 2023-05-25T15:40:04.953Z

At least two potentially important algorithmic improvements had papers out this week. Both fall under ‘this is a well-known human trick, how about we use that?’ Tree of Thought is an upgrade to Chain of Thought, doing exactly what it metaphorically sounds like it would do. Incorporating world models, learning through interaction via a virtual world, into an LLM’s training is the other. Both claim impressive results. There seems to be this gigantic overhang of rather obvious, easy-to-implement ideas for improving performance and current capabilities, with the only limiting factor being that doing so takes a bit of time.

That’s scary. Who knows how much more is out there, or how far it can go? If it’s all about the algorithm and they’re largely open sourced, there’s no stopping it. Certainly we should be increasingly terrified of doing more larger training runs, and perhaps terrified even without them.

The regulation debate is in full swing. Altman and OpenAI issued a statement reiterating Altman’s congressional testimony, targeting exactly the one choke point we have available to us, which is large training runs, while warning not to ladder pull on the little guy. Now someone – this means you, my friend, yes you – need to get the damn thing written.

The rhetorical discussions about existential risk also continue, despite moral somewhat improving. As the weeks go by, those trying to explain why we might all die get slowly better at navigating the rhetoric and figuring out which approaches have a chance of working on which types of people with what background information, and in which contexts. Slowly, things are shifting in a saner direction, whether or not one thinks it might be enough. Whereas the rhetoric on the other side does not seem to be improving as quickly, which I think reflects the space being searched and also the algorithms being used to search that space.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Small steps.

Language Models Don’t Offer Mundane Utility. A missing small step.

AI Comes to Windows 11. Windows Copilot, similar to 365 Copilot, coming soon.

Fun With Image, Sound and Video Generation. Adobe steps up, plus speculations.

Deepfaketown Right Now. Large momentary stock market drop, oh no.

Deepfaketown and Botpocalypse Soon. How high on our list of problems?

Can’

...
Solving the Mechanistic Interpretability challenges: EIS VII Challenge 2
by StefanHex — 2023-05-25T15:37:54.593Z

We solved* the second Mechanistic Interpretability challenge (Transformer) that Stephen Casper posed in EIS VII. We spent the last Alignment Jam hackathon attempting to solve the two challenges presented there, see here for our solution to the first challenge (CNN). The challenges each provide a pre-trained network, and the task is to

Find the labeling function that the network was trained with
Find the mechanism by which the network works.

*We have understood the network’s labeling mechanism, but not found the original labeling function. Instead we have made a strong argument that it would be intractable to find the labeling function, as we claim that the network has not actually learned the labeling function.

A notebook reproducing all results in this post can be found here (requires no GPU, around ~10GB RAM).

Note that our solution descriptions are optimized with hindsight and skip all wrong paths and unnecessary techniques we tried. It took us, two somewhat experienced researchers, ~24 working hours to basically get the solutions for each challenge, and a couple days more for Stefan to perform the interventions, implement Causal Scrubbing tests, interventions & animations, and to write-up this post.

Task: The second challenge network is a 1-layer transformer consisting of embedding (W_E and W_pos), an Attention layer, and an MLP layer. There are no LayerNorms and neither the attention matrices nor the unembedding have biases.

The transformer is trained on sequences [A, B, C] to predict the next token. A and B are integer tokens from a = 0 to 112, C is always the same token (113). The answer is always either the token 0 or 1. If we consider all inputs we get 113x113 combinations which we can shape into an image to get the image from the challenge (copied below). Black is token 0, and white is token 1. The left panel shows the ground truth, and the right panel the model labels. The model is 98.6% accurate on the full dataset.

Spoilers ahead!

Summary of our solution (TL,DR)

We found that the model basically just learns the shapes by heart, it does not learn any mathematical equations. Concretely we claim that

The model barely uses the attention mechanism. Even if we fix the attention pattern to the dataset-mean, the model labels 92.7% of points correctly. We reverse-engineer the fixed-attention version of the transformer for simplicity, and we don’t exp

...
Malthusian Competition (not as bad as it seems)
by Logan Zoellner — 2023-05-25T15:30:18.534Z

source

Sooner or later, someone is going to do this.

What rules govern "ungovernable" AGI?

In a state of nature, all animals exist in a Malthusian Equilibrium. That is to say, the population of a species increases until some environmental factor (food, habitat, disease) limits its growth.

population increases until carrying capacity is met

By contrast, in economics the production of goods is limited by the laws of supply and demand.

price increases or decreases until supply meets demand

While these may look like different pictures, they are actually the same. As an illustration, consider pets. Are there a commodity manufactured in order to meet demand within the market, or are they animals evolved to fit into a particular niche (human companionship)? The answer is, obviously, both.

This very good boy was created by AI to meet the demand for cute dog pics

These two laws have very different emotional valances "the invisible hand of the market" vs "nature red in tooth and claw". But in fact the same law is at work in both places.

So when we ask "what law will govern ungovernable AGI?", the answer is the same: the law of supply and demand which is also the law of Malthusian Competition.

A great deal of hand-wringing in the alignment community is based around the idea that AGI will "out-compete" humans economically. But taken another way, this is simply another way of stating that AGI is going to make all of us enormously rich.

Capitalism: making people richer for over 200 years

So long as property rights are respected, humans will continue to have a comparative advantage in something, and whatever that is we will be much richer in a world with hyper-competitive AGI than we are today.

What does this have to do with alignment?

Consider two AGIs, otherwise identical:

The first AGI is "owned" by a human being and is programmed to fulfill the owner's wishes as perfectly as possible
The second AGI exists in perfect competition with its peers and competes to sell its services to human beings (who own all of the capital)

I postulate that there is no meaningful difference between AGI 1. and AGI 2. We should not expect "ownership" to confer any particular safety or other benefits over market competition when building AGI.

But what if the AGI owns all of the capital and we're the ones subject to Malthusian Competition?

If you're reading this essay, I ...
You Don't Always Need Indexes
by jefftk — 2023-05-25T14:20:07.901Z

Sometimes you have a lot of data, and one approach to support quick
searches is pre-processing it to build an index so a search can
involve only looking at a small fraction of the total data. The
threshold at which it's worth switching to indexing, though, might be
higher than you'd guess. Here are some cases I've worked on where
full scans were better engineering choices:

Ten years ago I wrote an interoffice messaging application for
a small billing service. Messages were stored in MySQL and I was
going to add indexing if full-text searches got slow or we had load
issues, but even with ten years worth of messages to search it stayed
responsive. (This was before MySQL supported FULLTEXT
indexes with InnoDB, added in v5.6,
and at the time indexing would have meant installing and maintaining
Sphinx.)

I recently came across
someone maintaining a 0.5GB full text index to support searching their
shell history, 100k commands. I use
grep on a flat file, and testing now it takes 200ms for a
query across my 180k history entries.

My contra dance search
tool ranks each dance in response to your query, with no
geospatial indexing, because there are just ~350 dances.

The viral counts
explorer I've been playing with for work searches the taxonomic
tree of human viruses in real time, scanning ~15k names with JS's
"includes" command about as fast as you can type.

When I worked in ads I would often need to debug issues using
production logs, and would use Dremel (Melnik
2010, Melnik
2020) to run a distributed scan of very large amounts of data at
interactive speeds. Because queries were relatively rare, an index
would have been far more expensive to maintain.

Unless you know from the start that you'll be searching hundreds of
millions of records, consider starting with simple scans and only add
indexing if you can't get acceptable performance. And even then, if
queries are rare and highly varied you may still do better to do
the work at query time instead of ingestion time.
Theories of Biological Inspiration
by Eric Zhang — 2023-05-25T13:07:10.972Z

Many influential AI techniques either explicitly draw inspiration from or are similar to mechanisms found in biology.

Biological neurons and artificial neural networks
The visual cortex and convolutional neural networks
Attention and attention mechanisms like Transformers
Short-term memory and LSTMs
Episodic memory and experience replay
Curiosity and exploration techniques
lots of others, probably

Some basic reasons why this might be:

Because those answers are convergent. Any two good solutions to a problem are going to get the same answers because there is just one answer.
Because we're stealing. Evolution did a whole lot of work for us and it's easier to steal than figure it out on our own. If we did try to do it on our own without stealing, we might find totally different answers, but that would take a whole lot more time and effort.
Because we're romantics. We like looking for solutions that look to us like ourselves, that are "in our own image", as it were. We didn't gain much technical advantage from doing it this way and could've just as well done it a different way, but it was more appealing to us to do it this way.
It's just marketing. These techniques aren't really all that similar to the biological mechanisms, but it gets more hype if you can draw a connection to something the brain is doing.

All of these play some role, and in differing degrees depending on the particular technique. The extent to which any of these explanations is generally the case may have strategically important implications for AI safety.

Insofar as 2 is more the case, then progress in neuroscience may be something of a limiting factor in further AI progress. It may also point to progress in techniques which more closely imitate brains (such as SNNs) being things to watch.

To the extent 1 or 2 are strongly the case, searching for a paradigm very different from deep learning which might be more interpretable may be hopeless (even more than it already is, let us say).

I'd be interested in hearing what people think about to what extent these four stories (or a different one I didn't think of) apply either in the general case or to specific techniques.