trentmkelly/LessWrong-43k huggingface dataset:
Give the AI safe tools
by Adam Jermyn — 2022-06-03T17:04:53.050Z

One kind of concern with AI is that:

There are some tools that are instrumentally useful for an AI to have.
Most/the most accessible versions of those tools are dangerous.
The AI doesn’t care which versions are dangerous.
Hence, the AI will probably develop dangerous tools for instrumental reasons.

You might call concerns like this Instrumental Danger Problems. This post aims to examine some existing approaches to Instrumental Danger Problems, and to introduce a new one, namely “Giving the AI safe tools”.

A few examples

Here are a few concrete examples of Instrumental Danger Problems:

An (incomplete) statement of the inner alignment problem:

Mesa-optimizers are instrumentally useful for an AI to have.
The easiest mesa-optimizers to find are misaligned, hence dangerous.
The AI doesn’t care which mesa-optimizers are dangerous.
Hence, the AI will probably develop dangerous mesa-optimizers for instrumental reasons.

One of the arguments in

Are Minimal Circuits Deceptive?

:

It is instrumentally useful for an AI to search for programs.
Many simple programs are dangerous because they cannot encode or point to our values.
The AI doesn’t care which programs are dangerous.
Hence, the AI will probably find and run dangerous programs for instrumental reasons.

Concerns about optimizing on human feedback:

It is instrumentally useful for the AI to have a model of the human’s values.
The most accessible models of human values are dangerous because they don’t generalize out of the training distribution.
The AI doesn’t care which versions are dangerous, just which do well in training.
Hence, the AI will probably develop a dangerous model of human values for instrumental reasons.

How can we avoid these concerns?

I think this is a useful lens because it suggests some approaches we can use to make AI safer.

(1) Make the tools useless

We’re only worried about tools that are instrumentally useful. What if we make the tools-of-interest useless? This seems hard without sacrificing competitiveness, but I think there are some cases where it’s possible.

For instance some tools are only useful because of choices we made in specifying a goal or setting up the training environment. Training a narrow microscope-AI in a multiplayer environment artificially makes all sorts of tools instrumentally useful (e.g. theory-of-mind) which wouldn’t have been otherwise.

That’s a silly example, but it illustrates that we ...
Rationality camps in Oxford for 16-20 yo: Apply by June 12th
by Anna Gajdova — 2022-06-03T15:22:10.577Z

Our team aims to run rationality camps that we would have enjoyed and benefited from as teenagers ourselves. If you know someone between 16-20 yo who might be interested, please forward them this.

The European Summer Program on Rationality (ESPR) is an immersive workshop for mathematically talented students with a desire to understand themselves and the world.

Where: Oxford, UK.

When: ESPR has two runs this year:

17th - 27th August
18th - 28th August

Who: Applicants between 16-20 yo.

How to apply: Fill in the application form by June 12th.

The curriculum covers a wide range of topics, from game theory, cryptography, and mathematical logic, to AI safety, styles of communication, and cognitive science. The goal of the program is to help students hone rigorous, quantitative skills as they acquire a toolbox of useful concepts and practical techniques applicable in all walks of life.

ESPR’s tuition, room, and board are free for all admitted students. Travel scholarships are also available.

For more information, visit our website. You can also subscribe for updates about future programs.
I'm trying out "asteroid mindset"
by Alex_Altair — 2022-06-03T13:35:48.614Z

This is a personal note, and not an advocation that anyone do the same. I'm honestly not really sure why I'm writing it. I think I just want to talk about it in a place where other people might feel similarly or have useful things to say.

Like many others, the past few months of AI advancement (and more generally, since GPT-3) have felt to me like something of a turning point. I have always been sold on the arguments for AI x-risk, but my timelines were always very wide. It always seemed plausible to me that we were one algorithm around the corner from doom, and it also seemed entirely plausible to me that I would die of old age before AGI happened.

My timelines are no longer wide.

I should make it clear to any readers that I am under no illusion that I am a particularly notable or impactful. This is emphatically not a post telling you that you haven't worked hard enough. I am in the least position to chide anyone for their impact.

My biggest problem has always been productivity/focus/motivation et cetera. I have a pretty unusual psychology, and likely a pretty unrelatable one. I have always been "pathologically content", happy and satisfied by default. This has its advantages, but a disadvantage is that I'm rarely motivated to change the world around me.

But in the course of paying attention to my own drives, I have repeatedly observed that I am reliably motivated at the pointy part of hyperbolic discounting. My favorite example is spilling a glass of water. I absolutely never react with, *sigh* I guess I should go clean that up.... Instead, I'm just up, getting paper towels. There's no hesitation to even overcome. Similarly, when I've been in a theatre production, or when I've worked at a busy cafe, there's no attempt to save energy or savor the moment -- I just do the thing. Again, I'm not saying that I'm any good at the job; otherwise I'd be doing ops right now. But the drive is reliable. I always finish my taxes on time. I always find a job before my money runs out.

A couple years ago I picked up the book Seveneves. (The following isn't really spoilers, because it is the premise of the book.) In it, the moon has broken into pieces, and the debris will eventually rain onto the Earth, raising the atmospheric temperature to hundreds of degrees, rending the entire planet profoundly uninhabitable. The characters calculate that there are two years remaining. ...
Monthly Shorts 5/22
by Celer — 2022-06-03T13:10:11.963Z

Justice

A fun toy that shows how prison populations would change in response to various modifications. See how little ending the war on drugs actually does! See how cheaply you can punish drunk drivers! And remember that the majority of people in prison are in for violent crime, and a serious attempt to grapple with that will mean cutting prison sentences for violent crimes dramatically.

Well, it’s about 36 years late, but the DoJ now claims that they will no longer prosecute people for playing around with computer systems in good faith.

This article on the limitations of an elite-focused politics of identity-based deference is well-crafted, and has everything I wish I could have said.

I’ve discovered an actual origination of the million:statistic line, though it ends up being more plausibly attributed to Stalin than I’d initially expected. Kurt Tuchlsky, German journalist. He’s got a number of other quotes (more in German here, and some deeper quote history investigation here).

A country is not only what it does but what it tolerates.

Revolution! The people howls and cries, Freedom, that’s what we’re needing! We’ve needed it for centuries, our arteries are bleeding. The stage is shaking, the audience rock. The whole thing is over by nine o’clock.

You may have thought that the US military over-recruited among the nation’s poorest. Nope!

I’m not entirely clear on why. Pitch me why I should believe one theory (drug and criminal exclusions) over another (education requirements) over another (honor cultures and pride) over another (positive view of the state).

Learning

For your enjoyment, a FOIAed training game from the CIA! It’s not particularly *good*, but I think it’s interesting.

"Was Reading Recovery harmful? I wouldn't go as far as to say that," he said. "But what we do know is that the kids that got it for some reason ended up losing their gains and then falling behind."

Yes, a widely employed program for helping kids learn to read was harmful. If you make kids fall behind in reading, you have harmed them. Normally this is not complicated!

Speaking of complicated, this Pro Publica piece in Plain English on the failures of Arizona to support developmentally disabled people is fascinatingly written.

Max Gladstone is one of my favorite panelists at SFF cons, and this substack on Moon is a Harsh Mistress is good. But it’s also about something I’ve noticed, which is that ma...
On effective altruism, utilitarianism and localness
by David Hugh-Jones — 2022-06-03T10:34:31.661Z
Link: https://wyclif.substack.com/p/on-effective-altruism-utilitarianism?utm_source=twitter&utm_campaign=auto_share&s=w

Utilitarianism and others

Folk physics evolved as a way of predicting what will happen to objects. It works fine within its domain, but fails beyond that. To give more precise answers, we developed Newtonian physics. But even Newtonian physics has limits. It breaks down at very large or small scales.

Morality evolved as a system for getting on well with our neighbours. Our folk theories of morality work fine within their domain, but fail beyond that. To deal with higher level problems, like policy questions facing whole societies, we developed the theory of utilitarianism. But even utilitarianism has its limits. Faced with questions of the infinite future, it swiftly devolves into fun with maths.

Utilitarianism treats all human concerns as preferences, and turns them into numbers. That is a helpful tool for trading off competing concerns. But don’t mistake the map for the territory.

The Effective Altruism community worries about paper clip maximisers: Artificial Intelligences which serve their single goal obsessively. You tell them to produce paperclips; they recycle humans into paperclips. The only way to dissuade them is to appeal to their inhuman goals: “spare me and I’ll slave in your paperclip factory!”

Utilitarians reduce all concerns to maximising utility. They can’t be swayed by argument, except about how to maximise utility. This makes them a bit like paperclip maximisers themselves.

Utilitarianism is originally an outgrowth of Christianity.[1] Like Christianity, it imposes extreme moral demands on people. Take up your cross and follow me. Sell all your possessions and give them to the poor. But utilitarianism is infinitely more scientific — and infinitely more one-sided. Jesus thought even the rich might be forgiven. For utilitarianism, forgiveness doesn’t come into it. Either someone is maximizing utility, or they’re in the way.

Morality evolved as a system for getting on well with our neighbours. It’s lucky that the Axial religions came along and pushed it to be more than this — since your neighbours may be, say, fellow Athenian slave holders. But there are losses as well as gains. One possible loss, in replacing the man who has to get on with his neighbours with the lone actor responsible before God for the whole world, is humility.

C.S. Lewis famously said that when people stop believing in God, they don’t believe in nothing — they believe in anyt...
[Linkpost] A Chinese AI optimized for killing
by RomanS — 2022-06-03T09:17:42.028Z

The AI depicted in the Terminator movies is rather stupid: there are much more efficient ways to kill all humans than robots with guns.

We can safely ignore the unrealistic Terminator-like scenario of AI X-risk.

...Or can we?

Tsinghua University is a top university located in Beijing. It is heavily involved in research for the Chinese military. One of its military labs is called "The State Key Laboratory of Intelligent Technology and Systems".

In 2021, two of the university's researchers released a paper called Counter-Strike Deathmatch with Large-Scale Behavioural Cloning.

Some highlights:

The rewards are calculated as r = 1.0K − 0.5D − 0.02F, where K is a kill, D is own death, and F is a shot fired. One could interpret it as follows: 1) the agent must kill, 2) the agent must protect its own existence, as long as such protection does not conflict with the first rule, 3) the agent must spare ammunition, as long as it does not conflict with the first and the second rule.
"To determine when to stop training, we evaluated the agent after each epoch, measuring kills-per-minute"
"Kill/death ratio (K/D) is the number of times a player kills an enemy compared to how many times they die. Whilst useful as one measure of an agent’s performance, more information is needed – avoiding all but the most favourable firefights would score a high K/D ratio, but may be undesirable. We therefore also report kills-per-minute (KPM). A strong agent should have both a high KPM and high K/D"
"In this paper we take on such a challenge; building an agent for Counter-Strike: Global Offensive (CSGO), with no access to an API, and only modest compute resources (several GPUs and one game terminal)."
"Our solution uses behavioural cloning - training on a large noisy dataset scraped from human play on online servers..."

A video linked in the article:

From the article and the authors' affiliation, I drew the following conclusions:

It is likely that China is already working on fully autonomous weaponry
One can already build autonomous weaponry with very modest computational resources and publicly available data
Efficient autonomous weaponry of mass destruction doesn't require human-level intelligence. In the game environment, even the primitive agent described in the article can kill at the rate of 30 humans per hour, while fighting against skilled armed humans. Slightly more intelli

...
Intergenerational trauma impeding cooperative existential safety efforts
by Andrew_Critch — 2022-06-03T08:13:25.439Z

Epistemic status: personal judgements based on conversations with ~100 people aged 30+ who were worried about AI risk "before it was cool", and observing their effects on a generation of worried youth, at a variety of EA-adjacent and rationality-community-adjacent events.

Summary: There appears to be something like inter-generational trauma among people who think about AI x-risk — including some of the AI-focussed parts of the EA and rationality communities — which is

preventing the formation of valuable high-trust relationships with newcomers that could otherwise be helpful to humanity collectively making better decisions about AI, and
feeding the formation of small pockets of people with a highly adversarial stance towards the rest of the world (and each other).

[This post is also available on the EA Forum.]

Part 1 — The trauma of being ignored

You — or some of your close friends or colleagues — may have had the experience of fearing AI would eventually pose an existential risk to humanity, and trying to raise this as a concern to mainstream intellectuals and institutions, but being ignored or even scoffed at just for raising it. That sucked. It was not silly to think AI could be a risk to humanity. It can.

I, and around 100 people I know, have had this experience.

Experiences like this can easily lead to an attitude like “Screw those mainstream institutions, they don’t know anything and I can’t trust them.”

At least 30 people I've known personally have adopted that attitude in a big way, and I estimate many more. In the remainder of this post, I'd like to point out some ways this attitude can turn out to be a mistake.

Part 2 — Forgetting that humanity changes

Basically, as AI progresses, it becomes easier and easier to make the case that it could pose a risk to humanity's existence. When people didn’t listen about AI risks in the past, that happened under certain circumstances, with certain AI capabilities at the forefront and certain public discourse surrounding them. These circumstances have changed, and will continued to change. It may not be getting easier as fast as one would ideally like, but it is getting easier. Like the stock market, it may be hard to predict how and when things will change, but they will.

If one forgets this, one can easily adopt a stance like "mainstream institutions will nev...
Silliness
by lsusr — 2022-06-03T04:59:51.456Z

Before I saw Everything Everywhere All at Once all I knew was:

It's about a Chinese-American family that runs a laundromat.

It's the best movie ever.

Spoilers Ahead

Everything Everywhere All at Once is a silly movie.

What is silliness? Silliness is the opposite of seriousness. What is seriousness? Seriousness is the opposite of fun.

Does that mean silliness equals fun? Not quite. All silliness is fun but not all fun is silly. Silliness is a subset of play is a subset of fun.

I think silliness is a terminal value. Silliness is precious. Silliness must be protected.

Silliness is giving into harmless impulses. Silliness is non-consequentialist. Silliness doesn't have a point. Silliness is letting go of pretention and just having fun. Right. This. Instant.

XD

:P

😀 😃 😄 😁 😆 😅 😂 🤣 🥲 ☺️ 😊 😇 🙂 🙃 😉 😌 😍 🥰 😘 😗 😙 😚 😋 😛 😝 😜 🤪 🤨 🧐 🤓 😎 🥸 🤩 🥳 😏 😒 😞 😔 😟 😕 🙁 ☹️ 😣 😖 😫 😩 🥺 😢 😭 😤 😠 😡 🤬 🤯 😳 🥵 🥶 😱 😨 😰 😥 😓 🤗 🤔 🤭 🤫 🤥 😶 😐 😑 😬 🙄 😯 😦 😧 😮 😲 🥱 😴 🤤 😪 😵 🤐 🥴 🤢 🤮 🤧 😷 🤒 🤕 🤑 🤠 😈 👿 👹 👺 🤡 💩 👻 💀 ☠️ 👽 👾 🤖 🎃 😺 😸 😹 😻 😼 😽 🙀 😿 😾

🐶 🐱 🐭 🐹 🐰 🦊 🐻 🐼 ❄️ 🐨 🐯 🦁 🐮 🐷 🐽 🐸 🐵 🙈 🙉 🙊 🐒 🐔 🐧 🐦 🐤 🐣 🐥 🦆 🦅 🦉 🦇 🐺 🐗 🐴 🦄 🐝 🪱 🐛 🦋 🐌 🐞 🐜 🪰 🪲 🪳 🦟 🦗 🕷 🕸 🦂 🐢 🐍 🦎 🦖 🦕 🐙 🦑 🦐 🦞 🦀 🐡 🐠 🐟 🐬 🐳 🐋 🦈 🐊 🐅 🐆 🦓 🦍 🦧 🦣 🐘 🦛 🦏 🐪 🐫 🦒 🦘 🦬 🐃 🐂 🐄 🐎 🐖 🐏 🐑 🦙 🐐 🦌 🐕 🐩 🦮 🦺 🐈 ⬛ 🪶 🐓 🦃 🦤 🦚 🦜 🦢 🦩 🕊 🐇 🦝 🦨 🦡 🦫 🦦 🦥 🐁 🐀 🐿 🦔

Let's keep Less Wrong silly.

🫀
Another Calming Example
by jefftk — 2022-06-03T02:20:02.159Z

In reading about parenting I often feel like there's a bit too much
theory vs examples, so here's walking through a recent interaction
that others later commented went surprisingly well.

At a recent Tuesday Family Dinner, one of the kids (~6y) served
themself an absurd amount of pasta. They were told to put some of it
back, refused, and someone else put it back. They burst into tears
and completely fell apart. They were told they either needed to calm
down or leave the room, and they left. Lots of angry crying and
shouting from the other side of the house.

After waiting a bit to give them a chance to calm down some, I went to
see if they wanted to talk. I asked, and they said they did (if they
hadn't I would have turned around and gone back to the table). I sat
with them on the couch with a mindset of providing calm and patient
attention, and asked what had happened. They started to explain
through their sobs, but I told them that I couldn't understand and asked
if they could speak normally.

This isn't actually true: I'm generally pretty good at understanding
kids, even when they are crying pretty hard. I'm strongly opposed to
lying (to kids or anyone) in most circumstances, but this is one place
where I do make an exception. I pretend that I can't understand, ask
if they can speak normally, present myself as an eager listener, and
in response kids reliably pull themselves together. This has a strong calming
effect: something about no longer crying seems to filter back into
feelings not seeming so overwhelming.

In this case, they calmed down some, and explained that they were
upset because they had the amount of pasta they wanted and then people
took it away. We talked about what they didn't like about that and
they told me they were worried the pasta was going to run out and they
would still be hungry after dinner.

Personally, I think this is very unlikely to be why they fell apart
while at the table, but that doesn't actually matter! What's
important is that they've calmed down and put something into words:
once it's in words, we can work on it. I used a whispery voice to tell
them that I knew about some secret extra pasta, and that there was no
way we were going to run out. I asked if they wanted to sneak back
into the kitchen and see, which they were excited about. Together we
crept into the kitchen, as quietly as possible...
[MLSN #4]: Many New Interpretability Papers, Virtual Logit Matching, Rationalization Helps Robustness
by Dan H — 2022-06-03T01:20:44.875Z

As part of a larger community building effort, I am writing a safety newsletter which is designed to cover empirical safety research and be palatable to the broader machine learning research community. You can subscribe here, follow the newsletter on twitter here, or join the subreddit here.

Welcome to the 4th issue of the ML Safety Newsletter. In this edition, we cover:

How “model-based optimization” environments can be used to research proxy gaming
How models can express their uncertainty through natural language
A new plug-and-play state-of-the-art OOD detection technique
How “rationales” can improve robustness to adversarial attacks
Announcing our subreddit with safety papers added nightly
... and much more.

Alignment

Making Proxies Less Vulnerable

Functions without a smoothness prior can result in solutions that maximize a proxy but not the true objective.

A portion of “Model-based optimization” (MBO) research provides a way to study simple cases of proxy gaming: with MBO environments, we can learn how to build better proxies that yield better solutions when optimized.

MBO aims to design objects with desired properties, that is to find a new input that maximizes an objective. The objective is typically assumed to be expensive to evaluate and a black box. Since the black box objective is expensive to query, researchers are tasked with creating a proxy that can be queried repeatedly. An optimizer then finds an input that maximizes the proxy. To design proxies that yield better solutions according to the ground truth black-box objective, this paper incorporates a smoothness prior. As there are many mathematical details, see the paper for a full description. In short, model-based optimization environments can be used to empirically study how to create better, less gameable proxies.

[Paper]

[Video]

Other Alignment News

[Link]

Why we need biased AI -- How including cognitive and ethical machine biases can enhance AI systems: “a re-evaluation of the ethical significance of machine biases”

[Link]

Generating ethical analysis to moral quandaries

[Link 1]

[Link 2]

Examples of inverse scaling or anticorrelated capabilities: perceptual similarity performance does not monotonically increase with classification accuracy

[Link]

“comprehensive comparison of these provably safe RL methods”

[Link]

Inverse Reinforcement Learning Tutorial

[Link]

Single-Turn Debate Does Not Help Humans Answer H...