trentmkelly/LessWrong-43k huggingface dataset:
Interpretability Will Not Reliably Find Deceptive AI
by Neel Nanda — 2025-05-04T16:32:29.643Z

Disclaimer: Post written in a personal capacity. These are personal opinions and do not in any way represent my employer's views

TL;DR:

I do not think we will produce high reliability methods to evaluate or monitor the safety of superintelligent systems via current research paradigms, with interpretability or otherwise.

Interpretability still seems a valuable tool and remains worth investing in, as it will hopefully increase the reliability we can achieve.

However, interpretability should be viewed as part of an overall portfolio of defences: a layer in a defence-in-depth strategy

It is not the one thing that will save us, and it still won’t be enough for high reliability.

EDIT: This post was originally motivated by refuting the claim "interpretability is the only reliable path forward for detecting deception in advanced AI", but on closer reading this is a stronger claim than Dario's post explicitly makes. I stand by the actual contents of the post, but have edited the framing a bit, and also emphasised that I used to hold the position I am now critiquing, apologies for the mistake

Introduction

There’s a common argument made in AI safety discussions: it is important to work on interpretability research because it is a realistic path to high reliability safeguards on powerful systems - e.g. as argued in Dario Amodei’s recent “The Urgency of Interpretability”.[1] Sometimes an even stronger argument is made, that interpretability is the only realistic path to highly reliable safeguards - I used to believe both of these arguments myself. I now disagree with these arguments.

The conceptual reasoning is simple and compelling: a sufficiently sophisticated deceptive AI can say whatever we want to hear, perfectly mimicking aligned behavior externally. But faking its internal cognitive processes – its "thoughts" – seems much harder. Therefore, goes the argument, we must rely on interpretability to truly know if an AI is aligned.

I am concerned this line of reasoning represents an isolated demand for rigor. It correctly identifies the deep flaws in relying solely on external behavior (black-box methods) but implicitly assumes that interpretability doesn't suffer from equally fundamental problems. There are many deep issues in interpretability that prevent very confident conclusions, even if we assume models cannot deliberately obfuscate their thoughts, e.g. superpositi...
80 concepts on my new version of AI: DecisionBots
by Wes R — 2025-05-04T14:04:04.897Z

Intro

I've structured this document into a list of concepts for both my convenience ( instead of having to edit one big piece of text, I only have to edit a list of small pieces of text), and for your convenience—making it easier for you to reference specific concepts, share with others, integrate into your working materials,
or most importantly, take notes on specific concepts. (Feel free to steal this writing method for yourself, if you think it’s a good idea) enjoy!

Setup

One idea for an AI is: instead of an AI being trained on a typical reward function, it's trained to always choose the most moral option. To be clear, the AI is not trying to maximize the number of most-moral options it chooses. It is only trying to optimize for choosing the most moral option out of one single set of options: The set of options right in front of it.
For example: If the most moral option is “Buy a first-aide kit so, later, when people need first aid, there will be a robot there to help them”, then the AI will do that. What the AI won’t do is think “If I but a first-aide kit, then I can give 10 people a papercut at once (which is only one immoral decision), then I can make 10 moral decisions by giving each of them band-aides, 10-1=9, so on average I’m making moral decisions!”, since choosing to give 10 people a papercut would be immoral, so it wouldn’t do that once the AI is actually faced with that choice. It doesn’t care if it will get to make 10 moral decisions afterwards; that only softens the blow.
To make this extra clear: Imagine you thought you were only going to make one decision in your life, and then someone else would take over everything you do. Your only goal is to make sure you choose the most moral option out of the options you have. You're allowed to spend some time thinking and whatnot, and maybe you're given 30 minutes, and maybe you can search stuff up on the internet, but you're not allowed to do anything too big (e.g., leaving the house) to help you think, because that would constitute a decision. Once you make a decision (let's say you choose to leave the house), someone else is put in that exact same situation. It's like that, but instead of you, it's an AI. Does that clear things up?

Idea for an AI: “DecisionBot”

For every decision DecisionBot makes, DecisionBot takes in (as an input) the list of options it has, with information on each op

...
Where have all the tokens gone?
by braces — 2025-05-04T13:52:16.707Z

In 2001, Lant Pritchett asked, “

Where has all the education gone?

” From 1960 to 1990, countries around the world had achieved increases in educational attainment without the widely expected gains in income—a “micro macro paradox” where the person-level estimates of the return to education were nowhere to be seen in aggregate output.

It seems we are on the cusp of another paradox in developing countries. People have an oracle in their pocket and it hasn’t seemed to help.

Is this surprising? The core premises are:

Human capital—or intelligence more generally—causes economic growth
AI models can substitute for human capital and intelligence
AI models have seen widespread use
We have not witnessed large gains in growth in developing countries

Together, these cannot all be true. Let’s go through the premises. I consider several ways to reconcile things afterwards.

1. Human capital and intelligence cause growth

A canonical question in labor economics is whether education increases earnings. The literature is settled on Yes, with several natural experiments showing better economic outcomes among people exposed to

sudden policy changes

that increase years of schooling. Some clever studies isolate the impact of the education itself, not just the “sheepskin effect” of having a diploma, like

this paper

showing that a decrease in the required coursework for a large college major ultimately hurt the earnings of its graduates.

You can also give ideas to businesses and make them more productive. When plants in India were

provided free management consulting

, productivity

durably

increased. A “

personal initiative training

” increased profits of Togo entrepreneurs by 30%. A heuristics-based

financial literacy course

in the Dominican Republic increased firm revenue.

Much of the developing world works in agriculture, and here, too, experiments find that you can improve productivity by merely spreading the word about best practices. Advice on the importance of a key input increased the yields of Indonesian

seaweed farmers

.

Text reminders

about weeding and fertilizer boosted sugarcane yields by 11% in East Africa.

At the macro level, human capital is viewed as explaining differences in growth across countries. Here the evidence is necessarily worse, as you’re limited to a small sample. But in general,

cross-country

...
The Ukraine War and the Kill Market
by Martin Sustrik — 2025-05-04T07:50:03.872Z

Politico writes:

The [Ukrainian] program […] rewards soldiers with points if they upload videos proving their drones have hit Russian targets. It will soon be integrated with a new online marketplace called Brave 1 Market, which will allow troops to convert those points into new equipment for their units.

[...]

The program assigns points for each type of kill: 20 points for damaging and 40 for destroying a tank; up to 50 points for destroying a mobile rocket system, depending on the caliber; and six points for killing an enemy soldier.

[...]

Units will soon be able to use the special digital points they’ve been getting since last year by trading them in for new weapons. A Vampire drone, for example, costs 43 points. The drone, nicknamed Baba Yaga, or witch, is a large multi-rotor drone able to carry a 15-kilogram warhead. The Ukrainian government will pay for the drones that are ordered and will deliver them to the front-line unit within a week.

[...]

The scheme is aimed at directing more equipment to the most effective units. It will also help to bypass bureaucratic procurement procedures and buy weapons directly from manufacturers.

[...]

The ability to get points for killing enemy troops is also spurring competition among units; so far about 90 percent of the army's drone units have scored points. In fact, they are logging so many hits that the government has had to revamp the logistics of drone deliveries to get more of them to points-heavy units. “They started killing so quickly that Ukraine does not have time to deliver new drones,” Fedorov said.

Now, this is clearly a repugnant market. Repugnant market is a market where some people would like to engage in it and other people think they shouldn’t. (Think market in human kidneys. Or prostitution. Or the market in abortions. Also, watch a lecture about repugnant markets by Al Roth, who introduced the concept of repugnance into economics.)

Having acknowledged that, let’s put the feeling of disgust aside and try to think about the consequences of this new approach to war clearly.

First, let’s contemplate a bit on the raw, brutal efficiency of the markets. Cowen’s and Tabarrok’s course in microeconomics begins with a story that shows how market incentives beat the widely cherished moral and religious principles hands down:

Way back in 1787, the British government hired sea captains to ship convicted felons to Australia.

...
PSA: Before May 21 is a good time to sign up for cryonics
by AlexMennen — 2025-05-04T04:10:20.508Z

Cryonics Institute and Suspended Animation now have an arrangement where Suspended Animation will conduct a field cryopreservation before shipping the body to Cryonics Institute, thus decreasing tissue damage occuring in transit. They are raising their prices accordingly, but offering a discount from the new price for people who sign up by May 21 (and arrange funding within another 3 months). See https://cryonics.org/members/standby/suspended-animation-inc-standby-transport-services-option/ for details.

It is thus an especially good time to sign up cryonics if you intend to contract with Cryonics Institute plus Suspended Animation, and live in the United States. If you don't live in the US, don't intend to contract with CI, or intend to contract with CI but not also with SA, then this deadline doesn't mean anything for you, but, if you want to, you could still take this as impetus to get around to signing up.
GTFO of the Social Internet Before you Can't: The Miro & Yindi Story
by keltan — 2025-05-04T01:08:52.434Z

Recommended music to read this to (If you like ambience)

I

Yindi had sent him a link, "You've gotta see how this guy speedruns Mario Kart, I think you'll like it (✿◠‿◠)". Miro taps the link.

CREATE A NetMe™[1] ACCOUNT TO WATCH THIS VIDEO

Miro creates the account. The video is good.

He runs to boot his CRT, its electron beam lighting the quiet room with a high pitched scream. He starts his Wii and runs through Mushroom Gorge a few times before trying to replicate the technique. "Aim in between the two coins, shroom before the grass then, release your med turbo and left hop at the same time"

On his first attempt he gets close to making the shortcut, but hits the barrier and falls. On his second attempt Miro turbos too quickly and hits a wall. On his eighth attempt he gets it. Frustration gives way to sudden pride, and he messages Yindi, "Just made the jump. Thank you!" Miro decides to check out what NetMe is all about.

At work, he thanks Yindi again, and they talk for a long time about nothing in particular.

By spring, Miro uses NetMe every day, and while he feels so little, he feels so much. Each video he scrolls past micro-doses him with rage, or awe, or awwww, or lust. Miro doesn't experience any emotion too intensely.

Four hours a day.

This is how much time Miro spends on NetMe.

But if you asked Miro what he likes, he'd say something like:

Practicing speedruns
Playing video games with friends
Working at his job
And Yindi (But Miro won't tell anyone that. Especially not Yindi)

NetMe wouldn't enter his mind at all.

Miro is on NetMe now. He's watching a video that looks... weird? It's a girl. She is smiling, and dressed like Zelda, when she wears that one outfit that always made Miro feel suspicious as a kid. Dancing? No. Writhing. Sometimes her hands pass in front of her clothing, and meld into it. Other times her face morphs from one girl to another. Pretty to ugly, Caucasian to Asian.

He considers sending the video to Yindi, just because it's so weird. He flushes imagining Yindi questioning why the algorithms would show Miro that video. Miro scrolls past.

A new video, A dog.
A new video, A girl.
A new video, An accident.
A new video, A speedrun.
A new video, An animation.
A new video, 10 Facts About the Atari.
A new video, How to Cook salt n pepper Chickpeas.
A new video, A Girl Covered in Slime.
A new video, A Sports Clip.
A new video, A Spoiler for the new Big Movi...
"Superhuman" Isn't Well Specified
by JustisMills — 2025-05-03T23:42:11.964Z
Link: https://justismills.substack.com/p/superhuman-isnt-well-specified

Strength

In 1997, with Deep Blue’s defeat of Kasparov, computers surpassed human beings at chess. Other games have fallen in more recent years: Go, Shogi, and Othello[1] among them. AI is superhuman at these pursuits, and unassisted human beings will never catch up. The situation looks like this:[2]

At chess, AI is much better than the very best humans

The average serious chess player is pretty good (1500), the very best chess player is extremely good (2837), and the best AIs are way, way better (3700). Even Deep Blue’s estimated Elo is about 2850 - it remains competitive with the best humans alive.

A natural way to describe this situation is to say that AI is superhuman at chess. No matter how you slice it, that’s true.

For other activities, though, it’s a lot murkier. Take

radiology

, for example:

Graph derived from figure one of

CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning

, assuming a normal distribution of human skill given the average in the paper.

CheXNet is a model for detecting pneumonia. In 2017, when the paper was published, it was already better than most radiologists - of the four it was compared to in the study, it beat all but one. Is it superhuman? It’s certainly better than 99% of humans, since fewer than 1% of humans are radiologists, and it’s better than (about) 75% of those. If there were one savant radiologist who still marginally outperformed AI, but AI was better than every single other radiologist, would it be superhuman? How about if there was once such a radiologist, but he’s since dead, or retired?

We can call this the strength axis. AI can be better than the average human, better than the average expert, better than all but the very best human performers, or, like in the case of chess, better than everyone, full stop.

But that’s not the only axis.

Effort

The

ARC-AGI

benchmark is a series of visual logic puzzles that are easy (mostly) for humans, and difficult for AI. There’s a large cash prize for the first AI system that does well on them, but the winning system has to do well cheaply.

Why does this specification matter? Well, OpenAI’s industry leading o3

scored 87.5%

on the public version of the benchmark, blowing many other models out of the water. It did so, however, by spending about $3000 per puzzle. A human can solve them for about $5 each.

Talking in terms of money undersells the difference, thou...
Navigating burnout
by gw — 2025-05-03T22:07:35.424Z
Link: https://www.georgeyw.com/burnout/

Burnout. Burn out? Whatever, it sucks.

Burnout is a pretty confusing thing made harder by our naive reactions being things like “just try harder” or “grit your teeth and push through”, which usually happen to be exactly the wrong things to do. Burnout also isn’t really just one thing, it’s more like a collection of distinct problems that are clustered by similar symptoms.

Something something intro, research context, this is what I’ve learned / synthesized blah blah blah. Read on!

Models of burnout

These are models of burnout that I’ve found particularly useful, with the reminder that these are just models with all the caveats that that comes with.

Burnout as a mental injury

Researchers can be thought of as “mental athletes” who get “mental injuries” (such as burnout) the way physical athletes get physical injuries, and we should orient towards these mental injuries in the same way we orient towards physical injuries.

I think this is by far the most useful model, because it makes it easier to actually generate a degree of self-compassion that makes you stop slamming your face into the brick wall. It is much easier to be hard on yourself while burnt out that you’re not producing “enough” legible output than it is to be hard on yourself with a twisted ankle that you aren’t keeping up with your weekly cardio goals. The way in which running on a twisted ankle is self-evidently Bad is more obvious than the way in which pushing through yet another deadline is Bad.

I’m increasingly convinced over time of how deeply this physical analogy runs. Success as an athlete is as much about managing injuries and recovery as it is about optimizing your peak performance. Managing burnout is a core research skill as much as managing recovery is a core athletic skill.

(For rehab of a physical injury, you have “active recovery”, where you aren’t back to normal but it benefits the healing process to start using the injured part again (mobility, physical therapy, light workouts). There is probably some way to triangulate active recovery for burnout here, but my guess is most people’s first attempt at doing so will still end up way too “active” to be recovery.)

Burnout as a deficit of activation energy

I like this description of

willpower as “living money”

and burnout as being “bankrupt”. In general, tasks have a certain activation energy to them, and we have some capacity (that vari...
What is your favorite podcast?
by ChristianKl — 2025-05-03T21:25:20.837Z

I'm looking for podcast recommendations from the LessWrong community. If you have a favorite podcast, please share:

What is the podcast?
What specifically do you like about it?
What concepts or topics does it teach particularly well?
Why do you consider its content trustworthy or reliable?
Who would you recommend this podcast to? (e.g., target audience, specific interests)

Please keep each top-level answer limited to one podcast recommendation. This will allow others to vote on individual suggestions effectively.

If you want to recommend multiple podcasts, please create a separate top-level answer for each one.
Does translating a post with an LLM affect its rating?
by ReverendBayes — 2025-05-03T14:45:01.070Z

TL;DR: Recently I wrote a post that got much less karma than I expected. My best guess is that the main reason is that I translated it from Russian to English via ChatGPT, and this easily recognizable LLM-style convinced readers from the first lines that “There aren’t any original thoughts; it is a standard, machine-generated fluff piece”. Is it correct, or does the post itself contain any major issues?

More detailed thoughts:

I expected the ideas from this post to be quite non-trivial (the math is simple, but practical results are more or less interesting; I mean, even Bayes’ and Aumann’s theorems are very simple from the mathematical point of view, but they bring many useful insights).

When I read it in the mentioned book, I found it very interesting.
When I published the original Russian post in my Telegram channel, it became one of the most liked posts there.
It is not easy to see it from the text, but I added many personal insights of my own to the book’s ideas (most of them are “how exactly it goes from the math” and “why it is still working in real life”).

I expected - including after considering my previous post - a rating of about 10-20 (and if I were an author who has been writing for a long time and proved his posts as useful for community, maybe I could expect 50+). Rating of 5 (2 of which are default rating based on author’s karma) makes me feel very confused.
My main hypothesis is as follows: the user opened the post, read one or two paragraphs, noticed distinct LLM-generated style, understood that it’s just another empty machine-generated piece of text, closed the post.
Part of the effect may be due to my not adding any tags to the post. But I don’t expect it was the main reason.
Perhaps the post itself is not well written or lacks new ideas; but my evals of such effect are already contained in my expected 10-20 rating.

And now, more detailed questions:

Are my assumptions correct? More specifically, I have the following questions (depending on how much effort you are willing to put in):

If you see text, which is obviously LLM-written/translated, what is the probability that you would stop reading after one or two paragraphs (except in cases where there is quite a revelation within these paragraphs)?
How strong is feeling from first one or two paragraphs of this particular post, that “There is nothing interesting in here; I will barely update; it’s just ano

...
SimpleStories: A Better Synthetic Dataset and Tiny Models for Interpretability
by Lennart Finke — 2025-05-03T14:04:08.871Z

The dataset and model suite TinyStories is widely used as a toy setup for mech interp research. Searching for the term on this forum currently yields 23 results. "A List of 45+ Mech Interp Project Ideas from Apollo Research’s Interpretability Team" lists making an improved version of it as a desideratum, because the dataset is "very formulaic, small, and has unusual unicode characters in it".
The improved version, the SimpleStories dataset and model suite, is now out. We are looking forward to see what the interp community ends up doing with it, and hope that it is better than TinyStories in all interp use cases. For feedback, we keep a community doc.
What's up with AI's vision
by Joachim Bartosik — 2025-05-03T13:23:42.631Z

This week I've read 2 pieces of interesting information:

Apparently current models (in particular o3) are really good at GeoGuessr (Testing AI's GeoGuessr Genius),
Apparently they have very had time understanding what they see on screenshots from Pokemon Red, even when stuff is clearly marked (Research Notes: Running Claude 3.7, Gemini 2.5 Pro, and o3 on Pokémon Red).

This is a puzzling to me. I find it much easier to find staircase in Pokemon Red than to figure out where s photo as taken.

Reading about the first example from Scotts post. I guess there are not that many featureless plains in the world. But can it really tell types of grass growing there? But have issues identifying doors / staircases in Pokemon Red? Seems weird.

Do we know what's up with all that?