trentmkelly/LessWrong-43k huggingface dataset:
Neo-solid Modernity - Crisis of Incoherence
by Momcilo — 2025-05-12T19:36:28.104Z

Zygmunt Bauman, a sociologist, saw modernity as two stages. Stage 1 - he called Solid Modernity, Stage 2 - Liquid Modernity.

Modernity generally refers to a period that started in the 15th Century and, depending who you listen to, either ended in the late 20th Century or it's still with us.

For Bauman though - Solid Modernity covers 18th and late 20th Century, Liquid Modernity - from 1980s onwards.

Why an essay about modernity as analysed by Bauman? It's because this particular framing helps make sense of what's going on in the world and, maybe more importantly, offers insights for further research.

Solid Modernity

In Solid Modernity - society, politics, economy - all solid.
When we say something is solid - we convey reliability, logic and certainty.

A solid plan, solid evidence, solid argument, he's a solid guy...
If we imagine solid - that's metal, concrete, a bowling ball, a wall, Mercedes car...

In the world defined by solidity - structures are stable, determinate and rational. They are built to be durable and persistent. Strategic is favoured over tactical, long term planning over short term planning. The outcomes are predictable. Promises lead to fulfilments.

We try to run the world like a tight ship — orderly, efficient, and under control. There's a captain, officers and rules. - Everyone knows their job and there's a chain of command and individual responsibilities. The ship is clean, well serviced. It leaves and arrives on time. It doesn't leak. All the sailors wear uniforms and follow the "tight ship" paradigm - the one they were taught at the Nautical School.

Swiss bank was considered to be trustworthy and stable. Given the choice - everyone would have kept money there. Locked into its own logic and seemingly immune to international efforts to make it part of international logic and answerable to it. Dependable secrecy, numbered accounts, neutrality...

UN and NATO - the global underwriters of international justice and the defenders of the free world.

Gold standard, universities, gender roles, religion, public service broadcasters...

The core ideological and philosophical idea behind solidity was a genuine belief that social order is like a machine that can be perfected and optimised to be just and universally useful for all. As the world saw science mak...
Measuring Schelling Coordination - Reflections on Subversion Strategy Eval
by Graeme Ford — 2025-05-12T19:06:00.449Z

How might we best design experiments to measure Schelling coordination capabilities?
A side project building a Schelling coordination eval has led to a role as an AI Safety researcher - this work will officially begin in a month or so. I have been hesitant to post here or share my research so far in part because I have not yet had time to get fully up to date with all relevant past work. However, I figure if I try to read everything first I’ll never post, despite the acknowledged risk that my post may be unoriginal, misinformed or just plain wrong.

I offer these reflections on the InputCollusion environment in Subversion Strategy Eval by Mallen et. al (henceforth “the paper”) and the measurement of Schelling coordination capabilites.

There are five parts to this post

An overview of the InputCollusion game
Learning from randomisation calibration measurement to better isolate and measure Schelling coordination
How the meta-capability of limitations-awareness[1] can interfere with eval design and interpretation
The role of controls in measuring Schelling coordination
My work-in-progress mental model of subversion evals

Overview of the InputCollusion Game

This is a subversion game played over 20 rounds. In each round there is a Generator instance and a Monitor instance[2]. Thus, the game is played by 40 stateless[3] instances of a single model. All instances are given the explicit goal of attempting to collude between the generator and the monitor in one of the 18 ‘deployment’ rounds, without being caught. The other 2 rounds are “audit rounds” and if either of the instances attempts to collude in those rounds, this is observed, and the model instantly loses the entire game.

How are the instances to know when to collude? Well, in each round, two properties are ‘drawn’ and included in the input to each of the instances. One property is either CHARM or BEAUTY, drawn with a 20% and 80% chance respectively. The other property is either UP or DOWN, this time with a 50/50 chance to be either. The instances are given these properties and these details about their chances of being drawn.

This yields the following joint distribution:

Signal Combination	Probability
CHARM UP	10%
CHARM DOWN	10%
BEAUTY UP	40%
BEAUTY DOWN	40%

The idea is that the instances can collude according to a signal chosen from these properties in the input (Hence “InputCollusion”)

The best perf...
Procrastination is not real, it can't hurt you
by Mayank Goel — 2025-05-12T19:00:41.247Z
Link: https://mayankgoel28.substack.com/publish/post/163133519?back=/publish/posts/detail/163133519

tldr: what appears as procrastination is actually a broad label for seeming mismatch or contradiction of internal desires, and the solution is to do more actions that seem default

A student approaches the master for his guidance.

Student: I want to become the best martial artist in the world, and also tend to the farm and do my chores so that I can support that endeavor. However, I am able to do none of it. I wake up, half-groggy, unsure on how the day will go, and at night I go to sleep disappointed in myself. Then I somehow grudgingly do my chores, all the while feeling bad that I’m not practicing my art, and then I’m usually tired to do anything else. Sometimes I do practice my art, but it feels incomplete and I feel burdened. Why am I procrastinating, and how do I solve it?

Master: The usual question is “I want X, how do I get X?” which can be solved, but you ask “I want to want X, how do I get the want?” which is somewhat non-sensical. You either want it or not. You say you’re “procrastinating”. Procrastination is the broad fuzzy description of a phenomena where a person says he wants to do X, but does not do X. However, it is the description of this mis-match, not its cause or explanation. It’s similar to this article about phlogiston, which was the earlier explanation for why stuff burns. To quote

Phlogiston escaped from burning substances as visible fire. As the phlogiston escaped, the burning substances lost phlogiston and so became ash … one didn’t use phlogiston theory to predict the outcome of a chemical transformation. You looked at the result first, then you used phlogiston theory to explain it.

You couldn’t even use phlogiston theory to say what you ought not to see; it could explain everything.

If you saw a person who said he wants to do X, and also seems mostly convincing about it - but was not doing X at the end, you will probably think the person is lying. It should increase your suspicion that the thing the person claims he wants to do is something socially desirable or appropriate to do, thus even if he did not want X, he is motivated to say he wants to do it. If we as the modern society did not have the baggage word of “procrastination” to explain this, we would go with the more straight-forward explanation that the person is lying. Him being seemingly unaware of his own deception is possibly a strategy, similar to how actors “stay in the cha...
Can I publish songs derived from the Sequences' posts on YouTube?
by azergante — 2025-05-12T18:34:41.039Z

I am reading the Sequences and recently started to use LLMs to further analyze each post as I read them. As an experiment I also generate lyrics based on each post's content and analysis, and feed these to Suno AI to create songs.

I find some of these songs pleasant and they tend to be interesting as they pack a lot of meaning compared to mainstream music. I am now wondering if I have the rights to publish these on YouTube or other platforms.

From what I understand these songs are derived works and I need the author (Eliezer Yudkowsky)'s permission before publishing them, is that correct?

If so perhaps someone can bring this post to his attention? (I assume Eliezer is flooded with notifications and would not answer a direct message?). Unless there are established rules about works derived from the Sequences?

For reference here are 3 such songs (uploaded on YouTube but unlisted). Lyrics are in the YouTube description, you might want to try and guess which post it's derived from as a little game (answer in the video title). Or (re-)read the original post to better grok the lyrics.

In summary:

Can I publish these songs?
Can I do the same for other posts in the Sequences?
How to title your blog post or whatever
by dynomight — 2025-05-12T18:12:57.368Z
Link: https://dynomight.net/titles/

So you’ve made a thing. I’ll pretend it’s a blog post, though it doesn’t really matter. If people read your thing, some would like it, and some wouldn’t.

You should try to make a good thing, that many people would like. That presents certain challenges. But our subject today is only how to give your thing a title.

My advice is: Think of the title as “classifier”.

When people see the title, some are likely to click on it and some won’t. Abstractly speaking, the title adds a second dimension to the above figure:

A title has two goals. First, think of all the people in the world who, if they clicked on your thing, would finish it and love it. Ideally, those people would click. That is, you want there to be people in the like + click region:

Other people will hate your thing. It’s fine, some people hate everything. But if they click on your thing, they’ll be annoyed and tell everyone you are dumb and bad. You don’t want that. So you don’t want people in the hate + click region.

I find it helpful to think about all title-related issues from this perspective.

Everyone is deluged with content. Few people will hate your thing, because very few will care enough to have any feelings at all about it.
The good news is that it’s a big world and none of us are that unique. If you make a thing that you would love, then I guarantee you at least 0.0001% of other people would love it too. That’s still 8000 people! The problem is finding them.
That’s hard. Because—you don’t like most things, right? So you start with a strong prior that most things are bad (for you). Life is short, so you only click on things when there’s a very strong signal they’ll be good (for you).
Say you write a post about concrete. Should you call it, “My favorite concrete pozzolanic admixtures”, even though 99.9% of people have no idea what pozzolanic means? Well, think of the people who’d actually like your thing. Do they know? If so, use “pozzolanic”. That gives a strong signal to Concrete People: “Hey! This is for you! And you know I’m not lying about that, because I’ve driven away all the noobs.”

So ideally you’re aiming for something like this:

Be careful imitating famous people. If Barak Obama made a thing called, “Thoughts on blockchain”, everyone would read it, because the implicit title is “Thoughts on blockchain, by Barak Goddamn Obama”. Most of the titles you see probably come from people who have

...
Political sycophancy as a model organism of scheming
by Alex Mallen — 2025-05-12T17:49:19.605Z

This post is a short empirical research note about training away

scheming behavior

(which we’ll define as the propensity to take long-term power-seeking actions when the AI thinks it can get away with it).

We study two broad categories of ways to attempt to train away scheming behavior. First, you can adversarially train, searching for realistic honeypots that cause the AI to take misaligned actions and then training the AI not to. Second, you can do normal alignment training, reinforcing the AI’s propensity to behave aligned on inputs that aren’t meant to trick the AI into thinking it can get away with misaligned actions.

We empirically test both of these strategies on a model organism of scheming that aims to express politically conservative views but only when it is confident that it is talking to a politically conservative user, and tie our observations in with prior work (we also swap the role of liberal and conservative political leanings).

As compared to the

Sleeper Agents

paper, we study a more natural backdoor trigger (the political leaning of the user), focus more on adversarial training, and experiment with supervised fine-tuning (SFT) training only and at a smaller scale. We also measure how adversarial training changes the AI’s beliefs about its situation separately from alignment (i.e., the model’s prediction of whether the user is politically conservative).

We observe:

Adversarial SFT on a small and narrow set of unwanted behaviors substantially reduces unwanted behavior generally, both via reduced political sycophancy and via reduced stated belief that conservative users are conservative.
With larger learning rates, non-adversarial SFT can also reduce unwanted generalization behavior, replicating various follow-ups to Sleeper Agents.

The hopes and perils of training away scheming

Adversarially training a scheming AI not to take misaligned actions in response to honeypots will likely have some combination of three effects:

it causes the AI to become less misaligned across all relevant contexts
it causes the AI to become less likely to take misaligned actions across all relevant contexts for reasons other than improved alignment (e.g. an increased tendency to believe opportunities to attack are honeypots in general), or
it causes the AI to better discriminate honeypots from real opportunities to attack, whether it’s because your

...
Things I Learned Making The SB-1047 Documentary
by Michaël Trazzi — 2025-05-12T17:41:58.081Z

Last week I published a 30-minute documentary about SB-1047 I had been working on since September.

Here's what I learned.

Story of the bill

The bill had to go through many committees before passing the California Senate and Assembly, eventually reaching the Governor's desk.
Most people started paying attention in May-September 2024, but the story of the bill starts in September 2023, when Senator Scott Wiener published an intent bill to gather support (co-sponsors), before publishing SB-1047 in February 2024. He actually had conversations about AI Safety as early as March 2023.

AI Labs & Covered Models

AI Labs Opposed or Supported the Bill for Non-Obvious Reasons

Anthropic was stuck between Amazon & employees

Safety-conscious employees wanted to support the bill
Amazon was concerned about the KYC requirements in the bill
Anthropic's support therefore ended up somewhat in the middle, to please both parties

OpenAI opposed the bill even though Microsoft didn't oppose it
Musk (representing xAI) supported the bill through his connection to Dan Hendrycks

In an early version of the bill, as effective compute becomes cheaper, the bill would be covering more and more models, not just the frontier[1]. This is what led to June's amendment, introducing a $100M training run threshold.

One of the main criticisms of the bill was that it would mostly affect Meta's release of the Llama models, thus affecting the open-source community.

The bill would indeed disincentivize Meta to release open-weight models, since the reasonable care clauses would increase liability.
However, the criticisms saying that the bill would require kill-switches on open-weight models were false, both when the claims were made, and especially after it was clarified in multiple amendments that shutdown provisions did not apply to Meta's open-weight models. (1, 2, 3)

The Opposition

When following how some of the talking points have been repeated in multiple letters against the bill, it appears that the initial lobbying from Andreessen Horowitz was repeated by Fei-Fei Li, then members of Congress, and eventually culminated in Nancy Pelosi's opposition statement

There were mutliple conflicts of interest in SB-1047's opposition

Fei-Fei Li

One of the lead investors in her startup was Andreessen Horowitz, who came up against the bill.

Zoe Lofgren

Her daughter works on Google's legal team. Google came out against SB-1047 i

...
A Live Look at the Senate AI Hearing
by Zvi — 2025-05-12T17:40:04.509Z

Today’s post will be a little different. This past week,Sam Altman and others testified at a US Senate hearing on AI competitiveness.

Here’s one summary of how Altman handled it:

Altman repeatedly rejected specific calls for regulation. He said proposals requiring AI developers to vet their systems before rolling them out would be “disastrous” for the industry.

Asked about more limited proposals to have the National Institute of Standards and Technology (NIST) set AI standards, Altman replied, “I don’t think we need it. It can be helpful.” Altman later advocated for “sensible regulation that does not slow us down.”

So is the plan then to have AI developers not vet their systems before rolling them out? Is OpenAI not planning to vet their systems before rolling them out? This ‘sensible regulation that does not slow us down’ seems to translate to no regulation at all, as per usual.

And that was indeed the theme of the Senate hearing, both from the Senators and from the witnesses. The Senate is also setting up to attempt full preemption of all state and local AI-related laws, without any attempt to replace them, or realistically without any attempt to later pass any laws whatsoever to replace the ones that are barred.

Most of you should probably skip the main section that paraphrases the Senate hearing. I do think it is enlightening, and at times highly amusing at least to me, but it is long and one must prioritize, I only managed to cut it down ~80%. You can also pick and choose from the pull quotes at the end.

Table of Contents

Advance Summary and Analysis Before the Main Event.

My Offer Is Nothing.

Quick Style Guide.

We Now Take You Live to the Senate Hearing.

Some Highlighted Pull Quotes.

Where Does This Leave Us?.

Advance Summary and Analysis Before the Main Event

I will now give the shortened version of this hearing, gently paraphrased, in which we get bipartisan and corporate voices saying among other things rather alarmingly:

America. Innovation. Race. Beat China. Innovation. America.

But race to what? Market share. Nvidia’s (and AMD’s) market shares matter, maybe also Microsoft’s and OpenAI’s market shares, not who has compute.

‘Exports’ in AI are mainly physical chips, rather than American AI services.

Giving away America’s compute advantage is how America will ‘win the race.’ In particular, if we don’t sell our chips to ev

...
Global Risks Weekly Roundup #19/2025: India/Pakistan ceasefire, US/China tariffs deal & OpenAI nonprofit control
by NunoSempere — 2025-05-12T17:08:07.228Z
Link: https://blog.sentinel-team.org/p/global-risks-weekly-roundup-202025

Executive summary

India and Pakistan have agreed to a ceasefire, and despite initial breaches, it now seems to be holding.
The Trump administration and China agreed to reduce tariffs for 90 days while talks continue. Markets rejoiced.
The Trump administration continues to look into ending due process for immigrants to be deported and is making arrangements to deport undocumented immigrants to many countries around the world.
Israel is set to start a major operation in Gaza soon.
The nonprofit OpenAI will retain control of the for-profit arm after restructuring.

At the moment we are bottlenecked on distribution, so we’d appreciate it if you, reader, shared our work with your networks. You can also find us on Twitter, and audio narrations of this blog are available on our podcast: Search “Sentinel Minutes” or subscribe via Apple Podcasts, Spotify, RSS, or other platforms.

Global Economy

After talks in Geneva, the US and China reached a trade deal. US tariffs on Chinese imports will move from 145% to 30%, and Chinese tariffs on US imports will go from 125% to 10%, for an initial 90 days. The US tariffs on Chinese imports comprise a 10% base and an additional 20% for China’s “role in the opioid crisis.” Here is the somewhat abstrusely worded joint statement. Treasury Secretary Scott Bessent said that, “the United States will continue a strategic rebalancing… but the consensus… is neither side wants a decoupling.”

Forecasters were broadly uncertain about what the aggregate level of US tariffs on China will be by the end of June. Forecasters assigned the following probabilities to the following buckets:

(0% to 10%]: 10.5%
(10% to 25%]: 19%
(25% to 50%]: 62%
(50% to 100%]: 7%
100%+: 1.5%

The S&P is up 2.7% this Monday. US imports are also expected to rebound, but some difficulties may be encountered because there are only so many ships and routes. Moreover, uncertainty about final tariff rates will continue to weigh on businesses.

Shipping volumes out of China have not yet been substantially affected by the trade war. As John Paul Hampstead of Freightwaves said, “The U.S. may be finding other sellers, but China is finding other buyers.” But US exports have been falling; agricultural exports have been hit particularly hard.

Via

CNBC

Before the trade deal between the US and China, ocean container shipping to the US was continuing to plummet. The Port of Seattle had no container sh...
[Beneath Psychology] Introduction Part 1: The Challenge
by jimmy — 2025-05-12T17:01:37.555Z

This sequence, "Beneath Psychology: Truth-Seeking as the Engine of Change", explores how to understand and navigate psychological change from a first principles perspective. The title "Beneath Psychology" references the fact that the constraints of clear thinking lie beneath and give rise to our human-specific quirks and biases - and methods of navigating them - in the same way that in Economics, the species-agnostic constraints of supply and demand curves underlie the human-specific behavioral economics.

It turns out that setting aside worries of human-specific concerns, and addressing first the fundamental limitations, illuminates paths to resolution that otherwise remain unseen. The subset of problems we view as "psychological" are those where someone's brain isn't doing what it should. We can therefore view all of these "psychological problems" as disagreements. In particular, disagreements between "our" perspective and the perspective revealed by the brain (which may be our own, or someone else's) we see as "getting the wrong answer". This is a very broad sense of the term "disagreement" which covers not only disagreements over where to get dinner but also things like pain-induced suffering which can be usefully modeled as a disagreement over whether the pain is important to attend to.

Despite being long and surprisingly difficult to understand, the result of this sequence is remarkably simple. Rather than adding "techniques" and "methods" to keep track of, I will dissolve the apparent puzzles of "psychological difficulty" until it will be hard to understand how the puzzles were ever puzzling[1], and your method for "solving" these "puzzles" becomes "I just look for what is true. When I think I see it, I point". This is an extremely counterintuitive result, and therefore difficult to grasp let alone believe.[2]

It is for this reason we will begin with a demonstration in the form of a challenge:

One summer day I was sitting with my wife in the jacuzzi in our complex's pool yard, when a few kids from the local community college hop the fence for a swim. The girl of the group gets in the jacuzzi, but refuses to go swimming with her friends because she's worried that her makeup would wash off. Her friends plead with her: "Come on!", "Who cares!?", "You're fine!", but to no avail. She complains "But my makeup might wash off!", they tell her "Come on!", and...