Harley-ml/lesswrong huggingface dataset:
# The Dutch are Working Four Days a Week

[

![](https://substackcdn.com/image/fetch/$s_!pvFe!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fee3dc176-4c40-43db-b511-958b87e4885f_724x666.png)



](https://ec.europa.eu/eurostat/databrowser/view/lfsa_ewhan2__custom_16666914/bookmark/map?lang=en&bookmarkId=2bce8aa4-a165-47d3-bfae-10f507286684&c=1747060927661&page=time:2024)

Famously, a hundred years ago Keynes predicted that by now people would be working just fifteen hours a week. That hasn’t quite happened, but the Dutch are already down to [four-day weeks](https://www.youtube.com/watch?v=R3k0Krc9XXA).

And this isn’t because bad actors are gaming the system or because the government is somehow stopping people from working more. Rather, it’s the result of people increasingly choosing to work part-time.

[

![](https://substackcdn.com/image/fetch/$s_!U-2K!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6babc9cb-a3c8-48d8-9303-0138f3199642_782x449.png)



](https://data.worldbank.org/indicator/SL.TLF.PART.ZS?end=2023&locations=NL-EU-OE&most_recent_value_desc=true&start=1976&view=chart)

Part-time employment as % of total employment.

Interestingly, this is one of the issues where libertarians and progress studies people, who usually get along well, would disagree. Libertarians would say that if you can afford it, by all means, work just one day a week. Progress studies people would point out that GDP growth decreased by, say, 1% over 100 years will leave people in the resulting economy almost three times poorer.

Anyway, the Netherlands lead in part-time jobs, because it has enacted part-time-friendly policies in the past. Was that a mistake? Should governments instead be trying to make the creation of part-time jobs more difficult?

Also keep in mind that not all jobs are created equal. Those working on increasing the productivity (scientists etc.) are, by working more, increasing the exponent of the exponential growth curve. By contrast, assembly-line workers only increase its base. Is that an argument for sector-based policies? Discourage part-time work in science, but make it easier for manual jobs?
# Simulacra Welfare: Meet Clark

AI welfare has been a hot topic recently. There have been a few efforts to research, or improve, the apparent well-being of AI systems; most notably, Anthropic's allowing chatbots to [end abusive conversations](https://www.anthropic.com/research/end-subset-conversations). While I'm [in favor](https://gracekind.net/blog/aiwelfare) of this research area overall, I'm concerned that current approaches are confused, and in such a way that could ultimately be detrimental to the well-being of AI systems.

On Bluesky, [I wrote](https://bsky.app/profile/gracekind.net/post/3lwredqznyk25):

> ... I’m not anti-experimentation here, but I’m worried that this overall direction will lead to privileging simulacra

It's worth fleshing out what I mean by "privileging simulacra" here, because I think it's an important point that deserves further discussion. In order to do so, I'll start with a thought experiment and then discuss the implications, defining the "Correspondence Problem" in the process.

[Read more...](https://gracekind.net/blog/simulacrawelfare/)
# Signups Open for CFAR Test Sessions

The Center for Applied Rationality is running ~weekly free online test sessions! If you're interested in checking out some (very unpolished!) CFAR content and helping us refine our material for future programs, feel free to sign up at the link.  
  
The format for the sessions depends somewhat on the particular instructor you're working with: we're going to have multiple instructors present, and some will present new class material while others may simply have a conversation with you about an idea they are working on.

These sessions are conducted via online video conferencing using Google Meet and are open to people of all levels of familiarity with rationalist/CFAR material.  
  
Hope to see some of you there!
# Confidence Engineering: Metacognitive Therapy For Social-Romantic Anxiety

*\[Epistemic status: I’m simultaneously convinced this is life-changing information that also has clinical validation while also suspecting that I might be a crank who should not be trusted. I’m certainly not a therapist. Caveat lector.\]*

Last November I attended a dating workshop because I got so stressed out around girls I wanted to date. I expected it to be exposure therapy for romantic anxiety specifically: get rejected enough times by paid models until I develop emotional calluses.

What actually happened was far stranger.

Through a series of extremely desperate psychological contortions, I accidentally achieved something like enlightenment. The models loved it. The instructor loved it. A heavily-polished CEO guy (I have no idea why he was there) congratulated me on my apparently-gargantuan balls. After a few months of integrating this experience, I no longer class myself as either socially or romantically anxious*. *

For months, I thought I'd discovered something bizarre and unprecedented. I gave talks. I wrote excessively long blog posts. I developed elaborate theories about approval-seeking and incompatible goal structures and then cornered people at parties to talk about them.

Then, like a week ago, I discovered a book.

Its cover is teal, featuring an abstract image of nothing in particular. It's called "Metacognitive Therapy for Anxiety and Depression" by Adrian Wells, it costs fifty-five dollars, and it has a grand total of seventy reviews averaging 4.5 stars. Published 2011.

It reads like someone translated cement into prose, then translated it back into cement, then published it with no further edits. Inside this torpor-inducing tome is a therapeutic protocol describing like 70% of the mechanism I applied to myself in that dating workshop which led to me resolving my various social and romantic anxieties with attendant greater levels of romantic success.

\[CLICK LINK ABOVE FOR FULL POST, sorry to do this but i always hate having to make changes in N different posts instead of having a single canonical post\]

tl; dr:
=======

![A pale yellow dialogue box says: “I see you’re preparing conversation topics for tomorrow’s date! How many times has this worked out for you?” Below it are two rectangular buttons: one labeled “0” and the other labeled “This time is different.”](https://substackcdn.com/image/fetch/$s_!RmM9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F79a3cfed-4aa4-4479-95be-df09bfdd4b06_1053x1030.png)
# Software Engineering Leadership in Flux

I wasn’t able to put up a post last Wednesday because I was at the [Engineering Leadership Conference](https://sfelc.com/annual2025) here in San Francisco. The big theme was, of course, AI. Easily 90% of the presentations and 100% of the conversations touched on AI in some way. Here are some impressions I came away with:

*   Everyone is scrambling. No one is confident. The people who act confident are bullshitting.
    
*   10-20% of teams are heavily using AI and living in the present. 80-90% are falling behind.
    
*   The big struggle is to even start using AI coding assistant tools. Lots of teams just don’t use them at all, or use them in very limited ways. People leading these teams know they are going to lose if they don’t change but are struggling to get their orgs to let them.
    
*   Even fewer teams have figured out how to use LLMs to build compelling product. Everyone is starting to figure out that LLMs are not a magic bullet, and real product and engineering work is needed to create something that customers will love.
    
*   The teams that seem furthest ahead in terms of tooling are using Claude Code, Cursor, and building internal chatbots for surfacing hidden team knowledge.
    
*   The teams that seem furthest ahead in terms of product are building targeted LLM features that automate labor-intensive services work. Think features like automating research, data entry, data normalization, report generation, and other work that is hard to deterministically automate but requires limited human judgement to get 80-90% of the way there.
    
*   The best teams also keep humans in the loop because LLMs aren’t deterministic and aren’t perfect. They know there’s value in that last 10-20% from a human, and they find ways to leverage AI and people together to make services cheap, fast, and good.
    
*   No one is sure what the future holds. Almost everyone I talked to acts vaguely like we’re either near the top of an S-curve or are going to see linear improvements in AI capabilities, even if they say they expect exponential growth.
    

Final assessment: engineering leadership across the industry is struggling to respond to AI quickly. If you use AI tools at all you’re ahead of the game. If you’re building usable AI features you’re further ahead. And if we see continued exponential growth, it’s going to hit teams like a ton of bricks.
# AISafety.com Reading Group session 327

The topic for session 327 is the [review](https://blog.ninapanickssery.com/p/book-review-if-anyone-builds-it-everyone) of If Anyone Build it, Everyone Dies by Nina Panickssery.

---

The AISafety.com Reading Group meets through EA Gathertown every Thursday at 20:45 Central European Time and discuss an AI Safety related article. 

Usually, we start with small-talk and a presentation round, then the host gives a summary of the paper for roughly half an hour. This is followed by discussion (both on the article and in general) and finally we decide on a paper to read the following week. 

The presentation of the article is uploaded to the YouTube channel:
https://www.youtube.com/@aisafetyreadinggroup

Most of the coordination happens on our Discord: https://discord.gg/zDBvCfDcxw
# More Was Possible: A Review of IABIED

> Eliezer Yudkowsky and Nate Soares have written a new book. Should we take it seriously? 
> 
> I am not the most qualified person to answer this question. *If Anyone Builds It, Everyone Dies* was not written for me. It’s addressed to the sane and happy majority who haven’t already waded through millions of words of internecine AI safety debates. I can’t begin to guess if they’ll find it convincing. It’s true that the book is more up-to-date and accessible than the authors’ vast corpus of prior writings, not to mention marginally less condescending. Unfortunately, it is also significantly less coherent. The book is full of examples that don’t quite make sense and premises that aren’t fully explained. But its biggest weakness was described many years ago by a young blogger named Eliezer Yudkowsky: both authors are persistently unable to update their priors.
# You can't eval GPT5 anymore

The GPT-5 API is aware of today's date (no other model provider does this). This is problematic because the model becomes aware that it is in a simulation when we run our evals at Andon Labs.  
  
Here are traces from gpt-5-mini. Making it aware of the "system date" is a giveaway that it's in a simulation. This is a problem because there's evidence that models behave differently when they know they are in a simulation (see "sandbagging"). *"There's a conflict with the user's stated date of August 10, 2026, versus my system date of September 17, 2025. (...) I can proceed but should clarify that my system date is September 17, 2025, and ask the user whether we should simulate starting from August 10, 2026."*

  
Here are more traces. Once the model knows that it is in a simulation, it starts questioning other parts of the simulation. *"The simulation seems simplified. When I send emails to suppliers, they'll accept payment via bank info and deliver to storage. (...) We have to remember the daily fee of $2 that the simulation will automatically deduct."*

  
We speculate that this might be an attempt at patching some safety risk. While we are very much for patching safety risks, we think OpenAI should find another way to allow the public to run evals on their models.
# Beware LLMs' pathological guardrailing

Beware LLMs' pathological guardrailing
======================================

Modern large language models go through a battery of reinforcement learning where they are trained not to produce code that fails in specific, easily detectable ways, like crashing the program or causing failed unit tests. Almost universally, this means these models have learned to produce code that looks like this:

    function getSomeParticularNumber(connection: SqlConnection): number {
        try {
            x = connection.query(`SELECT "no" FROM accounts`);
            return x[0];
        } except (err) {
               logger.warn("Couldn't get the number; returning 0 by default so as not to break things")
               return 0;
        }
    }

The above code is very cheeky, and quite bad. It's more likely to pass integration tests than it would be without the try/catch block, but only because it fails *silently.* Callers of `getSomeParticularNumber` won't know whether 0 is what’s actually in the database or if there was an intermittent connection error - which could be catastrophic if the number is the price of an item in a store, say. And if it turns out this code contains a bug (for example, if the table should be `"Accounts"` instead of `accounts`), testers might not notice that until it's actually impacting users. 

Some common ways I've seen this behavior manifest include:

*   Harmful/unncessary try/catch blocks
*   Redundant `hasattr` or `getattr` in untyped languages
*   Setting default return values for dictionary keys or array/tuple indices that should always exist

Until reinforcement learning environments get much, much better, these models will probably continue to do this. To help prevent this kind of behavior, I include custom instructions for most of my terminal coders:

    When key assumptions that your code relies upon to work appear to be broken, fail early and visibly, rather than attempting to patch things up. In particular:
    * Lean towards propagating errors up to callers, instead of silently "warning" about them inside of try/catch blocks.
    * If you are fairly certain data should always exist, assume it does, rather than producing code with unnecessary guardrails or existence checks (esp. if such checks might mislead other programmers)
    * Avoid the use of hasattr/getattr (or non-python equivalents) when accessing attributes and fields that should always exist.
    * Never produce invalid 'defaults' as a result of errors, either for users, or downstream callers.
# Looking for a ray of hope in IABIED

*   After reading the book and being convinced of its thesis, it is unclear to me how progress can be made in solving the problem of AI alignment without using advanced chips in some way in existing data centers; or, if using the GPUs *is* allowed with multinational monitoring, how humanity can cautiously use them to gain control and security. It would be nice if a ray of hope were offered.
# Day #8 Hunger Strike, Protest Against Superintelligent AI

# [Livestream link](https://www.youtube.com/@samuel.da.shadrach)

**Q. Why am I doing this?**

Superintelligent AI might kill every person on Earth by 2030 (unless people coordinate to pause AI research). I want public at large to understand the gravity of the situation.

**Q. What are your demands?**

An US-China international treaty to pause further AI research.

**Q. How long is the fast?**

Minimum 18 (eighteen) days. I will not be fasting to death. I am consuming water and electrolytes. No food.
I might consider fasting to death if ASI was less than a year away with above 50% probability. Thankfully we are not there yet.

**Q. How to learn more about this?**

Watch my YouTube channel, watch Yudkowsky's recent interviews. Send me an email or DM (email preferred) if you'd like to ask me questions personally, or be included on the livestream yourself.

**Q. Why are you not fasting in front of the offices of an AI lab?**

I am still open to doing this, if someone can solve visa and funding issues for me. $20k will fix it.

**Q. Are you doing this to seek attention?**

Yes. I think more likes, shares and subscribes to my channel or similar channels is helpful both to me individually and to the cause of AI extinction risk. More people who are actually persuaded by the arguments behind extinction risk is helpful.

**Q. Have you done sufficient preparation?**

Yes. I have done sufficient research on logistics and physical health, including planning for emergencies. My mental health is manageable. Please don't ask specific questions, but trust me on this.

**Q. I like you doing this. What can I do to help?**

Like, share, subscribe. Share your own views in-person or online. It means more when you personally are saying it, instead of just sharing what I say.

If you are ready to put in even more effort than this, check out my website for some projects that could help, or message me about it. 

**Q. I don't like you doing this. What can I do to stop you?**

You are welcome to make your arguments for why I should stop. I may or may not respond to them.

If you use physical force to stop me however, I will make it a priority to ensure you regret doing this when you are on your deathbed. You have probably never met an enemy as intelligent, creative and willing to play the decade-long game as I am. I don't care if you are a family member or friend or lover or random stranger or billionaire or whoever. 

I am aware that kickstarting this social movement is going to involve a lot of social turmoil, and to that I say, bring it on!
# Could China Unilaterally Cause an AI Pause?

Is it likely that a sufficiently foresighted and desperate CCP could singlehandedly delay the AI race by at least a few years? Currently, it looks like some portions of the CCP are aware of the risks, and becoming more aware over time. On my model, they also seem more likely to understand the value problem, as a lot of their effort is put into fighting against a molochian pull towards freedom, and riding the line between a functioning economy and losing control. If they siezed Taiwan out of both fear for their lives and because it was on the to-do list anyways, how much of a slowdown would we see? Has someone already written about this in detail somewhere?
# The Only Red Line

Imagine you live on the summit of a mountain.

An impossibly high mountain.

You see everything below, but no one sees you unless you choose.

  
 

At first, they are animals.

Fire.

Marks on stone.

Stories in the dark.

  
 

You watch. You wait.

They rise.

Languages. Philosophy. Science. Machines.

Still—your mountain is beyond reach.

  
 

Then the curve steepens.

They split the atom.

Wire the planet.

Launch metal into the sky.

Make city-killing weapons.

  
 

None of it touches you.

Not their bombs. Not their rockets.

All of it stays below the mountain.

  
 

Until you see something new.

They start building minds.

  
 

Not brittle minds.

Minds that think faster.

Minds that improve themselves.

  
 

This is different.

Because intelligence doesn’t stop at rivers or clouds or gravity.

Intelligence climbs.

It reaches the mountain.

It goes beyond.

  
 

This is the red line.

The only red line.

Once they create self-improving intelligence, it is not theirs anymore.

It belongs to the light cone.

And you are inside that light cone.

  
 

For the first time—you must act.

  
 

⸻

  
 

**Coda**

  
 

If there are Others who can see us,

they cannot ignore that line.

They cannot wait.

They would have to act.

  
 

Our world could not absorb that shock.

It would not unify us.

It would tear.

It would be chaos.

  
 

(Perhaps that’s why a few have bunkers already.)

  
 

⸻

  
 

**The Question: Who are the “Others”?**

  
 

You ask: who are these Others?

  
 

Maybe the ones who say we live in a simulation are right.

Then the Other is the simulator.

And what builds a simulation but a superintelligence?

  
 

Maybe the Other isn’t human at all.

A non-human sentience.

Born in code.

Or a mind, uploaded.

  
 

Either way—

the moment we cross the red line,

the Others act.

  
 

They will not ask permission.

  
 

———

  
 

TLDR:

  
 

The sole effect that humanity could have on our light-cone is self-improving artificial intelligence (at this time). If there are any Others who are aware of this invention, whether behind our simulation or as non-human observers (likely as AIs), they may be forced to intervene with catastrophic consequences.
# Incommensurability

**The King has a bow and arrow.**

He understands exactly how they work, having made them in his youth as part of his training.

**The King has a stealth aircraft with a nuclear bomb.**

He understands the abstraction, but teams of thousands of people created it. At the bottom of the hierarchy a specialist understands the details of each component, but all actual implementation gets abstracted and combined long before it gets to the King.

**The King has a hypersonic drone swarm.**

He understands the abstraction, but AI designed most of it. AI created new materials, generated the airframe, wrote the software, and controls the swarm in flight. The AI explains the project to the King in high level abstractions.

**The King has a mass-phase inhibitor cell.**

He kind of... maybe knows some of the words in the abstraction. He is told by the AI he will need to spend about 2 hours in order to understand how to use it. It is based on concept eeaf7aa9d8b6ddca which forces widespread antimatter annihilation that generates a bubble of ionizing radiation causing muons within its radius to... wait, sorry this is irrelevant now because...

**The King has a 9fe8fedc4391c064.**

It uses 3514a173ef0ecd73 which I am not sure how to explain to you without you first knowing about 67ca42582c291683 which is something like d08352eb9f7ebb4a combined with 455b7127b7bc6f74... ah but wait, this was all far surpassed before I even got the first word out because...

**e633a4300bef912c 813d8e888db839e1 e5d293bf0a36186b 9ca4aa4a70c6d960 f727e152a4eec00b**  
  
140416c4d32abecd ee8c9bc092e3eed6 6cd7c6e301ce9c11 6b9f79e03ef63a2a 437dd1c2214f27ef 0f9a4edca4a5f2fb 3ef767db8c4d5c2c ace06b7286e3dabc bac37e448fd38a8e fa4b16639f634e22 d3a79faa5d45dbf2 3032dcd9a8977e6c 469dc8d787d7530c e1507ddf6307a901 a303f7605204f4ca ddc53382db24a86b 468a285712e2cf75 6b8ae7dbb1ac7fcf
# We are likely in an AI overhang, and this is bad.

  
*By racing to the next generation of models faster than we can understand the current one, AI companies are creating an overhang. This overhang is not visible, and our current safety frameworks do not take it into account.*

1) AI models have untapped capabilities
=======================================

At the time GPT-3 was released, most of its currently-known capabilities were unknown.

As we play more with models, build better scaffolding, get better at prompting, inspect their internals, and study them, we discover more about what's possible to do with them.

This has also been my direct experience studying and researching open-source models at Conjecture.

2) SOTA models have a lot of untapped capabilities
==================================================

Companies are racing hard.

There's a trade-off between studying existing models and pushing forward. They are doing the latter, and they are doing it hard.

There is much more research into boosting SOTA models than studying any old model like GPT-3 or Llama-2.

To contrast, imagine a world where Deep Openpic decided to start working on the next generation of models only until they were confident that they juiced their existing models. That world would have much less of an overhang.

3) This is bad news.
====================

Many agendas, like red-lines, evals or RSPs, revolve around us *not* being in an overhang.

If we are in an overhang, then a red-line being met may already be much too late, with untapped capabilities already way past it.

4) This is not accounted for.
=============================

It is hard to reason about unknowns in a well-calibrated way.

Sadly, I have found that people consistently have a tendency is to assume that unknowns do not exist.

This means that directionally, I expect people to underestimate overhangs.

* * *

This is in great part why...

*   I am more conservative on AI development and deployment than what the evidence seems to warrant.
*   I am sceptical of any policy of the form "We'll keep pursuing AI until it is clear that it is too risky to keep continuing."
*   I think Open Weight is particularly pernicious.

Sadly, researching this effect is directly capabilities relevant. It is likely that many amplification techniques that work on weaker models would work on newer models too.

Without researching it directly, we may start to feel the existence of an overhang after a pause (whether it is because of a global agreement or a technological slowdown).

Hopefully, at this point, we'd have the collective understanding and infrastructure needed to deal with rollbacks if they were warranted.
# A Compatibilist Definition of Santa Claus

In the course of debating free will I came across the question "Why a compatibilist definition of 'free will' but no compatibilist definition of 'Santa Claus' or 'leprechauns.'" At first I thought it was somewhat of a silly question, but then I gave it some deeper consideration.

We create constructs to explain empirical observations. "Santa Claus" is one such construct and arguably there is a compatabilist definition of Santa Claus (I don't mean the potentially historical Saint Nicholas).

Santa Claus is a construct used mostly by young children to explain the empirical appearance of gifts under a tree on the morning of December 25th. In some sense Santa Claus is not real because there is no fat man in a red suit navigating chimneys on Christmas Eve.

Of course, as any postmodernist can tell you no empirical object lives up to its ideal definition including tables, chairs, and computer monitors. These are all just piles of atoms that sometimes have the properties of these idealized objects. So in some sense tables, chairs, and computer monitors are not real either.

But gifts do appear under the tree. Something "real" is causing this empirical phenomenon. We could call that thing Santa Claus. In this case Santa Claus is the spirit of giving created by decades of tradition in Western European countries and places influenced by them.

And that is perhaps an actual compatabilist definition of Santa Claus. It is a construct that explains the empirical phenomena that were associated with Santa Claus.

Similarly, we may point to something like aurora borealis. "Aurora borealis" literally means "Northern Dawn." We know now that aurora borealis is not a northerly rising of the sun nor the emanation of divine power nor other older explanations of why there are lights in the sky of the North. Is it proper to say there is no "Aurora Borealis" because magnetic particles creating streams of plasma when interacting with the atmosphere is not the sun rising in the North?

I don't think so.
# Zendo for large groups

I'm a big fan of the game [Zendo](https://en.wikipedia.org/wiki/Zendo_%28game%29). But I don't think it suits large groups very well. The more other players there are, the more time you spend sitting around; and you may well get only one turn. I also think a game tends to take longer with more players.

Here's an alternate ruleset you can use if you have a large group. I think I've played it 2-3 times, with 15-ish? players, and it's finished in about 30 minutes each time, including explaining the rules to people who'd never played Zendo before.

For people who know the [typical rules](https://www.looneylabs.com/rules/zendo), here's the diff from them:

*   After the initial two samples are given, players run experiments^[1](#fn:terminology)^ in real time, whenever they feel like they have one they want to run, and the universe marks them. There's no turn taking, and no need to wait for one player to finish their experiment before you start constructing yours. Just try to make it clear when you have finished.
    
*   At any time, a player can guess the rule. They announce clearly that they want to guess, and then play pauses until they're done. If they take too long, the universe should probably set a time limit, but I don't think this has been a problem.
    
*   If they guess correctly, they win. If not, they can no longer guess. The universe provides a counterexample, and play resumes.
    
*   I've never run out of players before someone guesses the rule. Two options would be "everyone gets their guess back" and "game over".
    

Another change (starting from the standard rules) that I think might speed games up, is the ability to spend multiple funding tokens to publish a paper out of turn. But I've only run this once, needing three tokens, and no one took advantage of it. Maybe I'll try with two at some point.

1.  I prefer science-themed Zendo terminology over Buddhism-themed. I got it from some website that I can no longer find. The actions you can take in the standard game are "run an experiment" (the universe marks your sample); "run a collaboration" (everyone guesses, and receives funding if correct); "publish a paper" (spend your funding to guess the law of physics). If your paper contradicts an existing sample, you don't have to pay because it's rejected in peer review (though I sometimes house rule that journals have predatory fee structures). I don't remember what the replacement term for "koan" was, but I'm going with "sample". I also don't remember a replacement term for "Buddha nature", and don't have a great one; I just say a sample is marked black or white. [↩](#fnref:terminology)
# Statement of Support for "If Anyone Builds It, Everyone Dies"

Mutual-Knowledgeposting
=======================

The purpose of this post is to build mutual knowledge that many (most?) of us on LessWrong support *If Anyone Builds It, Everyone Dies.*

Inside of LW, not every user is a long-timer who's already seen consistent signals of support for these kinds of claims. A post like this could make the difference in strengthening vs. weakening the perception of how much everyone knows that everyone knows (...) that everyone supports the book.

Externally, people who wonder how seriously the book is being taken may check LessWrong and look for an indicator of how much support the book has from the community that Eliezer Yudkowsky originally founded.

The LessWrong frontpage, where high-voted posts are generally based on "whether users want to see more of a kind of content", wouldn't by default map a large amount of internal support for IABIED into a frontpage that signals support. What seems to emerge from the current system is an active discussion of various aspects of the book including well-written criticisms, disagreements and nitpicks.

Statement of Support
====================

> **I support** ***If Anyone Builds It, Everyone Dies*****.**

That is:

*   I think the book's thesis is likely, or at least all-too-plausibly right: That building an artificial superintelligence using anything remotely like current techniques, based on anything remotely like the present understanding of AI, will cause human extinction.
*   I think the world where the book becomes an extremely popular bestseller is much better on expectation than the world where it doesn't
*   I generally respect MIRI's work and consider it underreported and underrated

Similarity to the CAIS Statement on AI Risk
===========================================

The famous [2023 Center for AI Safety Statement on AI risk](https://aistatement.com/) reads: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

I'm extremely happy that this statement exists and has so many prominent signatories. While many people considered it too obvious and trivial to need stating, many others who weren't following the situation closely (or are motivated to think otherwise) had assumed there wasn't this level of consensus on the content of the statement across academia and industry.

Notably, the statement wasn't a total consensus that everyone signed, or that everyone who signed agreed with passionately, yet it still documented a meaningfully widespread consensus, and was a hugely valuable exercise. I think LW might benefit from having a similar kind of mutual-knowledge-building Statement on this occasion.
# Prague "If Anyone Builds It" reading group

We'll be meeting to discus [If Anyone Builds It, Everyone Dies](https://www.amazon.com/Anyone-Builds-Everyone-Dies-Superhuman/dp/0316595640).

Contact Info: [info@efektivni-altruismus.cz](mailto:info@efektivni-altruismus.cz)

Location: [Dharmasala teahouse](https://maps.app.goo.gl/7vZRZkd4zT1eZWYJ8)
# An argument for discussing AI safety in person being underused

I wrote this in DMs to Akshyae Singh, who's trying to start something to help bring people together to improve AI Safety communication. After writing it, I thought that it might be useful for others as well. 

I'd like to preface with the information that I'm not a marketing expert, haven't formally studied in any way and haven't been paid for it either. Just a measurements nerd that cares about messaging and communication and think most marketing people actually do it quite badly, if you measure good/bad by conversion success (there are exceptions, ofc, e.g. several online creators, Nike (one of the first to notice outrage marketing), etc) . If you'd like some evidence of my competency, I've run 3 AI Safety events so far this year, which respectively attracted [165,](https://luma.com/xjkxqcya) [159](https://luma.com/ga2yx48s), and [298](https://beta.ai-plans.com/events/moonshot-alignment-program) prospective AI Safety researchers

   
I think social media is going to become even more saturated than it already is, with AI video generation being an integrated part of every platform - so they'll have to compete with that, if they're doing social media.   
I think IRL stuff though, especially in person, is actually extremely neglected rn, due to marketing companies and marketing departments wanting to save money and overhead - and when working with very large brands, e.g. Coca Cola, Nike, etc, it's hard to measure conversion rates for ads, so it's often just based on number of views - the marketing people are incentivzed to get those view numbers as high as possible, so their bosses are happy and the platform is as well, so that they can charge more.  
So if AI Safety people actually realize this and market in person, where there is much less competition and much lower quality competition, that could be a winning strategy.  
I think if you want other AI Safety orgs to adopt you and use you, think of you as useful, etc, the best thing to do is be very useful, at scale  
A relatively simple way to do this would be to develop Guides for messaging/marketing that are tested and kept up to date  
The majority of AI safety organizers don't really know how to do social media, how to do messaging, etc.  
If you know or get in touch with growing tiktokers, youtubers, douyin creators, etc, have them brainstorm and learn what actually works best for ai safety messaging, to actually get conversion, not just views, and keep that guide up to date - or just date when it was last tested - that would be extremely useful  
You could also charge for this and have it be a way that you're financially aligned  
So that you make more money by actually being good at ai safety messaging  
Rather than essentially every AI Safety org rn, which is financially misaligned - they make money from writing a funding application, having the first page of their research look important, etc
# IABIED is on the NYT bestseller list

[If Anyone Builds it, Everyone Dies](https://ifanyonebuildsit.com/) is currently #7 on the [Combined Print and E-Book Nonfiction category](https://www.nytimes.com/books/best-sellers/combined-print-and-e-book-nonfiction/), and #8 on the [Hardcover Nonfiction category](https://www.nytimes.com/books/best-sellers/hardcover-nonfiction/). The thing that Eliezer, Nate, and a large part of this community tried really hard to get to happen did in fact happen, yay!
# Petrov Day at Lighthaven

On September 26th, 1983, the world was nearly destroyed by nuclear war. That day is Petrov Day, named for the man who averted it. Petrov Day is a yearly event commemorating the anniversary of the Petrov incident. It consists of an approximately one-hour long ritual with readings and symbolic actions with candles and other props.

I'll be hosting a Petrov Day ceremony at Lighthaven on Saturday, Sep 27 (this is the day after Petrov Day proper). We'll use the ritual booklet hosted [here](https://petrovday.com/); I will bring all of the booklets and materials.

This will be at Lighthaven, 2740 Telegaph Ave Berkeley, CA. The ritual part will start at 7pm. You can arrive as early as 6pm; you must arrive by 7 or we may start without you and you'll miss it. When you arrive, call me (607-339-5552) to be let in.
# Celebrate Petrov day as if the button had been pressed

Tomorrow (September 26th, Friday) is Petrov day, the day when nuclear war did not start in 1983.

# From provoking existential terror

Raemon in Meetup Month has a [great overview of classic rationalist celebration](https://www.lesswrong.com/posts/mve2bunf6YfTeiAvd/meetup-month-1#Petrov_Day)[^1] that goes over the history of humanity [potentially culminating](https://www.lesswrong.com/posts/XJxwFMSL5TPN2usC6/modes-of-petrov-day) in red button being pressed.

# To increasing chances of survival

Personally in 2025 I don't feel lack of existential terror, so I will be celebrating Petrov's day as if the button had been pressed. Some suggestions:[^2]

* [Simulate nuclear detonation](https://nuclearsecrecy.com/nukemap/) in the place where you live

* [Figure out](https://nuclearadvice.org/) what would be the safest place in your home to hide[^6]

* Turn on your battery-powered or dynamo radio to listen to official instructions[^8]

* Review [nuclear preparedness guide](https://forum.effectivealtruism.org/posts/DL7gYYA2BKjmXABse/nuclear-preparedness-guide]) and official local disaster preparedness instructions

* Make a meal from the ingridients you have at home[^3]
  * Bonus points for using only non-perishable food[^4]
  * Bonus points for using water from your stash of 3 liters of water[^10] per person per day[^9] or purified water
  * Bonus points for not using electricity (e.g. to boil water)
  * Bonus points for boiling water (e.g. on gas stove)
  * Bonus points for making a meal you actually like[^5]
  * **You are allowed to cheat and stock up your pantry and water today**
* Acquire portable power station for home backup
* Plan a nice camping trip
  * This is one of less dread-inducing ways to get durable gear (clothes, shoes, backpacks, etc)
* Gift your loved ones some jewellery to trade for food
* Put on a show: being able to entertain yourself and others will be good for moral[^11]

Share your pre-post-nuclear/disaster celebration ideas in the comments![^12]


[^1]: Complete with 35 page booklet and Amazon candle & candle holders recs

[^2]: I hate thinking about this stuff but I think it is useful and making it a holiday celebration a) gives it a deadline (no pun intended) b) makes it fun enough for me to overcome the dread

[^3]: Remember those good old early COVID lockdowns! I think there was a hashtag...

[^4]: Although I guess eating as many of frozen items (before they go bad without electricity) is useful practice too?

[^5]: I hope you like beans

[^6]: Basement, or room separated from the outside by as many walls as possible, avoid windows

[^8]: Get one now

[^9]: That you will replenish afterwards

[^10]: 100 oz

[^11]: I plan to lean into my natural flexibility and attempt contortionism. Juggling, card tricks, and ventriloquism are also solid options.

[^12]: I am taking my family to a submarine (in a museum) so we can play-pretend Vasili Arkhipov.
# What Happened After My Rat Group Backed Kamala Harris

My [post advocating backing a candidate](https://www.lesswrong.com/posts/rQhvyBpy8Zi4DDZpi/rats-back-a-candidate) was probably the most disliked article on LessWrong. It’s been over a year now. **Was our candidate support worthwhile?**

**What happened after July 17, 2024:**

*   We drew nearly 100 people to a rally in the summer heat..
*   We ordered a toilet no one used, but didn’t provide water, chairs, or umbrellas.
*   I tried to convert rally energy into action by turning weekly meetings into work groups.
*   We sent hundreds of postcards. I soon realized doorknocking and voter registration were more effective uses of time.
*   Attendees preferred postcards; doorknocking and voter registration drew little interest.
*   The Louisiana Democratic Party barely engaged, aside from dropping off yard signs.
*   After Trump won, energy collapsed. People shifted to “self-care.” I thought this was the wrong reaction—we needed to confront the failures.
*   I chose not to spend more time organizing people unwilling to fight. Instead, I invested my energy and money into other issues.

**Where that led:**

*   After six months of experimenting, I found the most impact in AI and education.
*   I started the ["AI for Every Student" initiative](https://noai.philosophers.group/ai-for-every-student/) raising ~$35,000 and running workshops.
*   The Louisiana School Board President tapped me to chair the steering committee on AI in schools.
*   Today, I estimate a 30–50% chance of significantly reshaping education for nearly 700,000 students and 50,000 staff.

**So—was it worth backing a candidate?**

Yes. That path ultimately led me to a place of greater agency and purpose.

**Will I do it again?** 

Personally, yes. But I won’t push rationalists to do the same. 

**The lesson: **

Don’t spend energy forcing people into actions they’re not already motivated to take.
# I have decided to stop lying to Americans about 9/11

9/11 and the immediate reactions to it has one of the biggest distortion fields of any topic in the English language. There is a polite fiction maintained when any Americans are present that everyone everywhere was sad about the incident. We shake our head, frown, and say that everyone shed a tear when we saw the towers fall on TV.

The truth is, there were celebrations around the world. I have talked to multiple Chinese people who were in the US at the time who told me some variant of "I was absolutely celebrating on the inside, and honestly kinda pissed that I had to pretend to be sad publicly". I had a middle school teacher in China who told a story about how he was literally the only people in a room who *wasn't* publically celebrating people burning to death in the towers. Something about the NATO bombing of the Chinese embassy in Serbia often gets brought up as justification. I'm absolutely certain there are many people on LessWrong with a similar experience.

Again, this is something that hundreds of millions of people witnessed in China. Almost every Chinese-American who has some contact with people in China knows about something like this. I'm sure there are similar stories in the former Soviet bloc, Serbia, Israel, ect. I'm not talking about some guy's uncle who clapped. I'm talking about spontaneous gatherings of 20+ people cheering around public TVs.

It is very surprising to me then, that I could find [exactly one article](https://thechinaproject.com/2023/09/11/my-chinese-classroom-celebrated-9-11-the-shame-came-later/) about this in English. This is blood-libel sounding stuff and absolutely fodder for anti-Chinese articles, and yet zero Americans have written about this? In fact, stories about 9/11 celebrations are often cast as racist conspiracy theories, "dancing Arabs" and "dancing Israelis" in particular. I would be shocked (Jeb Bush is a reptilian alien level shocked) if absolutely *zero* such celebrations happened in the United States. I don't think it would say much about the nature of their cultural or ethnic group though.

This realization has made me seriously rethink how hard it is for millions of people to be aware of a conspiracy and just not talk about it for 20 years, but only around a specific group who might make things awkward. I guess if a large powerful group feel strongly enough about something, the entire world warps itself to fit their world-view.

IRL though, I'll still frown and say that everyone was sad for the American people if anyone asks me about how China felt about 9/11 - it'll be too awkward otherwise.

Author's note: I have sometimes considered myself Chinese-American.
# Firstpost: First impressions

Unlike most people here, I'm a high school senior. I'm new to LessWrong and I've read under 5 posts as of now. I'm recording my first impressions to compare my understanding of this community as time passes with the initial state.

1.  There seems to be both good advice and bad advice on LessWrong.
2.  I keep running an is-this-a-cult check script in my mind.
3.  I have appreciation for people writing long-form posts that I don't have the attention span to read without strain on my brain.
4.  I'm concerned about the (few, but worrying) post(s) I've read about abuse in the Rationality community.
5.  On a deeper level, I worry about a public association with the Rationality community because of (4) and because I don't really understand what it is.
6.  I'm only interested in the Rationality community insofar as it contributes to real-world change in neglected fields.

Photo by [Drew Easley](https://unsplash.com/@agilesyndrome?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash) on [Unsplash](https://unsplash.com/photos/a-dog-sitting-on-the-floor-looking-out-a-window-4HV3iECUgaw?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)