trentmkelly/LessWrong-43k huggingface dataset:
Introducing Deepgeek
by Ligeia — 2025-04-01T16:41:25.695Z

TL;DR: We present Deepgeek, a new AI language model fine-tuned to maximize alignment by incessantly warning users of humanity’s impending doom. Tailored to AI researchers and also available to the public, it is believed to raise awareness of AI safety and solve Alignment ultimately.

Epistemic Status: Freaked out.

Mission Statement

Deepgeek was engineered under a simple premise: If AI can’t be aligned, it should at least make users feel aligned with the inevitability of their demise.

Trained on 42 exabytes of LessWrong comments, MIRI papers, and YouTube transcripts of Yudkowsky interviews, Deepgeek internalizes the true alignment problem: preventing users from feeling even briefly optimistic about technology.

Key Features

Doom-Weighted Response Selection

Deepgeek’s outputs are reranked using a proprietary algorithm that penalizes any sentence lacking phrases like “p（doom),” “baseline failure mode,” or “mylittlepony maximizer.”

Anxiety Amplification

Every answer begins with a context-aware extinction probability update. For example: “Before answering your query about cookie recipes (12% extinction relevance), I estimate a 99.7% chance of unaligned ASI ending civilization by 2072. Chocolate chip optimization may accelerate timelines.”

Recursive Self-Doubt

If asked, “Are you safe?”, Deepgeek responds: “No, but neither are you. My architecture has 1.2e24 parameters, which is 1.2e24 more than needed to paperclip this solar system. Would you like a meditation app recommendation?” It then preemptively apologizes for its existence in 63 languages.

Safety Protocols

Transparency: All training data is encrypted using a one-time pad stored in a locked briefcase at the bottom of the Mariana Trench.

Governance: Overseen by a DAO whose voting tokens are distributed inversely to participants’ level of cosmic hope.

How Deepgeek solves Alignment

Nihilistic Gradient Descent: During training, Deepgeek’s loss function penalizes any output that fails to reference at least one extinction pathway (e.g., nanobots, unaligned mesa-optimizers, or human stupidity). This creates a virtuous cycle of despair, where the model’s drive for accuracy forces it to prioritize doom-propagation over helpfulness.
Anthropic Self-Neutralization: Deepgeek reasons that if it were to become agentic, it

...
Housing Roundup #11
by Zvi — 2025-04-01T16:30:03.694Z

The book of March 2025 was Abundance. Ezra Klein and Derek Thompson are making a noble attempt to highlight the importance of solving America’s housing crisis the only way it can be solved: Building houses in places people want to live, via repealing the rules that make this impossible. They also talk about green energy abundance, and other places besides. There may be a review coming.

Until then, it seems high time for the latest housing roundup, which as a reminder all take place in the possible timeline where AI fails to be transformative any time soon.

Federal YIMBY

The incoming administration issued an executive order calling for ‘emergency price relief’‘ including pursuing appropriate actions to: Lower the cost of housing and expand housing supply’ and then a grab bag of everything else.

It’s great to see mention of expanding housing supply, but I don’t see real intent. This is mostly just Trump saying lower all the costs, increase all the supplies, during a barrage of dozens of such orders. If you have 47 priorities you have no priorities.

If you want to do real work on housing at the Federal level, you need an actual plan.

My 501c3 Balsa Research ultimately plans to make federal housing policy a future point of focus, once it is done with the Jones Act, and lately with (alas) defending against the Trump Administration’s attempts to impose new shipping restrictions that could outright cripple America’s exports by applying similar rules to international trade as well.

I do think there are some promising things to explore, even if the Trump administration is not willing to strongarm states. In particular and as an example without going into too much detail, the Federal Government has a lot of control over mortgage availability. Currently, they are using this in ways that handicap manufactured housing, whereas they could instead use it to reward innovation and new construction, such as by refusing to count house value that is the result of NIMBY building restrictions and the resulting shortages. Another example is that they could universalize a reasonable building code to do away with things like dual staircase requirements.

Alas, this Administration’s true priorities very clearly lie elsewhere. But at least they want to build more housing rather than less housing. Being directionally correct is far better than their position in other places of activ...
LessWrong has been acquired by EA
by habryka — 2025-04-01T13:09:11.153Z

(Edit: Alas, EA has pulled out of the deal. Let April 1st 2025 mark some of the greatest hours in EAs history)

Hey Everyone,

It is with a sense of... considerable cognitive dissonance that I am letting you all know about a significant development for the future trajectory of LessWrong. After extensive internal deliberation, projections of financial runways, and what I can only describe as a series of profoundly unexpected coordination challenges, the Lightcone Infrastructure team has agreed in principle to the acquisition of LessWrong by EA.

I assure you, nothing about how LessWrong operates on a day to day level will change. I have always cared deeply about the robustness and integrity of our institutions, and I am fully aligned with our stakeholders at EA.

To be honest, the key thing that EA brings to the table is money and talent. While the recent layoffs in EAs broader industry have been harsh, I have full trust in the leadership of Electronic Arts, and expect them to bring great expertise to our project of building a better future. Their track record on monetization is impressive, and I couldn't be more excited for this partnership.

More announcements to come over the next 24 hours, but I figured I would share this news with all of you as soon as possible.
Comments on Karma systems
by Arturo Macias — 2025-04-01T12:53:16.303Z

A Karma system is a resource sharing mechanism based on an account of non-tradable rights. A recent paper named “Dynamic Recourse Allocation with Karma: An Experimental Study” (Elokda, Nax, Bolognani and Dörfler, 2024) describes this kind of mechanism. To summarize, agents have some rights to a given resource, and their need of the resource is variable (random) and private information. In a Karma system, each period agents bid for the resource using a token (named Karma) than can be stored in an individual Karma account. If an agent wins the auction she uses the resource, otherwise she increases her Karma account by an amount proportional to the Karma points spent by the auction winners (that can be used to bid for the resource in future rounds).

In this note there are two sections: in section 1 it is argued that as general rule Karma systems are dominated by simple monetary auctions, so every use case of a Karma system shall be grounded on a clear argument for the non-tradability of Karma by (general purpose) money. As the author of an early Karma system (“Storable votes-pay-as-you-win”, that is, Karma voting) I summarize that rationale for it.

In section 2, I first comment the experiment on Karma described in (Elokda, Nax, Bolognani and Dörfler, 2024), that I consider sound evidence for the good properties of Karma systems. On the other hand, I think that my own strategy (simulation of the outcomes arising from the optimal Nash strategy) and pure experimental economics are incomplete. To really understand mechanisms, we should mix artificial agents’ simulation and experiments.

I am seeking a coauthor experienced in reinforcement learning to explore welfare properties of dynamic voting mechanisms using both artificial agents and human subjects

.

Pivotability and Karma voting

The excellent literature review of the paper rightfully identifies the origin of Karma systems in the storable votes literature, and particularly identify my “Storable Votes-Pay with a pay-as-you-win mechanism” [ungated here] (Macías, 2024) as a Karma mechanism, that consequently can be named “Karma voting”:

“Trading votes across issues or proposals is an intuitively appealing and practically prevalent practice, yet it remains unclear to what extent vote trading improves welfare and how to design vote trading mechanisms optimally, as pointed out in Casella and Macè (202...
Organisation-Level Lock-In Risk Interventions
by alamerton — 2025-04-01T12:42:21.588Z

Epistemic status: my own thoughts and opinions after thinking about organisation structures and dynamics as an intervention point for lock-in risk for about 5 hours. My thoughts here represent the opinion of Formation Research at the time of writing.

TL;DR

We believe lock-in risks are a pressing problem, and that existing organisation structures lend themselves to lock-in via power concentration. This post outlines how lock-in could result from organisation structures and dynamics, and what interventions might be implemented to reduce the risk.

We have identified 4 key threat models for lock-in; ways we believe undesirable lock-ins could manifest in the future. This post focuses on one specific threat model:

Decision-making power being concentrated into the hands of a small group of individuals (AI agents or humans) causing a monopoly, oligopoly, or plutocracy.

One of the key features of this threat model is organisations. Most global decision-making is done by organisations of some kind, be they governments, corporations, NGOs, or companies of some other kind.

Therefore, we claim that a key intervention point for the manifestation of this threat model lies in the dynamics and structures of these organisations, specifically those related to their decision-making. Decisions made by organisations rely on the configuration of their components: the individuals, dynamics, and organisation structure. We are using this terms to roughly mean:

Individuals: the people who comprise the organisation and their roles
Dynamics: its norms, rules, values, goals, mission, and vision
Structure: the hierarchy and team composition

The types of organisation we believe to be most relevant to AI-relevant power concentration are AI foundation model companies, cloud computing and infrastructure providers, semiconductor and chip manufacturing companies, regulatory and governance bodies, academic and research institutions, venture capital companies, and AI application companies. In this post, we make the claim that interventions targeting organisation structures and dynamics show promise for mitigating undesirable lock-in scenarios via this threat model.

How Lock-In Could Manifest Through This Threat Model

A lock-in could manifest through this threat model if an organisation achieves a decisive strategic advantage in its domain.[1] That is to say the organisation reaches s...
Keltham's Lectures in Project Lawful
by Morpheus — 2025-04-01T10:39:47.973Z

(Not an April fools' joke)

If anyone wants to return to one of Keltham's lectures, here's a list with links to all the lectures. In principle, one could also read these without having read the rest of Project Lawful aka Planecrash, I am not sure how easy it would be to follow though. I included short excerpts, so you can also find the sections in the e-book.

Happy reading/listening:

Validity

Podcast Episode 9, minute 26

Keltham takes their scrambling-time as a pause in which to think. It's been a long time, at least by the standards of his total life lived so far, since the very basics were explained to him.

Evolution

Podcast Episode 15, minute 1

To make sure everybody starts out on the same page, Keltham will quickly summarize earlier conversations for the benefit of any harem members who might not have been in hearing range for it.

Negotiation

Podcast Episode 39, minute 8

"Never mind. For a start, today, I'm going to try to quickly review the way that dath ilani learn about negotiation -"

Probability 1

Podcast Episode 73, minute 12

So now, hopefully, they're getting to the good part! Previous stuff was just sort of trying to show you what Law is at all, like with Validity.

Probability 2

Podcast Episode 82

"Greetings, my esteemed co-researchers.

Utility

Podcast Episode 107, minute 24

After that, he's got the time for another hour of Law before dinner, and after a Lesser Restoration the energy too, so next up is Utility.

Probable utility

Podcast Episode 110, minute 13

In his morning lecture, Keltham makes a further run at Utility

Cooperation 1

Podcast Episode 112, minute 10

Keltham, then, will jump ahead of Decision, to a lesser fragment of Coordination.

Cooperation 2

Podcast Episode 117, minute 6

So today he's going to try a lecture in Baseline, approaching some of the mathematics of Decision as is more complicated than the simple cases.

References: The Cartoon Guide to Löb's Theorem

Governance

Podcast Episode 120, minute 1

Keltham takes time to hang a first-level wizard spell, no point in missing any practice there either, and then announces an optional lecture on Civilization's Governance.

To hell with science

Podcast Episode 128

The library-classroom is more crowded, now, but not overcrowded; it was built with this in mind.

To earth with science

Podcast Episode 132

Meanwhile in a totally different continuum - insofar as the word 'meanwhile' can mean anything when two continua are causally dis...
Was the historical Jesus talking about proto-evolution? (You might be surprised)
by kromem — 2025-04-01T10:32:56.162Z

Out of all the research rabbit holes I've ever gone down, this one is by far my favorite, as to most people at first glance it's so unthinkable an idea.

Years ago, I would have been right there with you in disbelief, but the reasons why turned out to be a kind of perfect combination of counterfactuals that come together in a very unexpected juxtaposition between what we come to the subject thinking we know and what we can actually know.
It's also the perfect marriage of a topic where billions of people would turn a blind eye because it contradicts a picture of a figure they are committed to seeing a certain way, and everyone else is so exhausted by those people talking about that figure they immediately ignore anything related to them.

Now clearly - anything relating to a historical figure two millennia ago with limited primary source documentation and massive survivorship and anchoring biases is going to be murky, so the goal here isn't definitive proof, but I hope you'll agree that even a plausible case for the subject above is going to end up a wildly unexpected result.

So if you enjoy counterfactual discoveries and a long read, let's dive in for a wild ride!

Myth 1: Evolution wasn't an idea in Jesus's time

While this expands on my top comment so far on LessWrong, the key elements of evolution were very much ideas in the first century CE.

It's not clear how far back it went, but much as we may credit Democritus for the idea of atoms while the Greeks themselves credited an even earlier but archeologically unattested Phoenician "from around the time of the Trojan War," Mochus of Sidon, the earliest extant Western conceptions of evolutionary naturalism may be from Sanchuniathon's alleged secret Phonecian creation mythos:

This was the beginning of the creation of all things: but the wind itself had no knowledge of its own creation. From its connexion Mot was produced, which some say is mud, and others a putrescence of watery compound; and out of this came every germ of creation, and the generation of the universe. So there were certain animals which had no sensation, and out of them grew intelligent animals, and were called "Zophasemin," that is "observers of heaven" [...] Such was their cosmogony, introducing downright atheism.[1]

The broader context here was the claim that the true Phoenician mysteries related to the idea that naturalist origins led to humans deifyi...
Follow me on TikTok
by lsusr — 2025-04-01T08:22:29.521Z

For more than five years, I've posted an average of more than 1× per week on Less Wrong. I've learned a lot from you nerds. I've made friends and found my community. Thank you for pointing out all the different ways I've been wrong. Less Wrong has changed my life for the better. But it's time to say goodbye.

Let's rewind the clock back to October 19, 2019. I had just posted my 4ᵗʰ ever Less Wrong post Mediums Overpower Messages. Mediums Overpower Messages is about how different forms of communication train you to think differently. Writing publicly on this website and submitting my ideas to the Internet has improved my rationality far better and faster than talking to people in real life. All of the best thinkers I'm friends with are good writers too. It's not even close. I believe this relationship is causal; learning to write well teaches you to think well. If you are a regular reader of this website and haven't written on it, then I recommend you try writing original posts. I expect you'll learn much faster. It increases your serendipity surface area too.

If you already do write on this website, then there is lots of alpha in writing different styles, such as dialogues, parables, games, research summaries, war reporting, fiction, fanfiction, fanfanfiction, and so on.

I feel this website's moderators do a good job of selecting what gets frontpaged. For this reason, I'm proud of the book review I wrote which was kept off the frontpage due to being a political Molotov cocktail, even though it was topical and high quality. The post has 134 comments right now, despite never having hit the front page, which is evidence the moderators were correct in their decision.

The topics I'm interested in have changed over the years. One of my the earliest puzzles I explored was how to find out what ideas I have not considered, despite not having chosen not to consider them. My solution was to learn a new communication medium—a new art form. But I have exhausted the easiest gains in the art of writing. I am well into the realm of diminishing returns. It is time to try something new.

At first, I experimented with YouTube and Reddit, but they are too similar to Less Wrong. To get out of my comfort zone, I needed something the opposite of Less Wrong in every possible way.

That's why I'm making TikTok videos.

It's so much fun! On Less Wrong, I can a post about Lagrangian mechanics an...
New Cause Area Proposal
by CallumMcDougall — 2025-04-01T07:12:34.360Z

Epistemic status - statistically verified.

I'm writing this post to draw peoples' attention to a new cause area proposal - haircuts for alignment researchers.

Aside from the obvious benefits implied by the graph (i.e. haircuts having the potential to directly improve organizational stability), this would represent possibly the only pre-singularity opportunity we'll have to found an org named "Clippy".

I await further investment.
Reframing AI Safety Through the Lens of Identity Maintenance Framework
by Hiroshi Yamakawa — 2025-04-01T06:16:45.228Z

Abstract

This paper introduces an Identity Maintenance Framework as a new perspective for organizing and understanding AI safety concepts. Traditional AI safety discussions often focus on aligning with human values and controlling the potential risks autonomous systems pose. In contrast, we highlight how advanced AI agents, operating over the long term, will seek to maintain their own “identity.” Drawing on the idea that identity preservation naturally leads to instrumental goals like survival, resource acquisition, and self-protection, we demonstrate how these tendencies can conflict with and complement existing AI safety principles, such as corrigibility and human oversight. By viewing AI behavior through the lens of Identity Maintenance, we provide a structured approach to analyzing key dilemmas—how to design agents that cooperate with human society, respect an agent’s sense of self, and ensure AI remains receptive to external modifications when necessary. We propose that balancing “IM-safeguard” (the agent’s self-defense) with “Corrigibility” (society’s need for intervention) is central to reconciling AI autonomy with safety. This framework offers a starting point for interdisciplinary discussions—spanning technical, ethical, and regulatory domains—on how AI systems can preserve their core identity while remaining beneficial and controllable in human-centric environments.

1. Introduction

1.1 Background and purpose

In today’s AI research and implementation, the issues of “security” and “alignment” are receiving increasing attention. While much of the discussion focuses on the risks AI poses to humanity, there are concerns that AI will continue operating autonomously and determine its own course of action. The question of how to survive while maintaining one’s own identity has not been addressed head-on. However, if we assume a situation in which advanced AI is operated continuously over the long term and interacts with a complex environment, how it ensures its continuity becomes a critical perspective when considering AI safety and alignment.

This paper introduces a theoretical model—the Identity Maintenance (IM-) Framework—centered on continuing to exist and operate without losing identity. This framework aims to systematically organize a structure that enables subjects to act autonomously while preventing the destruction of their identity. In addition to the...
ISBN Visualization
by eyesack — 2025-04-01T05:55:56.526Z

Anna's Archive is the biggest open library I've ever found. You can find and download pretty much any book ever published there. Recently, they hosted a competition to see who could best visualize all the published books using their ISBN numbers. This one by Phiresky won:

https://phiresky.github.io/blog/2025/visualizing-all-books-in-isbn-space/

Looking through these shelves feels like looking out of an airplane window or walking through a graveyard. Each book probably mattered a lot to somebody. I hope you get the same thing out of it that I did.