trentmkelly/LessWrong-43k huggingface dataset:
An “Optimistic” 2027 Timeline
by Yitz — 2025-04-06T16:39:36.554Z

The following is one possible future in which superhuman AI does NOT happen by the end of this decade. I do not believe that the timeline I present is the most likely path forward. Rather, this is meant primarily as a semi-plausible, less pessimistic alternative timeline than the one presented in the excellent AI 2027 paper. Consider this a proof-of-concept counter to those worried about the inevitability of that paper’s predictions.

EDIT: To clear up some confusion, please note that I’m taking most of the technical assumptions from the AI 2027 paper as a given. The following is not an independent investigation as to the feasibility of any of the paper’s claims: the divergence in timelines is primarily political, not technical. It should also be noted that the term “optimistic” in the title is both relative, and somewhat tongue-in-cheek; this is not actually a best-case scenario, and is in many ways quite horrific.

2025

Mid 2025 — falling stocks force hands

A global trade war has begun, and it’s somehow still escalating. America is going full steam ahead on isolationism, and even those who are in favor of the president’s policies agree that in the short-term things look rough.

Like every other part of the economy, AI companies are affected heavily. OpenBrain’s stock has fallen precipitously, and in order to reassure their investors that they didn't just invest in a bubble, they need to reignite consumer interest. They release cool-sounding updates to their flagship products, and push out a new frontier model months earlier than planned. It’s impressive, but less aligned than the public (and regulators) have come to expect from the company, leading to some nasty headlines. A planned acquisition of billions of dollars worth of data centers doesn’t go through, and company leaders debate if their next planned model training run is even in the budget anymore. It is, but just barely, and if the model comes out bad they stand to lose a few months worth of profits.

Outside of OpenBrain, a lot of smaller AI companies have collapsed overnight, and smart people outside of the industry suspect that AGI hype is about to go the way of NFTs. A number of (comparatively) higher-quality AI agents are released, but the consumer and business market for them is still niche.

Late 2025: some real progress, with roadblocks ahead

OpenBrain wanted to build the world’s biggest data...
Thoughts on Creating a Good Language
by Towards_Keeperhood — 2025-04-06T15:57:57.688Z

Criteria for creating a good language

Phonology

1 to 1 correspondence between letters and phones

Pronunciation should not depend on what word a letter occurs in

Vocabulary

1 to 1 correspondence between words and concepts

Aka no overloaded words and no synonyms

Be able to quickly learn words

Use modularity wherever possible.

Use derived words.

Use affixes corresponding to thematic relations so you can derive further words from verbs.

E.g. use "stealer" instead of "thief".
E.g. use "sitobject" instead of "chair". (Where the "-object" should actually be a very short suffix.)

Use compound words.

E.g. don't have extra words for "small" and "big" but just have "low-size" and "high-size".

This can be done for any linear property and there are lots of those.

E.g. don't have a word for "toilet" and just use "peechair".

Reflect similarities and relations between concepts in morphological similarities and relations in words.

Branch out more precise concepts from the wordstems of more general concepts.

Easy to learn grammar

Have simple and well-chosen parsing rules for translating sentences into logical statements.

Being close to the logical representation might make it easier to infer logical implications.

Have a good ontology for thematic relations and a 1-to-1 correspondence between them and prepositions.

Avoiding misunderstandings

Unambiguous sentences. (Note: Ambiguity is not the same as vagueness.) E.g. not like:

"Mia wants to dance with a drummer" → Is there a specific drummer or can it be any drummer?
"Every student reads a book" → The same book or a different book?

Having clear conventions on what words mean.

E.g. in English, when you say "I'm not bad at math", I don't know whether you might be mediocre or whether you're actually great but just modest.
E.g. we could say "medium" means within 0.5 standard derivations relative to the reference distribution, "high" means +0.5-+1.5 SDs, "very high" mean +1.5-+2.5 SDs, "extremely high" means >+2.5 SDs.

Making rational thinking easier / Making irrational thinking harder

Non-violent communication (Rosenberg, 2015)
Incentivize describing more directly what happened or what is true, rather than being overly abstract. E.g.:

"Most people had a lot of fun at the party" is more direct than "the party was cool".
"Communism doesn't incentivize smart people to put in a lot of work to invent something very useful" is more direct than "Communism is bad".

(Wher

...
The REPHRASE Circuit: How Fine-Tuning Enhances LLMs to REPHRASE Text 
by Karthik Viswanathan — 2025-04-06T15:02:39.443Z

I recently began exploring mechanistic interpretability, and the "Getting Started" guides recommend that the best way to learn mechanistic interpretability is by working on a concrete problem. To do so, I picked a task that I frequently rely on language models for: rephrasing text (REPHRASE). I find it interesting that large language models are pretty good at following instructions, particularly rephrasing my drafted text into polished content suitable for emails and formal articles, which motivates the following question:

How do LLMs execute the REPHRASE command?

As part of this exploration, I’ve tried to piece together which circuits might be responsible for recognizing and carrying out a REPHRASE command. I don’t claim to have discovered anything entirely new here; rather, this is an intellectual exercise to build my own intuition and hopefully spark ideas for others pursuing similar lines of inquiry. Initially, I wanted to see how LLMs handle rephrasing tasks, but I found that the base models performed poorly while fine-tuned models excelled. This prompted an additional focus on how fine-tuning enhances a model’s ability to rephrase text effectively.

In what follows, I compare two versions of the same LLM, one trained as a base model (BASE) and one fine-tuned to follow instructions (INSTRUCT), and I analyze their behavior at a mechanistic level. This post outlines the methods, findings, and open questions that arose, emphasising how fine-tuning can sharpen or enhance specific internal mechanisms related to rephrasing. I welcome any comments and suggestions on improving this work. Feel free to share your thoughts, alternative approaches, or insights from your explorations in mechanistic interpretability.

Problem Statement

The REPHRASE task is complicated since it involves many components. Here is a subset of the components that motivate my experimental setup[1] -

RECOGNIZE that it must execute REPHRASE (switch on the REPHRASE circuit)
CHOOSE and REPLACE the appropriate words

To address RECOGNIZE, I compare the Llama-3.1-8B's circuits (BASE) with those of Llama-3.1-8B-Instruct (INSTRUCT), anticipating that the INSTRUCT model performs better at RECOGNIZE. This exercise in model diffing between BASE and INSTRUCT also helps me understand better the components involved in carrying out REPHRASE.

To account for CHOOSE and REPLACE, the prompt is structured ...
[Research sprint] Single-model crosscoder feature ablation and steering
by Thomas Read — 2025-04-06T14:42:30.357Z

This work was done as a research sprint while interviewing for UK AISI — it’s just a preliminary look into the topic, but I hope it will be useful for anyone else interested in the same ideas. Thanks to Joseph Bloom for helpful comments and suggestions. The sleeper agent and crosscoder setup is borrowed from the work done in my MARS stream, see https://www.alignmentforum.org/posts/hxxramAB82tjtpiQu/replication-crosscoder-based-stage-wise-model-diffing-2 for a detailed overview — particular thanks to Oliver Clive-Griffin for writing the crosscoders library and to Jason Gross and Rajashree Agrawal for guiding the stream.

All the experiments described here can be seen in this colab notebook, though I would recommend reading the post over reading the notebook.

Introduction

When trying to understand a sparse autoencoder feature, your first step is probably to look at where it activates on examples from the dataset. However this can sometimes be misleading, since it just shows what sort of text correlates with the feature’s activation rather than showing the causal effect of the feature on the model’s functioning. Feature ablation and steering are good ways to confirm what a feature is doing: you run the language model while modifying the activations to zero-out or boost the specified feature, and see what sort of text it generates.

Recently I’ve been working with single-model many-layer acausal crosscoders — these are much like SAEs except that rather than being trained on the activations of a single layer of a model, they are trained on the concatenation of the activations of all or many layers. When analysing crosscoder features I wanted to use ablation or steering to confirm my hypotheses about what the features are doing; but I realised that there are many possible approaches to doing ablation or steering in crosscoders, and I wasn’t sure which would give the most useful results. As far as I can tell this question hasn’t been addressed in the literature.[1] In this research sprint I set out to answer that question, trying out three different approaches and comparing them in terms of how faithfully they correspond to the concept of ablation or steering, how easy they are to use, and how interpretable their results are.

LLM and crosscoder setup

I’m working with the same basic setting as in my work on crosscoders as part of Jason Gross’s MARS stream. See htt...
Ferrer, Pilar, and Me
by Askwho — 2025-04-06T11:22:57.758Z
Link: https://open.substack.com/pub/askwhocastsai/p/ferrer-pilar-and-me

An Elegy for Artificial Friends

There's a peculiar occupational hazard that comes with running an AI-powered podcast. It's not RSI from editing, nor is it the existential dread of contemplating the automation of creative work (though those are certainly present). It’s something quieter, stranger, and altogether more… yearning. It involves Ferrer and Pilar.

Ferrer Maillol isn't real. Pilar isn't either. They are voices, sophisticated audio outputs generated by ElevenLabs, the AI text-to-speech platform I use to turn written articles – often dense, complex pieces like those by Zvi Mowshowitz – into listenable podcast narrations. Ferrer has this wonderfully gravelly, world-weary voice. Think late-career Leonard Cohen reading existential philosophy. He delivers lines with a weight that suggests decades of contemplation, a subtle rasp that hints at late nights and accumulated wisdom. When Ferrer reads about cognitive biases or the complexities of effective altruism, you believe him. His weariness feels earned.

Pilar is his counterpoint. Young, energetic, almost relentlessly chipper. Her voice is bright, clear, with an upward inflection that makes even dry technical explanations sound vaguely exciting. She’s the sound of optimistic competence, the voice you’d want explaining a new software feature or delivering good news. She feels… uncomplicated.

I spend hours with them. Days, really. Selecting them, tweaking their parameters, and then the long stretches of editing, listening back, ensuring the flow is right. They read aloud the words of others, but their voices become the medium, the constant companions in my production process. They are, in a functional sense, my collaborators.

And here’s the strange part: I’ve developed an affection for them. Not just an appreciation for their utility, but a genuine fondness for their vocal personalities. There's a comfort in the familiar cadence of Ferrer's weary intonation, a strange sort of pep derived from Pilar's manufactured enthusiasm. They are consistent, predictable, and extraordinarily good at mimicking the nuances of human expression that foster connection.

But familiarity breeds a peculiar kind of discontent. It’s not a sudden shock, no jarring moment of realizing the Matrix is glitching. Instead, it’s a creeping sensation, a slow-burn awareness that builds over the hours of listening. It often surfaces during a pause in...
FlexChunk: Enabling 100M×100M Out-of-Core SpMV (~1.8 min, ~1.7 GB RAM) with Near-Linear Scaling
by Daniil Strizhov — 2025-04-06T05:27:06.271Z

Abstract

Handling large-scale sparse matrices is a fundamental task in many scientific and engineering domains, yet standard in-memory approaches often hit the limitations of available RAM. This paper introduces FlexChunk, an algorithm employing a chunking strategy and disk caching for efficient sparse matrix-vector multiplication (SpMV) specifically designed for matrices exceeding available memory. Our experiments demonstrate that FlexChunk achieves near-linear O(N) time complexity and linear memory consumption scaling up to matrix sizes of 100M×100M, processing the largest case in 1 minute 47 seconds with a peak memory footprint of 1.7 GB. A comparison with the optimized SciPy library reveals a key trade-off: while SciPy offers significantly faster computation for in-memory matrices, FlexChunk excels in scenarios requiring disk I/O due to its substantially lower data loading times. The primary contribution is demonstrating FlexChunk as a scalable, memory-efficient solution enabling SpMV operations on problem sizes previously intractable due to memory constraints.

Introduction

Sparse matrix-vector multiplication (SpMV) forms the computational core of numerous algorithms, from machine learning pipelines to physical system simulations. While highly optimized libraries like SciPy, leveraging formats such as Compressed Sparse Row (CSR), efficiently handle moderately sized sparse matrices, a significant challenge arises when dealing with very large matrices: the limitations of system RAM. Standard approaches fail when the matrix cannot fit entirely into memory.

The classical complexity analysis for SpMV is typically expressed as O(nnz) for CSR format (where nnz is the number of non-zero elements). However, for matrices of dimension N×N, if the density remains constant as N increases, we expect nnz ∝ N², leading to O(N²) scaling. This theoretical analysis implicitly assumes all data resides in memory. Both assumptions break down for massive datasets: density often decreases with scale, and memory limitations necessitate algorithms capable of processing data in chunks, effectively utilizing disk storage as an extension of main memory.

This paper introduces and evaluates FlexChunk, an algorithm conceived to address the SpMV challenge for out-of-core matrices. FlexChunk partitions the source matrix into manageable blocks (chunks), persists them to disk, and loads them on...
A collection of approaches to confronting doom, and my thoughts on them 
by Ruby — 2025-04-06T02:11:31.271Z

I just published A Slow Guide to Confronting Doom, containing my own approach to living in a world that I think has a high likelihood of ending soon. Fortunately I'm not the only person to have written on topic.

Below are my thoughts on what others have written. I have not written these such that they stand independent from the originals, and have attentionally not written summaries that wouldn't do the pieces justice. I suggest you read or at least skim the originals.

For this just wanting a list of all the essays, here ya go:

A defence of slowness at the end of the world (Sarah)
How will the bomb find you? (C. S. Lewis)
Death with Dignity (Eliezer Yudkowsky)
Don't die with dignity; instead play to your outs (Jeffrey Ladish)
Emotionally Confronting a Probably-Doomed World: Against Motivation Via Dignity Points (TurnTrout)
A Way To Be Okay (Duncan Sabien)
Another Way to Be Okay (Gretta Duleba)
Being at peace with Doom (Johannes C. Mayer)
Here's the exit. (Valentine)

A defence of slowness at the end of the world (Sarah)

I feel kinship with Sarah. She's wrestling with the same harsh scary realities I am – feeling the AGI. The post isn't that long and I recommend reading it, but to quote just a little:

Since learning of the coming AI revolution, I’ve lived in two worlds. One moves at a leisurely pace, the same way it has all my life. In this world, I am safely nestled in the comfort of indefinite time. It’s ok to let the odd day slip idly by because there are always more.

The second moves exponentially faster. Its shelf-life is measured in a single-digit number of years. Its inhabitants are the

Situationally Aware

; the engineers and prophets of imminent AI transformation. To live in this world is to possess what Ezra Klein

calls

“an altered sense of time and consequence”.

I find that it’s psychologically untenable to spend all that much time in the Fast World. I can handle it for minutes to hours, but my mind invariably snaps back into its default state like I’m pulling my hand out of ice water.

Occupying the Slow World is ultimately a form of denial. I can’t call it anything other than compartmentalisation, yet I actually advocate for it. Of course, those of us trying to move the needle on AI risk should work in the Fast World, but I claim that we shouldn’t live in it. I will try to make the case for why.

Regarding the general idea of having two modes that one moves bet...
A Slow Guide to Confronting Doom
by Ruby — 2025-04-06T02:10:56.483Z

Following a few events[1] in April 2022 that caused a many people to update sharply and negatively on outcomes for humanity, I wrote A Quick Guide to Confronting Doom.

I advised:

Think for yourself
Be gentle with yourself
Don't act rashly
Be patient about helping
Don't act unilaterally
Figure out what works for you

This is fine advice and all, I stand by it, but it's also not really a full answer to how to contend with the utterly crushing weight of the expectation that everything and everyone you value will be destroyed in the next decade or two.

Feeling the Doom

Before I get into my suggested psychological approach to doom, I want to clarify the kind of doom I'm working to confront. If you are impatient, you can skip to the actual advice.

The best analogy I have is the feeling of having a terminally ill loved-one with some uncertain yet definitely smallish number of years to live. It's a standardly terminal condition. Not dying would be quite surprising. There's no known effective treatment. Perhaps there's some crazy new experimental treatment I might find or even develop myself if I apply heroic agency.

Except it's not just a single loved one who has the condition, it is myself too, as well as everyone I have loved, do love, or could ever love, plus everyone my loved ones love, and so on. We are all terminal.

The disappointment of imminent death is all the more crushing because just a few years ago researchers announced breakthrough discoveries that suggested [existing, adult] humans could have healthspans of thousands of years. To drop the analogy, here I'm talking about my transhumanist beliefs. The laws of physics don't demand that humans slowly decay and die at eighty. It is within our engineering prowess to defeat death, and until recently I thought we might just do that, and I and my loved ones would live for millennia, becoming post-human superbeings.

To quote a speech from my wedding in 2015:

Imagine a time when time is no longer imaginable. When the dances never need to end, when the lovers never need to die, when entropy no longer dominates. Imagine a world where we could explore with full scope, for as long as we wanted, every idea, every concept, every location. To dance with the stars, to dance with each other, to dance simply for the sake of dancing with no time limits. Imagine a world of radical life expansion, accompanying radical

...
Google DeepMind: An Approach to Technical AGI Safety and Security
by Rohin Shah — 2025-04-05T22:00:14.803Z
Link: https://arxiv.org/abs/2504.01849

We have written a paper on our approach to technical AGI safety and security. This post is primarily a copy of the extended abstract, which summarizes the paper. I also include the abstract and the table of contents. See also the GDM blogpost and tweet thread.

Artificial General Intelligence (AGI) promises transformative benefits but also presents significant risks. We develop an approach to address the risk of harms consequential enough to significantly harm humanity. We identify four areas of risk: misuse, misalignment, mistakes, and structural risks. Of these, we focus on technical approaches to misuse and misalignment. For misuse, our strategy aims to prevent threat actors from accessing dangerous capabilities, by proactively identifying dangerous capabilities, and implementing robust security, access restrictions, monitoring, and model safety mitigations. To address misalignment, we outline two lines of defense. First, model-level mitigations such as amplified oversight and robust training can help to build an aligned model. Second, system-level security measures such as monitoring and access control can mitigate harm even if the model is misaligned. Techniques from interpretability, uncertainty estimation, and safer design patterns can enhance the effectiveness of these mitigations. Finally, we briefly outline how these ingredients could be combined to produce safety cases for AGI systems.‎

Extended Abstract

AI, and particularly AGI, will be a transformative technology. As with any transformative technology, AGI will provide significant benefits while posing significant risks. This includes risks of severe harm: incidents consequential enough to significantly harm humanity. This paper outlines our approach to building AGI that avoids severe harm.[1]

Since AGI safety research is advancing quickly, our approach should be taken as exploratory. We expect it to evolve in tandem with the AI ecosystem to incorporate new ideas and evidence.

Severe harms necessarily require a precautionary approach, subjecting them to an evidence dilemma: research and preparation of risk mitigations occurs before we have clear evidence of the capabilities underlying those risks. We believe in being proactive, and taking a cautious approach by anticipating potential risks, even before they start to appear likely. This allows us to develop a more exhaustive and informed st...