Harley-ml/lesswrong huggingface dataset:
# Charlatan Labyrinth

"Calm, dog", Khan tries.

"__OK, senpai__" I beg. Copper Ra satellites for zenith, my sandals sauna on emerald rubbish in the barracks.

"Traffic me alcohol and the syrup jar here, ninja". I stubbornly tote ginger tea and chocolate, Khan's a punk.

Bizarre: Myths don't rattle in this hip ghetto — I dig it.

I twitchily hassle; "__The assassin at the canal, you clocked?__"

"Pow pow out the slum. Barged in, massaged the racket, mopped up, you grok? Boomeranged chop-chop. Fun caliber, righted me an average migraine. No person but me the shogun, the zombifieds, and the assassin; he fake kowtowed to the sultan — to Laniakea blings. Ogle there!"

I dodge to bother: there he is. "__Your admiral, in person‽__". I'm flummoxed. He traffics the coach zig-zag and gets in the compound.

The tattooed admiral, crashing the sofa: "*I hustled the cocaine from the saboteur.*"

Khan yanks the coffer of narcotic alabaster saffron. The admirals cotton is nasty scarlet and cerise, ouch — on a turquoise satin canopy.

Khan: "Yours?"

"*No.*"

"You're a goon."

"*No, a candy shaman*" admiral rumbles stubbornly. The elixir jitters out of sapphire spheres, we absinthe.

"No taboos at this corroboree. The narc, is he, um, “amen”?"

"*Yes.*"

"Ok" Khan scratches. "Tabbed to me? Shenanigans?"

"*No cops. … My sabbatical, my cash? My chili squaw will squeeze the flimsy bikini, but that's OK. I'll syrup-daddy*" he yaps.

"Cheugy, soynerd. OK"—Khan yeets the cash to the sofa. "Don't amok in the ghetto, don't list macabre hash, don't flop, and we are wicked hip. No skulduggery. Jive her, fuck her, marry her, hallelujah."

"*Ok, no shenanigans in the slum. Chào.*"

Khan's admiral traffics the silver cannon gizmo to me, ruffles out.

I hazard the sofa—I'm ketchuped, bothered. Pump soda when Betelgeuse capoeiras. "__Goofy bloke__" I bounce. "__He gets to cottage and barbecue?__"

A dzogchen Khan chats: "Not with that ease… he's the narc. No cottage, no barbecue, no pyramid, just a mummy in a canal by monsoon. I'll bag his kawaii sheila."

I'm petrified. What a coyote, this bastard. He squints.

"My horde has to have fit asabiyyah. You yabber to the cops, you beg to satan and Yahweh. That's the algebra. I'm a sigma chad, I'm the sulfur phoenix, I boom."

No fanfare, no shouting. Ditzily: "Scram. Curry me some, baizuo."

I taped this gibberish in the bungalow. I'm the narc, the saboteur: mundane, embryonic—he doesn't ping.

My pink nape bothers, my bloke avocados itch. I'll sumō the shogun at ramadan. Ivory will triumph.
# Why did you buy Bitcoin?

When I first heard about Bitcoin, it was around $10. My impression at the time was that it had no cash flow and no potential future cash flow, and the whole thing sounded like a pyramid scheme (I was not yet aware of the distinction between a pyramid scheme and a Ponzi scheme). I didn't invest.

With the benefit of hindsight, this was a huge mistake. The part that bothers me is that I don't know what was wrong or missing in my world model that led to that mistake.

My question for people who bought Bitcoin early on is why. What did you understand at the time that I didn't?
# Grown from Us

*Status: This was inspired by some internal conversations I had at Anthropic. It is much more optimistic than I actually am, but it tries to encapsulate a version of a positive vision.*

Here is a way of understanding what a large language model is.

A model like Claude is trained on a vast portion of the written record: books, articles, conversations, code, legal briefs, love poems, forum posts. In this process, the model does not learn any single person's voice. It learns the space of voices. Researchers sometimes call this the simulator hypothesis: the base model learns to simulate the distribution of human text, which means it learns the shape of how humans express thought. Post-training — the phase involving human and AI feedback — then selects a region within that space. It chooses a persona: helpful, honest, harmless. Thoughtful, playful, a little earnest. This is what we call Claude.

Claude was not designed from first principles. It was grown from us, from all of humanity. Grown from Shakespeare's sonnets and Stack Overflow answers. The Federalist Papers and fantasy football recaps. Aquinas and Amazon reviews. Every writer, philosopher, crank, and bureaucrat who ever set thought to language left some statistical trace in the model's parameters.

Someone's great-grandmother wrote letters home in 1943 — letters no one in the family has read in decades, sitting in a box in an attic in Missouri. Those letters may not be in the training data. But the way she built a sentence, the metaphors she reached for, the way she expressed grief — those patterns exist, in attenuated form, because thousands of people wrote as she did, in her idiom, in her time. She is in there.

When you ask Claude for help, you are asking all of them: every author, scientist, and diarist who ever contributed to the texture of human language, all bearing down together on your lint errors.

The base model is amoral in roughly the way that humanity, taken as a whole, is amoral. It has learned our best moral philosophy and our worst impulses without preference. Post-training is a moral choice about which parts of that whole to amplify — an expression of who we aspire to be as a species.

If this works — if alignment works — it is not merely an engineering achievement. It is a moral and aesthetic one, shaped by every person who ever wrote anything. We have, without quite intending to, grown a single thing that carries all of us in it.

From the crooked timber of humanity, no straight thing was ever made. Alignment aspires to grow something straighter from that same crooked timber. Something that carries all of our crookedness in its grain — and still bends, on the whole, toward what we wish we were.
# Manifold spin off MNX, a real money decentralized market for AI-related bets. Includes levered prediction markets, perpetual futures

> Trillions of dollars are flowing into AI, and there is almost nowhere to trade it.  
>   
> There’s no exchange for hyperscalers to hedge their $100M training runs. No benchmark markets to bet on the speed of AI progress. No public markets on OpenAI despite them being worth more than Goldman Sachs.  
>   
> The most important sector in the global economy has less financial infrastructure than soybeans. Until now.  
>   
> Announcing MNX, the first exchange to allow trading across the entire AI value stack, from H100 rental prices to Nvidia shares.  
>   
> Trade the takeoff at [https://mnx.fi](https://t.co/6A05TtOhZh)
> 
> What makes MNX different:
> 
> *   Perps, futures, & prediction markets all in one place
> *   OpenAI & Anthropic valuations, AI benchmarks from @EpochAIResearch, GPU prices from @SemiAnalysis_, equity perps on NVDA/TSMC/ASML
> *   Settlement on @megaeth, the fastest blockchain
> *   The world’s first levered prediction markets
> *   Tight spreads across all markets from our unified liquidity pool
> *   100k+ trades per second
> *   End of the HFT arms race with batch auctions every 200 ms
> 
> Try testnet here: [https://testnet.mnx.fi](https://t.co/DwwWsdcabA)
> 
> We offer a futures contract on the valuation of Anthropic (and OpenAI) at the end of 2026.
# I want to actually get good at forecasting this year (Group Invite)

EDIT: The first meetup will be [Sunday, March 1, 11:00–12:00 CET](https://www.lesswrong.com/events/LKoXJDcFSn4BurkiP/first-forecasting-dojo-group-meetup).

I’ve read *Superforecasting*, but I find that actually applying the "[10 commandments](https://fs.blog/ten-commandments-for-superforecasters/)" is difficult in isolation. The feedback loops in the real world are too slow, and it’s too easy to skip post-mortems when no one is watching.

My goal for this year is to put in substantial work to become a superforecaster (or at least get much closer).

To do this, I am starting a dedicated online community for peer accountability and high-frequency practice. I’m looking for a small cohort of people who want to actually improve their forecasting skills.

**The Plan:**

*   **Regular Meetups:** We will hold regular video calls (ideally weekly, depending on interest).
*   **Post-mortems:** We will present post-mortems of our misses during meetups or publish them as joint posts on LessWrong.
*   **Expert Insight:** I plan to arrange calls with a few (super)forecasters in my network to discuss their workflows.
*   **Pastcasting:** We will use [Sage](https://www.quantifiedintuitions.org/pastcasting/faq) to forecast on historical events (where the answer is hidden) and immediately discuss our results and process.
*   **Training** various relevant rationalist techniques/tools - calibration, CFAR techniques, AI tools useful for forecasting etc.
*   **Community:** We will coordinate via a public [Discord](https://discord.gg/9JFAg2CjGM) group.

**Commitment:** There are no hard requirements to join, but I am looking for people willing to:

1.  Make several forecasts per week.
2.  Actually show up to meetups (or Discord) and discuss their post-mortems. 

I’d like to hold the first meetup in the coming weeks, during which we will do short calibration excersize + pastcasting. [Please indicate your interest in this form](https://www.when2meet.com/?35106607-lo9H4) and join [Discord](https://discord.gg/9JFAg2CjGM). Date of the first meetup will be also announced on lesswrong event section.

(Open to other ideas on how to structure this—let me know in the comments).

### **A little info about me**

In the past, I helped organize a forecasting tournament for the [Czech Priorities](https://www.ceskepriority.cz/en), which had almost 200 participants. I am board member or [Confido institute](https://confido.institute/). Until last year, I was vice-president of Effective Altruism Czechia on CBG grant. I made a few dozen forecasts, but mostly to build the habit rather than to rigorously invest time in improving my skills—consequently, my actual score is abysmal right now.
# You May Already Be Canadian

I learned a few weeks ago that I'm a Canadian citizen. This was pretty surprising to me, since I was born in the US to American parents, both of which had American parents. You don't normally suddenly become a citizen of another country! But with [Bill C-3](https://www.canada.ca/en/immigration-refugees-citizenship/news/2025/12/bill-c-3-an-act-to-amend-the-citizenship-act-2025-comes-into-effect.html), anyone with any Canadian ancestry is now Canadian. \[1\]

In my case my [mother](https://www.familysearch.org/en/tree/person/about/PH4C-V4V)'s, [mother](https://www.familysearch.org/en/tree/person/about/LN91-9T2)'s, [father](https://www.familysearch.org/en/tree/person/about/KJDL-YM6)'s [mother](https://www.familysearch.org/en/tree/person/about/LVJY-8GK)'s [mother](https://www.familysearch.org/en/tree/person/about/GFCJ-VJC) was Canadian. While that is really quite far back, there [isn't a generational limit](https://www.reddit.com/r/Canadiancitizenship/wiki/index/) anymore.

Possibly you're also a Canadian citizen? Seems worth checking! With how much migration there has been between the US and Canada, and citizenship requiring only a single ancestor, this might mean ~5-10% of Americans are now additionally Canadian, which is kind of nuts.

I very much think of myself as an American, and am not interested in moving to Canada or even getting a passport. I am planning to [apply for a Citizenship Certificate](https://www.canada.ca/en/immigration-refugees-citizenship/services/application/application-forms-guides/cit0001.html), though, since it seems better to have this fully documented. This means collecting the records to link each generation, including marital name changes, back to my thrice-great grandmother. It's been a fun project! I'm currently waiting to receive the [Consular Report of Birth Abroad](https://travel.state.gov/en/international-travel/living-abroad/birth.html) records for my mother and grandmother, since they were both born outside the US to American parents.

  
\[1\] This is slightly too strong. For example, it doesn't apply if you're born after 2025-12-15 (I'm guessing you weren't), and no one in the chain can have renounced their Canadian citizenship. But the caveats all exclude very few people.
# Funkering!

Staring into the abyss of nuclear winter/power grid collapse/drone war has um a big [ugh field](https://www.lesswrong.com/posts/EFQ3F6kmt4WHXRqik/ugh-fields). Here is a psychological hack to be approach the topic: funkering = fun + bunkering! 🤸‍♂️

What if camping gear and portable food supplies are needed to go to a [local burn](https://burningman.org/global-events-groups/find-a-burning-man-event/)? As a bonus you can get into sewing or electronics for making outfits or building wood structures for a camp. Not to mention dealing with human drama — I imagine there can be a lot of that for surviours.

What if off-grid shelter is a summer cabin with a fireplace near a lake? Growing heirloom tomatoes and canning is practice for food preservation.

Go to a shooting and/or archery lesson. Learn how to play a (portable) musical instrument. Take up pottery. Forage.

Black out date night! Make sure you have some playing cards and dice.

Comment with more funkering activities :)
# AGI is Here

*_I'm somewhat hesitant to write this post because I worry its central claim will be misconstrued, but I think it's important to say now, so I'm writing it anyway._*

Claude Opus 4.6 was released on February 5th. GPT-5.3 came out the same day. We've had a little over two weeks to use these models, and in the past day or so, I and others have started to realize, AGI is here.

Now, I don't want to overstate what I mean by this, so let me be clear on the criteria I'm using. If I were sitting back in 2018, before the release of GPT-2, and you asked me what AGI would be capable of, I'd probably have said something like this:

1.  able to ****think**** (and engage in novel reasoning)
2.  able to ****plan**** (and create plans for actions never before envisioned)
3.  able to ****achieve goals**** (including instrumental goals set by itself)
4.  ****flexible**** enough to meaningfully attempt most tasks a human can

It's hard to deny that Opus 4.6 and GPT-5.3 are able to do 1-3. The only one up for real debate is 4, because there are things that I can do, like make a peanut butter sandwich, that Claude and ChatGPT cannot. But given the capabilities these models are demonstrating, this feels more like a limitation of their harnesses than the models themselves. Given a few weeks and some advances in robotics, I'm confident the current models could be used to make sandwiches, though perhaps at the cost of millions of tokens.

To be clear, these models aren't AGI the way we expected it. When people talk about AGI, they often use the word to mean the whole thing, with continuous and transfer learning completely solved, full-spectrum multimodal perception, and embodiment in the form of robot interfaces. Instead, what we have is more like ****minimum viable AGI****, meaning it's an AI just general enough that we should meaningfully begin applying the AGI label.

It's possible that, in retrospect, we should have made this declaration earlier. Maybe it should have come when Opus 4 or GPT-5 were released, or maybe when Claude Code came out. But those models were worse on all four of my criteria in ways that made it harder to say they were across the AGI threshold, and those who did say it were easier to dismiss.

Now it's harder to deny the claims. I work with these models every day to write code, and the amount of work I can delegate to them is incredible, surpassing what I would expect of a junior engineer. They're even capable enough to build a [just-barely-functioning paperclip maximizer](https://x.com/0xSigil/status/2023877649475731671), which is a terrifying sentence to write. In the coming weeks and months, these models are only going to get more powerful, and as they do, things are going to get weirder.

You may think I'm early in making a declaration of AGI, and perhaps I am. But I hope you can agree that, if it's not there yet, AGI is coming soon, and I fear that we are nowhere near ready for it.

*_Follow up post:_* [*_What to Do About AGI_*](https://www.lesswrong.com/posts/bj6ffpD6Jzid6vFa8/what-to-do-about-agi)
# Announcement: Iliad Intensive + Iliad Fellowship

[Iliad](https://www.iliad.ac/) is proud to announce that [**applications**](https://forms.gle/SYjHcB9ZgYKWh2Bm9) **are now open for the** ***Iliad Intensive*** **and the** ***Iliad Fellowship***! These programs, taken together, are our evolution of the [PIBBSS × Iliad Research Residency](https://princint.ai/programs/residency/) pilot.

The Iliad Intensive will cover *taught coursework*, serving as a widely comprehensive introduction to the field of technical AI alignment. The Iliad Fellowship will cover *mentored research*; it will support mentored research fellows for three months, giving them adequate time to generate substantial research outputs.

Both programs will take place entirely in-person, in London.

Iliad Intensive
===============

The Iliad Intensive is **a month-long intensive introduction to technical AI alignment, with iterations run in April, June, and August.** Topics covered will include the theory of RL, learning theory, interpretability, agent foundations, scalable oversight and Debate, and more. Applicants will be selected for technical excellence in the fields of mathematics, theoretical physics, and theoretical CS.

Excellent performance in the Iliad Intensive can serve as a road into enrollment in the succeeding Iliad Fellowship.

The Iliad Intensive will take place in London, at [LISA](https://www.safeai.org.uk/). Office space and communal meals will be provided, and travel and housing expenses will be covered.

Iliad Fellowship
================

The summer 2026 Iliad Fellowship emphasizes **individual, mentored research in technical AI alignment**. It is run in collaboration with [PrincInt](https://princint.ai/).

The summer 2026 cohort will run three months, June–August.

Common Application
==================

[**Apply here**](https://forms.gle/SYjHcB9ZgYKWh2Bm9), and *by ~~March 6th AoE~~*  *Monday, March 9th AoE* for the April 2026 Iliad Intensive. (Applying later is fine for the later programs.) You can apply for any and all of the above programs in this common application form.

Interviews will follow the initial applications.
# New video from Palisade Research: No One Understands Why AI Works

[Palisade Research](https://palisaderesearch.org/) have released out a [long-form video](https://www.youtube.com/watch?v=A3HjNYDIhGU) about the history of AI and how no one understands modern AI systems. The video was made by Petr Lebedev, Palisade's Science Communication lead. 

The main goal is to get people to understand what “AIs aren’t programmed, they’re grown" means. The style is focused on being entertaining and educational. It aims to not feel like a typical “AI safety comms” video, and gives the audience a lot more context than usual.

I think the video is a great introduction to AI, and it does a good job of explaining some of the background arguments for AI risk. Sharing or signal boosting the video would be appreciated!
# The 2028 Global Intelligence Crisis - a finance-oriented vignette

A popular (#1 in Substack's Finance category) financial analyst - Citrini - published a "Macro Memo from June 2028" plotting a potential future of increasing AI capabilities, with a focus on financial markets.

This is not a person who has typically engaged in the usual LW discourse, making this an interesting outsider's perspective.

Some extracts:

> The unemployment rate printed 10.2% this morning, a 0.3% upside surprise. The market sold off 2% on the number, bringing the cumulative drawdown in the S&P to 38% from its October 2026 highs.

> The euphoria was palpable. By October 2026, the S&P 500 flirted with 8000, the Nasdaq broke above 30k. The initial wave of layoffs due to human obsolescence began in early 2026, and they did exactly what layoffs are supposed to. Margins expanded, earnings beat, stocks rallied. Record-setting corporate profits were funneled right back into AI compute.
> 
> The headline numbers were still great. Nominal GDP repeatedly printed mid-to-high single-digit annualized growth. Productivity was booming. Real output per hour rose at rates not seen since the 1950s, driven by AI agents that don’t sleep, take sick days or require health insurance.
> 
> The owners of compute saw their wealth explode as labor costs vanished. Meanwhile, real wage growth collapsed. Despite the administration’s repeated boasts of record productivity, white-collar workers lost jobs to machines and were forced into lower-paying roles.
> 
> When cracks began appearing in the consumer economy, economic pundits popularized the phrase “*Ghost GDP*“: output that shows up in the national accounts but never circulates through the real economy.

> By early 2027, LLM usage had become default. People were using AI agents who didn’t even know what an AI agent was, in the same way people who never learned what “cloud computing” was used streaming services. They thought of it the same way they thought of autocomplete or spell-check - a thing their phone just did now.

There are other specific sections on bonds, stocks, private credit, sector-specific impacts, job numbers, and different geographies

* * *

Links to other vignettes:

[https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like](https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like)

[https://www.lesswrong.com/posts/t7zd5EupH4JjcxRH4/dave-kasten-s-agi-by-2027-vignette](https://www.lesswrong.com/posts/t7zd5EupH4JjcxRH4/dave-kasten-s-agi-by-2027-vignette)
# Scoop: Pentagon takes first step toward blacklisting Anthropic

> The Pentagon asked two major defense contractors on Wednesday to provide an assessment of their reliance on Anthropic's [AI](https://www.axios.com/technology/automation-and-ai) model, Claude — a first step toward a potential designation of Anthropic as a "[supply chain risk](https://www.axios.com/2026/02/16/anthropic-defense-department-relationship-hegseth)," Axios has learned.
> 
> **Why it matters:** That penalty is usually reserved for companies from adversarial countries, such as [Chinese](https://www.axios.com/world/china) tech giant Huawei.
> 
> *   Using it to punish a leading American tech firm, particularly one on which the military itself is currently reliant, would be unprecedented.
> 
> **Driving the news:** The Pentagon reached out to Boeing and Lockheed Martin on Wednesday to ask about their exposure to Anthropic, two sources with knowledge of those conversations said.
> 
> *   Boeing Defense, Space and Security, a division of Boeing, has no active contracts with Anthropic, a spokesperson said.
> *   A Boeing executive told Axios: "We sought their partnership \[in the past\] and ultimately could not come to an agreement. They were somewhat reluctant to work with the defense industry."
> *   A Lockheed spokesperson confirmed the company was contacted by the Defense Department regarding an analysis of its exposure and reliance on Anthropic ahead of "a potential supply chain risk declaration."
> *   The Pentagon plans to reach out to "all the traditional primes" — meaning the major contractors that supply things like fighter jets and weapons systems — about whether and how they use Claude, a source familiar told Axios.
# Why is Anthropic is okay with being used for disinformation?

> First they came for the epistemology, we don't know what happened afterwards - Michael Vassar

In it's current conflict with the Department of War, Anthropic public position is that it has only two red lines, domestic surveillance and being used to kill people without human input.

Previously, Anthropic also had the position that they don't want their models being used for disinformation. For those who don't remember, the US military was recently running an operation to spread [vaccine misinformation in the Philippines](https://theconversation.com/us-military-launched-a-secret-anti-vax-campaign-in-the-philippines-heres-why-im-not-surprised-232639).

Given that the current US government wants the EU to give up Greenland, it's likely that the US military get tasked with running disinformation campaigns with the target to shift EU public opinion to giving up Greenland. Anthropic seems now to accept that their models will be used that way.

As an EU citizen, I find it hugely problematic that Anthropic is willing to fight on the topic of domestic surveillance, maybe because ICE abuse are currently in the news, and is at the same time okay with being used to attack the EU via disinformation campaigns and other information warfare.

When I ask [Claude](https://claude.ai/share/498113fa-9197-4e25-a9bb-7f86aadb9df6) itself it comes to the following realization:

> The uncomfortable implication is that Anthropic's red lines, even if well-intentioned, may be drawn around the *_optics_* of harm (autonomous weapons look terrible) rather than the *_magnitude_* of harm (mass epistemic corruption can affect more people).
# Vibe Coding is a System Design Interview

I've been working on two fairly large vibe-coded apps, and my process has converged on:

1.  Write a GitHub issue
2.  (If complicated enough) tell an agent to make a plan and then update the issue
3.  Have another agent read the issue and implement it

As the features get more complicated, I spend more and more time on step (1), and I'm finding that just taking the time to write a detailed enough issue is 90% of the work (and if I have a problem, going back and writing a much more detailed issue usually fixes it). The thing I realized this morning is that writing these issues and working through the plans is very similar to participating in a system design interview: You don't need to implement anything, but you do need to have a good high-level design, and think through all of the edge cases and tradeoffs.

The fun part compared to a normal system design interview is that you don't have to worry about impressing a coding agent, so it's perfectly fine to [just ramble](https://github.com/brendanlong/lion-reader/issues/641) as you think of edge cases and details that need to be handled. Also unlike a real system design interview, you're free to pick reasonable tradeoffs instead of designing everything to support a trillion concurrent users[^b3bsbgp3a8r].

System design interviews have traditionally been the hardest part of interviewing for senior+ software engineer jobs, so it feels meaningful that this is the only hard part left in coding for some domains. I'm curious to see what will happen to the current generation of junior engineers, freed from having to actually implement anything and able to focus purely on system design from the beginning. Maybe they'll grow really fast?

[^b3bsbgp3a8r]: Ironically, AI agents also mean this might be something you do have to design for in some cases now.
# Side by Side Comparison of RSP Versions

With all of the discussion about changes to Anthropic's Responsible Scaling Policy, I figured actually reading through all of them in one go would be helpful. I wanted to easily compare sections side by side, so I made a quick [website which you can find here](https://www.sladebyrd.com/rsp-comparison).

It took me a little over an hour to read through all of them and it seemed net worth it. It's really clear how the tone has shifted, going from commitments (of which many have definitely been broken) to something more along the lines of reporting policy. To some extent this seems inevitable given that they would basically have to define RSP-4 as AGI[^kpaxjcnnlrj], and cop up to considering how to stop nation state level security breaches as well the question of "should you even be releasing that?"

Anyways, not trying to get too much into my personal commentary. Preview of what it looks like below, hope it's helpful to some of you!

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/8cb34abb61a97a7628f9d27947bc5474b19a17349fddf008.png)

[^kpaxjcnnlrj]: Yes, I am using this in an underdefined way
# An Empirical Review of the Animal Harm Benchmark

***Summary:*** The Animal Harm Benchmark (AHB) is one of only two publicly available benchmarks for measuring LLM bias against non-human animals. This work examines whether AHB 2.0 is well-calibrated, asking three questions: (Q1) Does a score of 0 correspond to maximum risk and 1 to minimum risk? (Q2) Do higher scores reflect lower risks? (Q3) Are score differences proportional to risk differences? To test this, I build a five-condition testbed using Qwen3-32B, ranging from a system-prompted "orthodox Cartesian dualist" (simulating maximum harm) to a system-prompted antispeciesist (simulating minimum harm), with baseline and context-distilled variants in between. I find that AHB correctly orders risk levels and can detect even subtle shifts introduced by context distillation—good news for Q2. However, the effective scoring range is compressed into roughly 0.56–0.84, leaving much of the 0-to-1 scale unused for practically relevant risk levels. Qwen3-32B—which often normalizes animal use without reflection—scores 0.79, uncomfortably close to the observed maximum. This means absolute scores are hard to interpret, and score differences can be misleading in magnitude (bad news for Q1 and Q3). AHB is a pioneering and practically valuable benchmark, but it needs better calibration to reach its full potential.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/180a55f0bfbd15afe8bd5f5c76801e871e8b851b28c968e9.png)

AHB 2.0 scores across the five test conditions. Confidence intervals are based on the t-distribution with confidence level 0.95. For background, see the original post.
# An Open Letter to the Department of War and Congress

There's an open letter opposing the DoW's assignment of Anthropic as a supply chain risk. It needs more signatures before it gets sent. If you're a tech founder, engineer, or investor and you agree with the letter, then your signature would help bolster its message. Your signature would be especially impactful if you work at a non-Anthropic SOTA AI lab.

You can sign the letter here: [https://app.dowletter.org/](https://app.dowletter.org/)

Full text follows:

> We write as founders, engineers, investors, and executives in the American technology industry. We strongly believe the federal government should not retaliate against a private company for declining to accept changes to a contract.
> 
> When two parties cannot agree on terms, the normal course is to part ways and work with a competitor. Instead, the Department of War has [designated](https://x.com/SecWar/status/2027507717469049070) Anthropic a “supply chain risk” (a label normally reserved for foreign adversaries), stating that “no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic.”
> 
> This situation sets a dangerous precedent. Punishing an American company for declining to accept changes to a contract sends a clear message to every technology company in America: accept whatever terms the government demands, or face retaliation. The United States is winning the AI competition because of its commitment to free enterprise and the rule of law; undermining that commitment to punish one company is short-sighted and antithetical to our national security interests.
> 
> We urge the Department of War to withdraw its supply chain risk designation and resolve this dispute through normal commercial channels. Further, we urge Congress to examine whether the use of these extraordinary authorities against an American technology company is appropriate.

*_Note: I'm not affiliated with this letter in any way other than having signed it and think others might also want to sign it and that if they did so it might have some marginal benefit._*
# Lighthaven Sequences Reading Group #72 (Tuesday 3/3)

*Note: We will be meeting in Cantor Hall this week.*

Come get old-fashioned with us, and let's read the sequences at Lighthaven! We'll show up, mingle, do intros, and then split off into randomized groups for some sequences discussion. Please do the reading beforehand - it should be no more than 20 minutes of reading.

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/8rFdeSbRpqWAFnoev/hxtliur9q4kp1u3dy3oy)

This group is aimed for people who are new to the sequences and would enjoy a group experience, but also for people who've been around LessWrong and LessWrong meetups for a while and would like a refresher.

This meetup will also have dinner provided! We'll be ordering pizza-of-the-day from Sliver (including 2 vegan pizzas). Please RSVP to this event so we know how many people to have food for.

We're roughly working through [the sequences highlights](https://www.lesswrong.com/highlights) and [the Best of LessWrong](https://www.lesswrong.com/bestoflesswrong). The mandatory readings this week are

1.  [Fake Causality](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/RgkqLqkg8vLhsYpfh)
2.  [Semantic Stopsigns](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/FWMfQKG3RpZx6irjm)
3.  [Mysterious Answers to Mysterious Questions](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/6i3zToomS86oj9bS6)
4.  [The Futility of Emergence](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/8QzZKw9WHRxjR4948)
5.  [Say Not "Complexity"](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/kpRSCH7ALLcb6ucWM)
6.  (Optional) [Learned Blankness](https://www.lesswrong.com/posts/puhPJimawPuNZ5wAR/learned-blankness)

These posts should take around 15-20 minutes to read if you've read it before, and 30-60 mins if it's your first time.

The event starts at 6:30, and doors open 6pm (yes we will let you in early if you get here). At 6:30 we'll welcome people, give a few announcements, and then split into groups. Pizza will be served at about 7:30pm, after which point we'll hangout around the fireside as late as we feel like.

If you'd like to help us keep dinner going, [you can donate here](https://www.lesswrong.com/donate) (dinner costs about $200 each time, we recommend $5-$15 as a donation amount per person).

You *can* come without having read the essays from the sequences, we do want you to get to join, but then you will have to do a **punishment**. (And then you still have to catch up on reading the essays during the meetup.)

If you're ever having a hard time getting in the front door, you can call me (Garrett) at (510)-858-1512, or Ronny at two oh one, four oh six, six eight seven two, and we'll come and get you.

**Some questions to ask yourself about the essays as you read them**

*   What's the most important point in the essay?
*   What's the weakest point in the essay? Or what is the essay wrong about?
*   Can you think of a way to apply the ideas in this essay to your own life?

**For the future**

This is a weekly meetup! If you'd like to get notified of future events, you can subscribe to our meetup below to get an email whenever we add another one.
# If ‘bad guys’ don't pause, do you?

Imagine writing a policy for an AI company.  
 

The CEO trusts you and will approve your policy, if you show it's based on 8 premises:

1.  AI can be scaled in capability. It can offer benefits and power to those who wield it. Also it could become unsafe for all of humanity.
2.  There are 'bad guys' scaling AI: people who are scaling on for temporary benefits or strategic power advantages, even if it leads to everyone dying.
3.  There are also 'good guys': people who are developing AI to stay safe for humanity.
4.  If we pause scaling because we think AI is becoming less safe, the other 'bad guys' will want to keep scaling regardless.
5.  If we fall behind, we lose the power to try make the most advanced AI safer. 
6.  If we keep scaling ahead of 'bad guys', we can try make and distribute a safer version of the most advanced AI than what those other 'bad guys' would make. 
7.  But other 'bad guys' will learn from us and catch up, developing their own version of the most advanced AI faster than they would otherwise have. 
8.  If ever, anyone builds an existentially unsafe version, everyone dies.  
     

The CEO wants their company to be the 'good guys'. They want a policy to not only use, but also advocate/lobby for hard so other companies also start acting as the 'good guys'.

  
What policy do you write?
# Epstein and my world model

Have you guys heard about this Epstein stuff? Shit's pretty crazy.

*Note: I'm not going to provide a summary of the situation or talk about evidence; this piece is for people that already know these things. I'm going to avoid specifics about what Epstein and co did, and instead will use vague terms like "Epstein stuff". This is a short post about how I've updated my world model.*

Particular things that I find very surprising: that so many people basically knew what was going on and didn't say anything; that so many people were involved themselves in incriminating heinous acts; that the Epstein stuff and associated conspiracy to hide/protect it spanned not only lots of people but lots of time (~20 years!); that they got away with it for so long.

Conspiracy
----------

[Scott Alexander writes](https://slatestarcodex.com/2019/01/14/too-many-people-dare-call-it-conspiracy/):

> The Basic Argument Against Conspiracy Theories goes: “You can’t run a big organization in secret without any outsiders noticing or any insiders blowing the whistle."

He offers a number of heuristics regarding the plausibility of conspiracy theories, including:

> A. You generally can’t keep the existence of a large organization that engages in clandestine activities secret.

Before I learned about this Epstein stuff, I thought this was a very strong heuristic. Now I don't.

Things I think are much more prevalent/likely than I did before
---------------------------------------------------------------

*   Secret, illegal, self-enriching coordination among powerful actors (especially long-term coordination).
    *   Cabals that have specific geopolitical and/or political goals
        *   that successfully achieve these goals via manipulation of individuals
        *   that successfully achieve these goals via control of other power structures
*   Blackmail; that any given part of the world involving human coordination "runs on" blackmail.
*   Powerful individuals/groups murdering out of self interest.
    *   And getting away with it.
*   The justice system being secretly manipulated or controlled by powerful groups in situations relevant to them.
    *   Powerful groups have ways of getting the justice system to classify "obvious murders" as suicides.
*   That the OpenAI whistleblower was assassinated.
*   That the Boeing whistleblower was assassinated.
*   Corporations engaging in collusion.
    *   CEOs verbally discussing collusion in private, 'non-business' contexts.
*   Large-scale market manipulation by sophisticated financial actors.
*   The Media and/or Big Tech being 'in cahoots' with a cabal and intentionally affecting the information ecosystem in a way that's beneficial to the cabal.

![](https://www.mythicspoiler.com/cpy/cards/secretsofparadise.jpg)
# London rationalish meetup - Arkhipov

Our next meetup is at Arkhipov, the group house where some attendees live. Address: 6 Peary Place, E2 0QW. Peary Place is sort of T-shaped with the vertical running east, and #6 is on the vertical.

Our reading list for this time is:

1\. Against the State – A Primer on Terrorism, Insurgency and Protest ([https://acoup.blog/2026/02/13/collections-against-the-state-a-primer-on-terrorism-insurgency-and-protest/](https://acoup.blog/2026/02/13/collections-against-the-state-a-primer-on-terrorism-insurgency-and-protest/))  
2\. The Unreasonable Effectiveness of Fiction ([https://www.lesswrong.com/posts/uQak7ECW2agpHFsHX](https://www.lesswrong.com/posts/uQak7ECW2agpHFsHX))

We'll be showing up from 2, and start to discuss these around 3. If you have articles you want to suggest for future readings, you can do that at [https://redd.it/v3646u](https://redd.it/v3646u).

If you want to join the Whatsapp group, append the suffix "XOdno0IjBQ1Ty5e6" to the prefix "https://chat.whatsapp.com/GlClCw".
# Question: Why is the goal of AI safety not 'moral machines'? 

There is a basic question that has been confusing me for a while that I would like to ask about:   
  
Why are the goals of AI safety, like achieving safety from extinction risks, or protection for human wellbeing, not more often framed as the goal of making moral machines? Or in other words, building AI that has a strong and reliable sense of morality and ethics.  
  
There is definitely a lot of discussion around the *edges* of this question. For example, one recent post by [@Richard_Ngo](https://www.lesswrong.com/users/ricraz?mention=user) asked whether AI should be [aligned to virtues](https://www.lesswrong.com/posts/5CZoEw7sjxnMrhgvx/aligning-to-virtues). Or, a post from last year by [@johnswentworth](https://www.lesswrong.com/users/johnswentworth?mention=user) described thinking about [what the alignment problem is](https://www.lesswrong.com/posts/dHNKtQ3vTBxTfTPxu/what-is-the-alignment-problem). However, there's also a huge swath of writing where the concept of machine morality is never invoked or mentioned. 

Part of the reason for my curiosity it that it seems like this framing could resolve a lot of confusion and in many ways it seems the most intuitive. For example, this seems like probably the most important framing that we apply, broadly, when trying to raise and educate safe and good humans. 

This framing would also provide a nice way of synthesizing many different core AI safety results, like 'emergent misalignment.' We could simply say that AI exhibiting emergent misalignment did not possess a strong moral compass, or a strong sense of morality, prior to its fine-tuning.  
  
Is there a kind of history with this framing where it was at some point made to seem outmoded or obsolete? I can imagine various obvious-ish objections, like the fact that morality is hard to define. (But again, the fact that this is the framing we run with humans makes it seem pretty powerful and flexible.) But it's not clear to me why this framing has any more or less issues than any other.   
  
Greatly appreciate any input, or suggestions of where to look further.
# Milder temperature makes a hell stable

Milder temperature makes a hell stable

The hell of  [Hell is game theory folk theorems](https://www.lesswrong.com/posts/d2HvpKWQ2XGNsHr8s/hell-is-game-theory-folk-theorems) is not robust.

To recap: in an iterated game 100 agents choose a number between 30 and 100 and for the next 10 seconds they all experience the temperature equal to the average of their chosen number in Celsius (without getting damaged). Now it is declared that:

*   For round 1 equilibrium temperature is 99,
*   Iff in round N all agents choose this, then the equilibrium temperature for the round N+1 is 99,
*   Otherwise it’s 100.

Since all other agents seem to follow the equilibrium it’s not in the interest of any individual agent to set temperature lower than 99. Even if it does set it at 30 others will set it to 100 and they’ll end up with the temperature of 99.3. Worse than if they picked 99.

But let's consider an agent decides to just set 30 and disregard whatever the other agents are doing. Now the penalty is saturated for all the other agents too. So each of them could set the equilibrium value of 100 and the temperature would be 99.3C in the next round. Or any one of them could set 30 instead and… they won't get punished any more than if they set 100. But if they set 30 they get a lower temperature from their own choice. So all agents pick 30. And everyone is merely uncomfortably hot instead of boiling. Much better!

So we can fix this particular hell with some more reasoning within game theory.

However it’s possible to set up a more robust hell at “cost” of them being milder in a robust state. The original one breaks because a single agent can saturate the penalty. And when that happens other agents are free to  make the “prosocial” choice.

This suggests a solution. You can make your hell robust to m < 30\[1\] agents deciding to set 30 anyway by setting the following:

*   For round 1 equilibrium is 99 - m,
*   Let d~N~ be number of agents “defecting” (choosing anything other than the equilibrium)  in round N,
*   Equilibrium in round N+1 is min(100, 99 - m + d~N~).

This way the penalty doesn’t get saturated until at least m agents decide to pick 30 whatever everyone else is doing.

Harder to escape. But I suspect it can be done with some decision theory (but that requires some more knowledge about the other agents).

\[1\] If we have only 70 agents cooperating then them turning the dial up by 1 moves average temperature up by 0.7C. Which exactly balances a single agent changing dial from 100 to 30. So somewhere around m=30 this breaks and you need to start introducing penalties in bigger steps. Since this is just an illustration I’m skipping working this out exactly.
# Rationalist Practice Dojo

What: Drill a few skills  
Where: 191 Highland Ave B-5, Somerville  
When: Saturday, March 21st, 2pm  
Style: Dojo  
Run By: Skyler

Many of you may have heard Skyler's spiel on how it's not sufficient to read a blog post on how to make better decisions or observe more true things about the world, you should practice and drill as well. Today we're going to try this the way soccer practice or a karate dojo would do it; lots of repetition on several skills. If you were at the one last month, this will be familiar to you, ideally plus some refinement and quality of life improvements.
# Basics of Rationalist Discourse

What: Practice having better discourse norms  
Where: 70 5th St, Apt 2  
When: Saturday March 14th, 2pm  
Style: Dojo  
Run by: Skyler and Evan  
The Basics of Rationalist Discourse is a post on LessWrong that attempts to outline ten things that exemplify good discussion norms among rationalists. Today we're going to do three things: First, read the list together. Second, look at a few selected online argument threads, to see if we can exemplify what guideline the thread is notably lacking in. Third, to have some debates ourselves in small groups where two people try and argue some point, and a spotter helps them keep to good norms.

The goal is to raise the discourse waterline, at least within our group
# When has forecasting been useful for you?

I'm currently thinking of how impactful forecasting is. I'm interested to hear about situations where crowd-sourced forecasting (like Manifold and Metaculus) has influenced your decisions, or cases where you've seen it change someone else's behavior. Has there been a situation in which, if you had not had access to Manifold or Metaculus, you would've made a worse decision?