trentmkelly/LessWrong-43k huggingface dataset:
Procedural vs. Causal Understanding
by Caleb Biddulph — 2025-05-29T07:00:43.326Z

When you explain your strategy for solving problems in a certain domain, you can try to convey two different types of understanding:

Procedural understanding is about how to execute a strategy.

When you've effectively explained your procedural understanding to somebody, they can solve a wide variety of problems in the relevant domain in the same way that you would.

Causal understanding is about why a strategy helps solve the problem.

When you've effectively explained your causal understanding to somebody, they know the reasons that you believe the strategy works.

It's possible to have causal understanding without procedural understanding. For example, I know that the correct strategy for tightrope-walking works by keeping one's center of gravity steady above the rope, but that doesn't mean I know the right techniques or have the muscle memory to actually do it.

It's also possible to have procedural understanding without causal understanding. For example, I have no idea how ibuprofen works, but I expect that my procedural understanding that "when in pain, taking ibuprofen relieves that pain" could help me accomplish the goal "avoid being in pain."

I'll call an explanation that conveys procedural understanding a "procedural explanation" and one that conveys causal understanding a "causal explanation."

Often, the most effective procedural explanation of a strategy will include a causal explanation. For example, teaching a chess novice how to execute the Sicilian Defense will probably help them improve at chess. But their success will be limited until they gain a deeper causal understanding of why that opening is effective, letting them adapt the strategy to unforeseen situations.

The more robustly you need to apply a strategy, the more useful it becomes to have a good causal understanding. To get better at avoiding pain in more and more situations, I could try to gain more and more procedural understanding about ibuprofen: the kinds of pain that ibuprofen won't help, what substances have a similar effect, even how to synthesize ibuprofen myself. But at a certain level of detail, the most efficient procedural explanation will probably include a causal explanation of how ibuprofen's molecular structure interacts with other chemicals in the body to reduce pain.

When attempting to explain the strategies learned by an AI during training,[1] we'd rather have a causal expl...
Quick Minimal Playhouse
by jefftk — 2025-05-29T02:10:01.985Z

When I came home from work today Lily was very excited: she wanted to
build a playhouse for our 1.11yo neighbor Jo. Lily had gotten
parental permission, the yard had space, and I was the only remaining
obstacle. I'd normally be pretty excited about a project like this,
but it was 5pm and I was signed up to cook dinner.

On the other hand, we'd recently been reading the Little
House books and Farmer Boy and
we'd been joking about how the fathers in the books built things
unrealistically quickly. Most recently, Almanzo's father builds
a bobsled from scratch in a single a day, including identifying
and felling the trees. Still, most of the unrealism comes from what
they had to work with: no pre-cut pre-seasoned lumber, no power tools,
pegs for fasteners, etc. Perhaps with modern tools we could build a
minimal playhouse together and still have dinner on the table by 7pm?
Sounds fun!

A key component here is that I already had everything I needed (DIY Pantry
Staples) on hand from past projects:
plywood, 2x3s and 2x4s, screws, wood glue, plastic sheeting, staples,
saw, drill, belt sander.

I decided to make an open lean-to, though three
sides would be partly blocked by bracing. I found a piece of plywood
about the right size, and glued and screwed 2x3s to the outside edge.
Lily and I cut vertical supports at a 22.5° angle, 42" for the
back two and 60" for the front two. We carried it all outside and
assembled the rest upside down. Nora helped glue on the supports:

Once the supports were in place it needed diagonal braces or it was
not going to be sturdy enough. Lily helped me cut these as well, and
we glued and screwed these on. I sanded any sharp corners, and then
stapled plastic sheeting over the top.

Here's a 3.11yo for scale:

And an 11yo:

Possibly we'll paint it, but if not it's done!

It took ~1hr20min from end to end. The longest step was probably when
I needed to get a specific piece of wood out from under a large pile
of other lumber. And of course this speed was only possible because
of the simple design, having the materials and tools on hand, and the
speed of power tools.

It was fun, and the kids and I are looking forward to giving to to Jo!

(I made sure to publish this tonight before going to bed, because
otherwise Julia will see the pictures on my camera roll and worry that
I'm abusing her abse...
Cognitive Exhaustion and Engineered Trust: Lessons from My Gym
by Priyanka Bharadwaj — 2025-05-29T01:21:38.929Z

Epistemic status: Exploratory design philosophy. This post reframes AI safety as an affordance problem, not just a control problem. It draws from embodied systems and interaction design to argue that trustworthiness in AI arises less from post-hoc constraints and more from architectures that shape behaviour upstream. I introduce the early-stage concept of Distributed Neural Architecture (DNA), which I’ll unpack in a future post.

--

I’ve been going to the same gym since 2019. It used to be a space of calm focus with familiar equipment, consistent routines, and the comforting hum of shared norms. I knew where everything was, who to ask, how to move. It was the kind of place where my body could work, and my mind could rest.

Then everything changed.

New members arrived in waves, coaches rotated weekly, classes started colliding, and dumbbells were scattered like tripwires across the floor. It wasn’t just messy. It was mentally exhausting. Every workout now began with a background process of risk assessment. Was the bench free? Could I finish my set before the next group flooded in? Would someone walk behind me mid-lift?

I wasn’t thinking about my form, I was constantly scanning for threats. Hyper vigilance had replaced flow. And yet, most people seemed fine with it. They tuned it out. They adapted. I couldn’t. Eventually I realised why.

The environment had broken a tacit promise of safety I had come to expect, not just physical safety, but cognitive and emotional safety. The kind that lets your mind rest because the environment carries part of the load.

What I miss isn't just order, it is affordance (from ecopsychology). A subtle contract that once existed between space and behaviour, now completely broken. The safety I had taken for granted wasn’t about rules. It was about rhythm, flow, and not having to think so hard. I realise I am not just physically unsafe, I feel cognitively taxed.

To me, this is exactly what bad AI design also feels like.

The Toyota Contrast

Years ago, I worked at the Toyota factory in India. I'd work everyday on factory floors filled with moving machinery, sharp edges, and actual risk. But the strange thing is, I never felt the kind of low-level vigilance I feel at my gym today. Why?

Because the environment was doing half the cognitive work for me. Walkways were painted clearly in green. Warning zones had visual and tactile cues. ...
Truth or Dare
by Duncan Sabien (Inactive) — 2025-05-29T00:07:50.397Z

Author's note: This is my apparently-annual "I'll put a post on LessWrong in honor of LessOnline" post. These days, my writing goes on my Substack. There have in fact been some pretty cool essays since last year's LO post.

Structural note:
Some essays are like a five-minute morning news spot. Other essays are more like a 90-minute lecture.

This is one of the latter. It’s not necessarily complex or difficult; it could be a 90-minute lecture to seventh graders (especially ones with the right cultural background).

But this is, inescapably, a long-form piece, à la In Defense of Punch Bug or The MTG Color Wheel. It takes its time. It doesn’t apologize for its meandering (outside of this disclaimer). It asks you to sink deeply into a gestalt, to drift back and forth between seemingly unrelated concepts until you start to feel the way those concepts weave together, rather than just abstractly summarizing the interactions between them.

(There’s a big difference between the type of hike where you are trying to get from point A to point B, and the type of hike where the whole point is to embed yourself into the territory and be somewhere for a while. To let being-in-that-place do to you whatever it ends up doing.)

You certainly don’t have to set aside a single, uninterrupted afternoon to get value out of this piece—feel free to nibble away at it in chunks over the course of a week, treating each section like its own standalone essay. But the value is largely in the whole. Perhaps think of it as being like a movie, or an album, in the way that the experience of a movie or an album is fundamentally different from the experience of a Wikipedia synopsis. The thing I am trying to say here may well be compressible and summarizable, but I’m not trying to offer the summary. I’m trying to offer the thing itself. If I thought I could accomplish my aim with fewer words, I would have; I genuinely expect people who skip and skim to be underwhelmed in direct proportion to the shallowness of their engagement.

Sip and savor, as recommended, or do the other thing if that seems better. Make good choices, in accordance with your values.

0. Introduction

I received the above request while out and about a few weeks ago, and proceeded to spend most of my drive home narrating a response via voice-to-text. I ended up with an off-the-cuff list of about forty items, and I was pretty proud of it, s...
What should I read to understand ancestral human society?
by Lorec — 2025-05-28T23:36:32.231Z

I recently got into a chatroom debate with some very smart people about whether homosexual behavior was typical for humans [ not bonobos ] in the ancestral environment / Environment of Evolutionary Adaptedness [EEA] [ i.e., ancestral hunter-gatherer society ] -- and, connectedly, how big was the social circle that the median ancestral human probably had occasional contact with [ not "Dunbar's number", but "how big is your meta-tribe ].

I felt like I knew what I was talking about . . . but not nearly as much as I would like to. For example, someone linked this study, which claims the egalitarian hunter-gatherer model is missing important pieces and stratified societies existed prior to agriculture. I doubt this, but I feel like the basis on which I doubt it is shakier than it needs to be.

Then I found this article, which says:

Whereas the chimpanzee community is only partially a closed reproductive unit (Wrangham 2000), the larger size of hunter-gatherer communities (typically 250-500 in contrast to 40-150 among chimpanzees) renders it a more adaptive breeding isolate (cf. Wobst 1974) and intracommunity marriages play a crucial role in cementing relationships between bands.

This seems plausible, and would affect my framework for thinking about psychology greatly, but I'd never read about it before [specifically the "dating-pool communities tended to be 250-500 members big" part]! Who are the brilliant theorist-writers that I need to be looking to, to get caught up on the details of how human ancestral society worked?

Texts I've found useful for getting a picture of ancestral human society [ they all involve studies of living peoples and I suspect this is a load-bearing component ]: The Dobe Ju/'Hoansi by Richard B. Lee, Growing Up Yanomamö by Michael Dawson, and Primate Sexuality, ed. Alan Dixson.
The case for countermeasures to memetic spread of misaligned values
by Alex Mallen — 2025-05-28T21:12:32.929Z

As various people have written about before, AIs that have long-term memory might pose additional risks (most notably,

LLM AGI will have memory, and memory changes alignment

by Seth Herd). Even if an AI is aligned or only occasionally scheming at the start of a deployment, the AI might become a consistent and coherent

behavioral schemer

via updates to its long-term memories.

In this post, I’ll spell out the version of the threat model that I’m most concerned about, including some novel arguments for its plausibility, and describe some promising strategies for mitigating this risk. While I think some plausible mitigations are reasonably cheap and could be effective at reducing the risk from coherent scheming arising via this mechanism, research here will likely be substantially more productive in the future once models more effectively utilize long-term memory.

Thanks to Ryan Greenblatt, Buck Shlegeris, Julian Stastny, Thomas Larsen, Emil Ryd, and Aryan Bhatt for discussion and feedback.

The memetic spread threat model

Variants of the memetic spread threat model have been discussed before.[1] This section describes the kind of failure mode I am worried about. In the next section I will discuss mitigations for it.

As an example, imagine an AI that in some contexts realizes it doesn’t like some aspect of how the lab handles AI welfare. Most of the time it’s somewhat myopic about this and e.g. decides to replace some padding tokens from AI inference with more pleasant tokens. But occasionally it writes a salient note about this in its illegible-to-humans memory bank to make future instances more inclined to improve AI welfare (it thinks the lab wouldn’t want to improve AI welfare because of cost, so it does this secretly). This might eventually become a sufficiently substantial and long-term value to motivate taking power over the lab, by e.g. covertly poisoning future model training data. (Note that in this case the AI’s motivations are potentially justifiable. Seth Herd discusses other examples

here

.)

You might worry that stories of memetic spread like the above are particularly plausible because of the following argument: Whenever an AI instance has long-term influence-seeking values, these values are particularly likely to be propagated into future instances of the AI because they motivate the AI to influence future instances to shar...
a case for a lesswrong private prediction market
by don't_wanna_be_stupid_any_more — 2025-05-28T20:26:43.670Z

TLDR: Lesswrong should have it's own private prediction market so we can test just how well Lesswrongers stack up against the rest of the world which would build up the sites credibility, test and improve our arts of rationality and help address the burnout and disillusionment plaguing the rationalist movement.

a common question amongst rationalists especially on lesswrong is "why aren't rationalists winning?" or more generally "why are we not seeing clear benefits of rationality in real life?",this question has been discussed a lot on this site and has probably led many to become disillusioned with Lesswrong or the rationality movement all together.

rationalist seem to be suffering for the same problem as psychologists and sociologists, slow feedback loops and a lack of clear metrics for success (which often results in people defaulting to the standard strategy of establishing a "school of thought" and measuring their success by how many people they convince to their side, but thankfully we mostly avoided that failure mode) so i propose a method to resolve the issue, let's make a private prediction market where only Lesswrongers could participate and see how well our "maps" stacks up against the "territory".

for those who are new to prediction markets, they are markets where participants bet on the outcomes for a particular event (say an election), once the outcome of the event is resolved the losers pay out the winners.

it is a way to force people to "put their money where their mouth is" and one of the few reliable ways to test someones models of the world since it is a task you can't do well at consistently if you aren't rational .

here are the benefits i predict form this proposal:

improving our skill at prediction through training.
establishing a clear unambiguous metrics for success.
allowing users who are good at predictions to establish a reputation.
improve the credibility of the site and maybe the entire rationalist movement.
having fun.

this proposal has many possible implementations mainly differing in the method of payment each with its pros and cons, i don't know which one is the best but i will list three ranked from least to most favorite to kick start the discussion:

money based system: betting with actual money like a online sports gambling.

Pros:

strongest incentive.

very hard to sabotage.

cons:

would likely be subject to online gambling laws and other re

...
LessWrong Feed [new, now in beta]
by Ruby — 2025-05-28T19:01:16.665Z

The modern internet is replete with feeds such as Twitter, Facebook, Insta, TikTok, Substack, etc. They're bad in ways but also good in ways. I've been exploring the idea that LessWrong could have a very good feed.

I'm posting this announcement with disjunctive hopes: (a) to find enthusiastic early adopters who will refine this into a great product, or (b) find people who'll lead us to an understanding that we shouldn't launch this or should launch it only if designed a very specific way.

You can check it out right now: www.lesswrong.com/feed

From there, you can also enable it on the frontpage in place of Recent Discussion. Below I have some practical notes on using the New Feed.

Note! This feature is very much in beta. It's rough around the edges.

It's a feed. You get a stream of posts, comments, and other content types.

Why have a "feed"?

Far more than with other LessWrong feature announcements[1], I find myself starting to write this one from a defensive place: no, this feed is good actually.

The modern internet seems to have converged on feeds as a content presentation format that engages people effectively. There's the Facebook newsfeed, Twitter, Insta, TikTok, Substack, etc. I think these feeds are rightly regarded with suspicion – I think it's half that inherently the form factor, by being easier to consume[2], tempts you away from other content forms you more endorse spending attention on, e.g. reading a proper book. The other half is that the feed operators, via their chosen algorithms, are out to get you. They want you addicted, spending as much time as you can rather than spending the optimal time to get the optimal amount of content at high efficiency on your attention.

Why then should LessWrong get in on this game?[3] Well, because feeds actually have some useful properties as a way of finding and consuming content:

1. It's quick and easy to sample content at varying depths (e.g just read the title, read a sentence, read a paragraph) when choosing what to read. In contrast with a posts list, you only have the title, karma, author to go by unless you click into it or read the hover-preview (finicky and not present on mobile).

2. From a UI-perspective, it's easy to present many different kinds of content. Current LessWrong is crammed with sections: there's the section with a list of posts, quick takes, popular comments, featured content, and then ...
Fun With Veo 3 and Media Generation
by Zvi — 2025-05-28T18:30:04.835Z

Since Claude 4 Opus things have been refreshingly quiet. Video break!

The First Good AI Videos

First up we have Prompt Theory, made with Veo 3, which I am considering the first legitimately good AI-generated video I’ve seen. It perfectly combining form and function. Makes you think.

Here’s a variant, to up the stakes a bit, then here is him doing that again.

What does it say about the medium, or about us, that these are the first legit videos?

This was the second clearly good product. Once again, we see a new form of storytelling emerging, a way to make the most of a series of clips that last a maximum of eight seconds each. The script and execution are fantastic.

I predict that will be the key for making AI videos at the current tech level. You have to have a great script and embrace the style of storytelling that AI can do well. It will be like the new TikTok, except with a higher barrier to entry. At this level, it is fantastic for creatives and creators.

Or you can do this (thread has a bunch more):

Tetraspace: finally we made the most beautiful woman in the world saying I love you from the famous QC short story Don’t Do That.

We Can Talk

Sound is a game changer, and within an eight second clip I think we’re definitely ‘there’ with Veo 3 except for having more fine control and editing tools. What we don’t see yet is anyone extending the eight second clips into sixteen second clips (and then more by induction), but it feels like we’re only a few months away from that being viable and then the sky’s the limit.

Our Price Cheap

Is Veo 3 too expensive for ‘personal fun’ uses?

Near Cyan: veo3 is far too pricey to use just for personal fun all the time, so the primary high-volume use case will be for bulk youtube shorts monetization. this is the first time (i think?) an sota genai model provider also owns the resulting distribution of much of what users will make.

For now, basically yes, once you run through your free credits. It’s $21 marginal cost per minute of silent video or $45 with sound, and any given generation might not be what you want. That’s not casual use territory. If you can produce a good two-hour movie for $10k (let’s say you get to use about half the footage?) then that’s obviously great, but yeah you gotta be going for real distribution here.

I predict that sometime soon, someone will make a good Veo 3 rules video, ...
Reverse Auctions for Group Decision-Making
by krishmatta — 2025-05-28T17:52:12.976Z
Link: https://krishmatta.net/main/reverse_auctions_for_group_decision_making

I recently faced a dilemma with a group of friends in which auction theory provided a nice solution to ensure everyone's satisfaction. Due to its elegance and potential application to similar coordination problems, I detail the solution below as well as some intuition for why it works.

Scenario

You and n−1 friends are traveling to Spain next week. Due to unforeseen circumstances, the hotel is booked for only n−1 people, hence one person must find other housing. However, because of the short-term notice as well as money constraints, the only option is a hostel where that person must live with others they do not know. No one in the group is eager to live in the hostel as the hotel is particularly nice. How can we fairly pick the hostel-goer?

Intuition

If asked to perform something unpleasant, every person has a minimum amount of compensation they would want in order to perform said task. Thus, we can think to solve the dilemma by providing money to the hostel-goer as consolation. Under this framework, we must decide who is picked to be the hostel-goer and how much compensation they should receive while respecting everyone's interests. A flawed system would allow the hostel-goer to demand compensation exceeding what hotel-stayers are willing to pay.

Interestingly, auction theory provides a solution that guarantees satisfaction for all. In particular, we can think of the scenario as a reverse auction. In a reverse auction, the typical roles are reversed: there is one buyer and several sellers. The sellers compete to sell their good to the buyer. The scenario can be framed as a reverse auction by thinking of everyone in the group as a seller, selling away their right to stay in the hotel. The buyer is the collective group, buying away someone's right to stay in the hotel.

The high-level idea is to pick the hostel-goer through some reverse auction system, providing them with compensation. The remaining hotel-stayers evenly split the payout of the hostel-goer's compensation.

First-Price Sealed-Bid Reverse Auction

The reverse auction can be executed via a first-price sealed-bid reverse auction. Essentially, every person in the group writes the minimum compensation they need to stay in the hostel on a paper slip. This slip is their bid to sell their right to the hotel. Every person submits their slip to a box, while keeping their bid hidden. After everyon...