trentmkelly/LessWrong-43k huggingface dataset:
Debate experiments at The Curve, LessOnline and Manifest
by Nathan Young — 2025-06-13T22:35:14.805Z
Link: https://nathanpmyoung.substack.com/p/debate-experiments

I like debate. I have done for years. So I have been slowly trying to improve it. Here is a set of theories I had and things, experiments I've run so far.

Theory: Any debates are good.

Are any debates actually good at all? Should I give up?

Test: Watch different debates.

Evidence: I much prefer some debates to others.

Good debates:

Dr. Richard Carrier andDr. Michael Licona. I like how they chat to one another.
Destiny and Ben Shapiro. I recall liking this one. I remember them as having good chemistry.
Jubilee’s “Surrounded” debates. I love an experimental format and these get a lot of different arguments in a short amount of time[1].

Bad debates:

Finkelstein, Destiny and M. Rabbani & Benny Morris. Long and acrimonious. I think Lex Fridman is deeply guilty of the “I’ll just let them talk it out” school of debate. I think this is lazy.
Most things with William Lane Craig. Craig is an excellent debater on theology. I’m not sure I recall him ever losing. But his debates always hinge on niche points or technical arguments I don’t care about.
Anything with Jordan B. Peterson. Like trying to nail a cake to a wall.
Presidential debates. Trump in particular can lie with no cost at all, so he does.

Unclear:

Ezra Klein, Sam Harris. Bad that they don’t understand one another, but pretty interesting as a historical artefact to see two clever men who I like really fail to understand one another for very ~2018 culture war reasons.
Matt Dillahunty, Matthew Adelstein (aka Bentham's Bulldog). Dillahunty is sloppy but somehow his audience think he’s making good points. Frustrating to watch.

Status: Theory survived attempted falsification[2].

Theory: The format is the problem.

Test: Run some different debate formats (see next).

Theory: Debates are bad because debaters focus on their own status.

They have to focus on how they appear to the audience and this stops them admitting points where they are wrong.

Test 1: Find ways to protect the status of the debaters

Evidence:

I tried running two debates like this at The Curve (Daniel Kokatajlo vs. Sayash Kapoor; Dean W. Ball vs. Gabriel Weil). I tried to moderate a bit more strongly than people tend to, ensuring that there were blocks of time where each was in control of the discussion.

The debates were okay but not great.

In both, it took us a long time to get to what felt like the meat of the discussion. I recall Ball and Weil saying they didn’t really ...
Futarchy's fundamental flaw
by dynomight — 2025-06-13T22:08:11.031Z
Link: https://dynomight.net/futarchy/

Say you’re Robyn Denholm, chair of Tesla’s board. And say you’re thinking about firing Elon Musk. One way to make up your mind would be to have people bet on Tesla’s stock price six months from now in a market where all bets get cancelled unless Musk is fired. Also, run a second market where bets are cancelled unless Musk stays CEO. If people bet on higher stock prices in Musk-fired world, maybe you should fire him.

That’s basically Futarchy: Use conditional prediction markets to make decisions.

People often argue about fancy aspects of Futarchy. Are stock prices all you care about? Could Musk use his wealth to bias the market? What if Denholm makes different bets in the two markets, and then fires Musk (or not) to make sure she wins? Are human values and beliefs somehow inseparable?

My objection is more basic: It doesn’t work. You can’t use conditional predictions markets to make decisions like this, because conditional prediction markets reveal probabilistic relationships, not causal relationships. The whole concept is faulty.

There are solutions—ways to force markets to give you causal relationships. But those solutions are painful and I get the shakes when I see everyone acting like you can use prediction markets to conjure causal relationships from thin air, almost for free.

I wrote about this back in 2022, but my argument was kind of sprawling and it seems to have failed to convince approximately everyone. So thought I’d give it another try, with more aggression.

Conditional prediction markets are a thing

In prediction markets, people trade contracts that pay out if some event happens. There might be a market for “Dynomight comes out against aspartame by 2027” contracts that pay out $1 if that happens and $0 if it doesn’t. People often worry about things like market manipulation, liquidity, or herding. Those worries are fair but boring, so let’s ignore them. If a market settles at $0.04, let’s assume that means the “true probability” of the event is 4%.

(I pause here in recognition of those who need to yell about Borel spaces or von Mises axioms or Dutch book theorems or whatever. Get it all out. I value you.)

Right. Conditional prediction markets are the same, except they get cancelled unless some other event happens. For example, the “Dynomight comes out against aspartame by 2027” market might be conditional on “Dynomight de-pseudonymizes”. If you buy a contr...
The Pros and Cons of Being Among Your Tribe
by Sable — 2025-06-13T21:41:52.631Z
Link: https://affablyevil.substack.com/p/the-pros-and-cons-of-being-among

I recently attended the rationalist conference, LessOnline. I wanted to document some of the experience, and found myself bifurcating my emotional reactions. The conference itself was wonderful, with great people, food, and venue, but there were pros and cons to going. Though the pros greatly outweigh the cons, both deserve to be mentioned.

Let's split and commit.

Cons

Social Anxiety

I have some measure of social anxiety, but it’s kinda weird in that public speaking doesn’t affect me in the slightest while socializing at a party with friends of friends is terrifying. I’ve come to understand it as something of an inverted U:

I can socialize easily among strangers because it doesn't matter to me if they hate me. It's easy to be sociable when there's nothing on the line. It’s even easier to do public speaking in front of strangers, because that isn’t even socializing, just me talking in front of people.

On the other side of the inverted U, with close friends and family, I know that I’m safe if I fuck up. If I commit some social gaffe or look stupid, who cares? These are the people I love and trust, and they likely got that position because they’ve seen me at my worst and didn’t run away.

When I got to LessOnline, on the other hand, I was surrounded by people at the top of the inverted U: people whose opinions I care about, but who I don’t actually know very well. If I fuck up in front of them, who’s to say how they’ll react? Maybe Zvi will walk away thinking I’m a moron, or one of the many venerable Scotts will realize I secretly have no idea what I’m talking about.

Whether or not it would happen like that, it feels plausible that it could. I could make an ass of myself in front of the community I’ve chosen, the one whose opinions actually matter to me. I might sometimes feel like an asshole, but if Duncan Sabien thinks I'm an asshole, that's Really Bad. Within my social context, he’s an Authority on assholes. If he thinks I’m an asshole, I can’t ignore it or brush it off or excuse it, I have to actually go soul-searching because it’s pretty likely I’m an asshole.

The same is true for everyone at LessOnline: everyone there is either a rationalist or rationalist-adjacent; their opinions of me are more credible evidence about who I am than the opinions of any collection of strangers could ever be.

Status-Consciousness

I found, when examining my own mental processes, t...
Constraining Minds, Not Goals: A Structural Approach to AI Alignment
by Johannes C. Mayer — 2025-06-13T21:06:40.984Z

TL;DR: Most alignment work focuses either on theoretical deconfusion or interpreting opaque models. This post argues for a third path: constraining general intelligence through structural control of cognition. Instead of aligning outcomes, we aim to bound the reasoning process—by identifying formal constraints on how plans are generated, world models are used, abstractions are formed, etc. The key bet: all sufficiently general intelligences necessarily share architectural features that can be described and intervened on at the structural level. If we can formalize these at the right level of abstraction, we gain a language of minds—a framework for reasoning about and ultimately constructing systems that are powerful enough to perform pivotal acts, but structurally incapable of catastrophic optimization when carefully targeted.

Note: This line of thinking is probably not entirely novel. I expect there are overlaps I haven't mapped yet, and would appreciate pointers.

The problem of aligning artificial intelligence divides naturally into two distinct but interrelated agendas:

Goal-alignment: How can we specify or extract the right values such that a powerful AI system robustly optimizes for human-desirable outcomes?

Optimization-bounding: What are the minimal structural constraints on reasoning and planning that preclude unbounded or misgeneralized optimization, while still allowing a system to be powerful enough to perform pivotal acts?

This second agenda is less explored and potentially more tractable. Rather than attempting to solve the full problem of value specification and alignment, it aims to limit the agent’s optimization profile—its reach, generality, and unintended side effects—through structural, mechanistic constraints. The goal is to narrow the scope of cognition such that even with imperfect values, the system remains within a bounded regime where catastrophic failure modes are structurally precluded.

Throughout this post, we use the word bounded not to mean “less capable” or “less intelligent,” but rather to describe structured constraints on the reasoning process: interventions that shape which plans can be generated, how abstractions are formed, how far causal influence can propagate, and so on.

Motivation and Context

Most current alignment research clusters around two poles. On one side is mechanistic interpretability, which tries to understan...
The optimal level of optimization is suboptimal
by ellifournier — 2025-06-13T18:06:02.611Z
Link: https://ellifournier.substack.com/p/the-optimal-level-of-optimization

How much time should you spend optimizing any particular process you might engage in? Even assuming that you’re optimizing for a value of overriding importance or value there is only a limited amount of time available.

If all available time is spent optimizing clearly that would be suboptimal since there would be no time left to actually engage in any particular process pursuant to what we value. So the optimal level of optimization is always suboptimal.

However, that might seem to be trivial and only operant at some kind of asympoptic limit we need not worry about in our lives. The problem, though, is deeper. That the optimal level of optimization is suboptimal is both a kind of trivial truth as our time is finite but also a statement about our epistemology; our capacity to know what level of optimization is optimal (even if suboptimal) is itself subject to the same kind of suboptimality that any other kind of knowledge-pursuit is.

This is not to say we should pursue single-mindedly the optimization of any value(s) we desire; rather, it is a guard against the view that optimization is synonymous with perfectionism.
On Pruning an Overgrown Garden
by Vaatzes — 2025-06-13T17:54:09.725Z

As a new user, it's hard to know where to start, and how to contribute to a community. being a Good Samaritan by nationality, I was reading through the guides and posts pertaining to the LessWrong community. One article that stood out to me is the "Well-Kept Gardens Die By Pacifism" post. The rhetoric revolves around the fool, and where the fool goes, (intellectual) communities die. It resonated with me. I manage a community that's large on paper, but in practice often devoid of content that excites experts. Indeed, now devoid of the content that attracted (and was made by) the experts that grew the community in the first place, long before I joined.

Is our community dead? Even a community overtaken by fools can still be alive, albeit with little recollection of its former self. If a Well-Kept Garden dies by pacifism, the death of such a Garden is more like a return to nature. Overgrown, covered in weeds, and barren in places where nothing can grow. The community is not dead. There is still movement. Yet returning it to its former glory, or better yet, growing something beautiful in its own right; new and fresh for the modern eye, will take significant efforts. Efforts in pruning and espalier. And in bringing in new, fertile soil.

Before taking on this painstaking responsibility as the self-appointed gardener, I think it wise to ask myself what I can learn from the current state of the community. How did we get here, and what can I do to prevent this slow abandonment by our most valued members. And this is where I set out to question the fool.

Because I don't believe in the fool.

There is no fool

Obviously, online trolling is a real thing. More common than trolling, and likely vastly more deathly owing to its insidious and seemingly tolerable nature (it should not be tolerated) are indolence and deceit. Explicit malice can be counteracted, swiftly and confidently. But incomplete information in question asking and lazy hand-waving in replies is not always so easily spotted. And deceit, deceit prides itself on its stealth. "Perhaps if I add this to the discussion, I'll be thought of as more than I am", deceit thinks.

So how can I say there is no fool. If you're from the community, you'll know I've done more than my fair share of warning, kicking, and banning. Surely, they were fools. And they were, but they were not the fool. They were not the death of the community...
Learned helplessness about "teaching to the test"
by Viliam — 2025-06-13T17:53:10.649Z

I keep wondering why is there so much learned helplessness about "teaching to the test".

It is often used as an example of Goodharting (e.g. here), and the implied conclusion seems to be... that it is wrong to try testing students systematically, because it is known that it inevitably causes teaching to the test, which ruins education?

Every time I read something like that, I think: Why can't we simply fix the test, so that "teaching to the test" either becomes impossible, or becomes the right thing to do? To me this seems like the obvious reaction, so I am surprised that I don't see it more often.

*

Let me give you a fictional example: Suppose that a school teaches children the multiplication table up to 10×10. The department of education creates a test, containing two questions: "how much is 5×5?" and "how much is 7×8"?

Yes, if you are going to judge schools based on how well students answer these two questions, of course many teachers are going to follow the incentives, and instead of multiplication, they will spend all the time at classroom making the students memorize "5×5=25" and "7×8=56", even if doing so means that there will be no time left for other multiplication examples. So the next generation of students will have no idea how much is 6×6, despite officially having multiplication in the curriculum.

A scary story, isn't it? Does it mean that we should never test children on multiplication?

No, that would be completely stupid! (Looking around anxiously, hoping that someone agrees with me...)

The problem with the proposed test is that out of one hundred possible multiplication problems, it predictably tests two predetermined ones.

Well, how about instead of that, each year generate two multiplication problems randomly? That way, teachers won't know which specific multiplication problems they need to teach, so the best educational strategy will be to teach all of them.

Okay, one problem with this is so obvious that even I can predict it. If you literally choose the problems randomly, some years you are going to get "1×1" and "1×2" or something like that on the test, and people won't stop talking about how you ruined mathematical education forever, and how unfair it is that one generation of students got such easy problems, compared to the previous and the following years.

But if you do the simple fix and remove multiplication by one from the test, sooner or la...
Information-Dense Conference Badges
by ozziegooen — 2025-06-13T17:52:00.625Z
Link: https://ozziegooen.substack.com/p/information-dense-conference-badges

See previous discussion here.

I find a lot of professional events fairly soul-crushing and have been thinking about why.

I dislike small talk. Recently I attended

Manifest

, and noticed that it could easily take 10 minutes of conversation to learn the very basics about a person. There were hundreds of people at the conference, so meeting people felt expensive and haphazard.

My regular time is costly, and conference time should be even more expensive. It seems reasonable to aim for $50 to $300 of value per hour. So if I spend 10 minutes with someone only to find out we don't have much in common, that can easily be $20 to $100 lost (total, between both parties). Add the unpleasantness of slowly losing my voice and being in a crowded atmosphere. I enjoy quiet spaces and my energy reserves quickly deplete in crowded and loud settings.

The badges were incredibly basic. Most people's fashion choices were similarly basic. So there's little to go off of. You sort of have to start from scratch for each interaction.

After the first day I got the idea to use a small

sticker machine

to at least post some basics about me to my badge.

If you think about it, conference badges are an interesting design opportunity. We could probably get away with larger options than normal—maybe even 10" by 8" or similar. This gives a lot of space to work with. They can feature quite a bit of written and visual information.

I went back-and-forth with ChatGPT to design a few other options. These are obviously rough and imperfect, but I think they suggest some interesting directions. I'd naively expect events to maybe be 5% more pleasant and useful for me with badges like these. (See the end of this post for the mockups)

Practical Considerations

The obvious challenge is cost. Printing fully custom badges for each attendee would run around $40 per badge, plus design time and iteration. For a 200-person conference, that's $8,000+ just for badges alone.

The most practical approach is probably a hybrid system. Give people 3-20 pre-designed badge templates to choose from, then set up a sticker station with a curated selection of options. Recently I went to get a library card and was given 5 different design options. Doing this for event badges seems doable and more practical.

The tricky part isn't the printing—it's sourcing and organizing the stickers. I could easily imagine wanting 50 to 500 different ...
The Boat Theft Theory of Consciousness
by Lorec — 2025-06-13T16:38:21.608Z

[ Context: The Debate on Animal Consciousness, 2014 ]

There's a story in Growing Up Yanomamo where the author, Mike Dawson, a white boy from America growing up among Yanomamö hunter-gatherer kids in the Amazon, is woken up in the early morning by two of his friends.

One of the friends says, "We're going to go fishing".

So he goes with them.

At some point on the walk to the river he realizes that his friends haven't said whose boat they'll use [ they're too young to have their own boat ].

He considers asking, then realizes that if he asks, and they're planning to borrow an older tribesmember's boat without permission [ which is almost certainly the case, given that they didn't specify up front ], his friends will have to either abort the mission or verbally say "we're going to steal John's boat". This would destroy all their common-knowledge [ in the game-theoretic sense of common knowledge ] plausible deniability, making it so that no one would be able to honestly say, upon apprehension, "I was there, and we didn't really plan to steal any boats, we just . . . walked to the river and there was one there."

In order to be making the decision -- deliberate or not -- to omit facts that will later be socially damning from their explicit communication, while still getting away with ostensible moral violations -- Mike and his friends had to have a razor-sharp model of what was socially damning.

And, in order to differentiate between [ their razor-sharp model of what was socially damning ], versus [ what they personally felt they could get away with if certain facts were carefully omitted from their explicit communication ], they -- or rather, their brains, since the bandwidth of conscious human cognition couldn't realistically handle this explicitly -- had to have a very strong ability to navigate the use-mention distinction.

Use-mention almost had to be a primitive, in addition to all the other primitives -- social and otherwise -- their brains had natively.

If you've read GEB, you know the natural way to make use-mention a primitive is by running a self-model.

Monkeys are really bad at concealing their guilt.

If a monkey tries to steal something, it will usually give itself away to any watching conspecifics by its cringing posture.

It knows theft is wrong -- it has to know this, to avoid social punishment -- and it lacks the ability to partition use -- the act of reaping the...
Monthly Roundup #31: June 2025
by Zvi — 2025-06-13T16:20:01.739Z

It’s always a nice break to see what else is going on out there.

Bad News

Study finds sleep in male full-time workers falls as income rises, with one cause being other leisure activities substituting for sleep. It makes sense that sleep doesn’t cost money while other things often do, but the marginal cost of much leisure is very low. I don’t buy this as the cause. Perhaps reverse causation, those who need or prefer less sleep earn more money?

The productivity statistics continue to be awful, contra Alex Tabarrok part of this recent -3.88% Q1 print is presumably imports anticipating tariffs driving down measured GDP and thus productivity. The more I wonder what’s wrong with the productivity statistics the more I think they’re just a terrible measure of productivity?

A model of America’s electoral system that primary voters don’t know much about candidate positions, they know even less than general election voters about this, so they mostly depend on endorsements, which are often acquired by adopting crazy positions on the relevant questions for each endorsement, resulting in extreme candidates that the primary voters wouldn’t even want if they understood.

It’s not really news, is it?

Paul Graham: A conversation that’s happened 100 times.

Me: What do I have to wear to this thing?

Jessica: You can wear anything you want.

Me: Can I wear <clothes I usually wear>?

Jessica: Come on, you can’t wear that.

Jessica Livingston (for real): I wish people (who don’t know us) could appreciate the low bar that I have when it comes to your attire. (E.g. you wore shorts to a wedding once.)

Rules of Confidentiality

Alex Thompson says “If you don’t tell the truth, off the record no longer applies,” proceeding to share an off-the-record unequivocal denial of a fact that was later confirmed.

I think anything short of 100% (minus epsilon) confidence that someone indeed intentionally flat out lied to your face in order to fool you in a way that actively hurt you should be insufficient to break default off-the-record. If things did get to that level? If all of that applies, and you need to do it to fix the problem, then okay I get it.

However, you are welcome to make whatever deals you like, so if your off-the-record is conditional on statements being true, or in good faith, or what not, that’s fine so long as your counterparties are aware of this.

Something Is Wrong On

...