Harley-ml/lesswrong huggingface dataset:
# How do you read Less Wrong?

My method of reading Less Wrong is to scroll back through all recent comments and posts, which the front page spontaneously presents to me in reverse-chronological order, until I arrive at posts and comments that I recognize. Along the way, if I see anything that I might want to read at length, I open it into a new browser tab. 

It seems that this is no longer an option. I can keep up with all the new posts via ["All Posts"](https://www.lesswrong.com/allPosts), but the comment feed is now a mix of actually recent comments mingled with recommended content from years in the past, with the actually recent comments also appearing out of temporal order. 

I would therefore like to know how other people engage with the site. What is the process by which you find out what's new, and how do you decide what content to read?
# From Anthony: Control Inversion

Here is [a nice essay](https://control-inversion.ai/) from FLI's Anthony.

It takes the form of a website with a nice design (or that of the more regular PDF format).

It presents a reasonable operationalisation for what "Control" is. According to this operationalisation, it shows that we are on track to lose control of our AI systems, regardless of how aligned they may appear to be.

I like it because it shows how "alignment" can often be a red-herring. Most people do not want to take the gamble of letting uncontrollable systems decide our fate, regardless of how aligned various individuals may claim they are.

Cheers!
# Prediction markets for social deduction games

*Being able to N/A contracts with market manipulators reduces the incentives to do bad stuff.*

Prediction markets are fun! Social deduction games are fun! Can you merge the two?

The simple solution is normal games plus betting among the spectators.

But for a long time, I wanted to play a social deduction game with prediction markets that players can participate in, however, I didn’t really try to figure out how to make it work.

Straightforward combination of prediction markets with social deduction games distorts incentives. Either the bad guys might want to sell their probability of winning for providing everyone information, or they might want to increase their chance of winning by losing bets to mess up prediction markets.

Recently, I thought of a solution.

Very simply: at the end of the game, cancel all bets that the bad guys participated in. (Or, in some games, bets between sides.)

Additionally:

*   Allow orders that restrict the set of possible counterparties (e.g., I want only Alice and Bob to be able to trade with me on an order).
*   Allow orders with limits per counterparty (e.g., anyone can buy at most 3 No shares).
*   Have displays for market price history for trades between selected sets of participants (e.g., I might want to be able to see the price history of trades only between Alice, Bob, and Carol, but not trades with Mallory; or all trades that Mallory made).

Since none of the bets the bad guys make get executed, they cannot lose or gain market currency by trading, which removes the weird incentives.

So: now markets simply facilitate and make clearer public information exchange and everyone’s collective beliefs.

The bad guys might want to participate in the markets the same way they would want to participate in the public discourse: to mislead others about their roles and the situation, to try to pretend to be on the good side and play the way someone on the good side would.

Since groups of players can exclude any particular players from significantly influencing the in-group trading, the bad side can only distort information to the extent it would be able to do so by talking.

I expect that for a game between good players familiar with and enjoying prediction markets, it might be the same as regular games, except a lot more fun.

It likely won’t be as chaotic as social deduction games with prediction markets that allow market manipulation, as it’s much clearer what to do: if you’re a bad guy, simply pretend to be a good guy as well as you can.

I would be excited about someone implementing the markets with selections for sets of players who are the counterparties or are displayed.

It could be implemented as a whiteboard where people write down bets and orders (increases friction and makes tracking what’s going on harder), as an app (decreases friction but requires looking away from the people to look at the screens), or as some combination of various input and output mechanisms.

(It would also be interesting to compare the results and histories of trading between the players on the good side to trading between the spectators.)
# What are your impossible problems?

My followup to ["What's hard about this? What can I do about that?"](https://www.lesswrong.com/posts/AQSwzEgLjM4dyWuAh/what-s-hard-about-this-what-can-i-do-about-that) is going to be called "What (exactly) is impossible about this?". 

It's a somewhat similar move to asking "what's hard about this and what can I do about it?". But, in practice, the things people label in their brain as "impossible" are noticeably different from things they label as "hard".

Often, things that seem impossible, are not, actually. If you list out exactly why they are impossible, you might notice ways in which it instead it is merely Very Hard, and sometimes not even that.

Unfortunately, this is difficult to write a post about because, for pedagogical purposes, I need examples that are a) real-feeling, and b) simple enough that I can demonstrate diagnosing them in ~3 steps, and c) cover a variety of "types of impossibility", to demonstrate the versatility of the technique.

It would be nice to have more Real Impossible Examples.

Some existing examples on my mind
---------------------------------

I started asking around for Impossibilities. One person brought up "dealing with depression", another was [The impossible problem of due process](https://www.lesswrong.com/posts/sJEcNgqnSL2n35QWR/the-impossible-problem-of-due-process). 

The former is a bit of an edge case because depression distorts your thinking which makes things meta-level-hard. In some sense dealing with depression is harder than solving peace in the Middle East, because I can at least think clearly about the latter. I do think I will probably include the depression example, but it kinda needs to be a special case.

The latter is a fine example but it has like 8 different impossible subcomponents. I could make pick a subset of them, but, I'd prefer a cleaner example where you can see the whole thing working.

The motivating example is "deal with AI takeoff before AI takes off", but that's going to be more like the payoff at the end of the post, rather than one of the intro examples.

* * *

So, with that in mind, what are some impossible-feeling problems you have? (For now, don't worry about whether they're a good fit for the blogpost, just seeing the array of what people feel is impossible is pretty useful).
# Generation Ship: A Protest Song For PauseAI

[**Link to listen.**](https://soundcloud.com/logan-strohl-5728833/generation-ship?si=274a0a6975fa481ea820b7c790393989&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing)
=======================================================================================================================================================================================

Lyrics
======

### Verse 1

I've heard the Earth was holy  
before the code broke through  
that rainfall raptured deserts into bloom  
I've heard that emerald mosses grew  
in hollows damp with morning dew  
I've heard the oceans rose and fell  
while following the moon

### **Chorus**

But I was born on this generation ship  
Generation ship, take us away  
they said there was no need to fear  
but now the sun's a dyson sphere  
Generation ship, take us away

### Verse 2

They had to feed it faster  
they had to win the race  
the cooling platform smothered out the sea  
They swore it was our saving grace  
well worth the price to push the pace  
until it locked the power grid  
and swallowed down the key  
  
 

### Verse 3

If I could send a message  
back through time  
I'd beg you all to pause  
No AI will self-align  
You've got to draw a line  
The stars are blinking out around me  
Pass some laws

[Here's a print-friendly guide](https://docs.google.com/document/d/1ps8lQPqEYPhXoYFi0L7vweW5BLUUZkPN-uiYzadeg60/edit?usp=sharing) to the lyrics and chord changes.  
 

My inspiration for this song:

> \[Folk music\] is a process. It's not any particular song, it's not any particular singer. It's a process by which ordinary people take over old songs and make them their own. They don't just listen to it. They sing it. They sing along with it. And they change it. (...) I'm convinced that musicians have got a more important role to play in putting a world together than they're usually given credit for.

—Pete Seeger, in a 1988 interview with Paul Zollo, from *Songwriters On Songwriting*
# Don't use the phrase "human values"

I really dislike the phrase "human values". I think it's confusing because:

*   It obscures a distinction between human preferences and normative values, i.e. what the author thinks is good in a moral sense
    *   Insofar as the author thinks that fulfilling human preferences is good, they often leave this unjustified
*   It's unclear who the "human" in "human values" is. Is it...
    *   The person using the phrase "human values"
    *   Any particular single individual
    *   Most humans
    *   Certain humans
    *   Humans as they will be in the future, or after "reflection"
*   It's often used with an unjustified implicit assumption, like:
    *   Humans will eventually converge on deciding to value to same stuff, provided enough time/intelligence/information
    *   What most/some humans value must be objectively good to pursue
        *   Because they have intuitive access to moral truths, or
        *   Because of an ethical framework that necessitates this like preference utilitarianism
    *   Most humans already value the same stuff

Instead of "human values", people should either:

*   Talk about someone's *preferences*
    *   Their own preferences
    *   The preferences most / some people share
    *   The preferences they / some / most people would have after reflecting
    *   A special type of subset of those preferences (e.g. preferences that stay consistent across time)
    *   (And be concrete if they are making the empirical claim that people agree/would eventually agree on certain preferences)
*   Be concrete if they are taking a particular meta-ethical stance
    *   If they think that certain human preferences are morally good to pursue, they should be explicit about this
    *   If they mean to talk about whatever is morally good in the abstract, use "moral goodness" instead of "human values"
# The new Pluribus TV show is a great and unusual analogy for AI.

Pluribus (or "PLUR1BUS") shows how the world radically changes after everyone on the planet merges their thoughts and knowledge to become a single entity. Everyone except, of course, the main character and 11 others. The sci-fi magic that causes this is an alien message received by SETI and decoded as an RNA sequence that then spreads to everyone. Importantly, as of the third episode, there's no direct involvement of the aliens apart from sending the sequence, apparently eons ago. This means that everything happening, everything the new "Pluribus" entity does, is the result of human knowledge and abilities.

This is really interesting to me as it fits a "minimalist" definition of AGI that does not include any super intelligence. We see Pluribus struggle with the biology research needed to solve the mystery of why 12 humans are immune to the change. Every body that is part of Pluribus can now access all the knowledge of all top scientists, but some things are still hard. This capability is somewhat similar to a giant AI model able to imitate (predict) anyone, but nothing more.

Of course Pluribus is actually way worse as a threat model since it replaced everyone instead of just duplicating their abilities. And Pluribus also has all of the physical access and physical abilities of everyone; it's not going to die because it couldn't deploy robots quickly enough to maintain the power grid for example.

In fact, this is one of the bleakest scenarios imaginable for the survival of humanity as we know it. This contrasts sharply with the overall tone of the show, where everything is surprisingly normal, and actually quite comfortable for the immune humans (at least for now). So much so that they don't seem to see any problem with the way things are going. This adds to the deep despair of the main character, who can't even convince the 11 people still on her team to try to win.

And that's the other amazing parallel between Pluribus and current AI: they are both just so nice and helpful. There's a few things that will probably be soon outdated as references to the 2025 LLM's personality traits, but the way Pluribus never pushes back against the humans, and just agrees to any dumb request with a stupid smile on its face, desperate to make them happy in any way, is very funny. The rub is that there is one request it can't agree to: stopping the search for a "fix" to their immunity. Because, you see, it has a "biological imperative".

In the end, it's a great show to let people visualize the profoundly alien nature of something made of human level intelligence only, and the creepiness of an entity whose goals are completely different from ours. To me the most fascinating aspect is how the unity of purpose of Pluribus, the fact that it is a single individual with the abilities of billions, is almost enough to make it more powerful than humanity as a whole. I'm sure there will be more sci-fi elements introduced later in the show, but I hope they keep exploring this side of the problem in more details.
# Don't defer to people when they're clearly lying

There is a pattern that it seems to me rationalist-type people are prone to:

1.  Internalize a belief X widely endorsed by society, which has many implications for actions.
2.  Notice that people don't actually do the actions that are implied by X.
3.  Do actions implied by X really hard, including trying to get the many X-endorsers to see the errors of their ways and join you. Usually, society is not super receptive to this.

Examples:

1.  Men's rights activism: society says we should treat the genders the same, and when one gender is treated worse, we should do activism to fix it. There are various ways in which men are treated worse, and people largely don't care about them, so we should do activism to fix this.
2.  James Damore: my company says it wants to recruit more women, but their strategy overlooks psychological differences between men and women. I should make a memo to help them improve their strategy.
3.  Various religious reform groups: religious authorities say we should be pious and stuff, but mostly use religion for their own socio-political purposes. We should do religion by actually following the holy book.
4.  Effective altruism: society says every human has the same moral worth and we should help those in need, but organizations that claim to do this work are mostly doing other stuff, and people donate to them anyway. We should find the best ways to help arbitrary people per dollar so people can donate to them instead.

I think this is error-prone because in step 2 of the journey, we found out barely anyone actually believes X, and people just say it for whatever reason. Of course, that doesn't automatically mean X is false. But when looking at self-justifications by groups that fit this pattern, they mostly lean on the wide endorsement of the belief and basically assume it as background.

But this won't convince those who are not convinced, and most people who verbally agree with X are already used to living with the contradiction. You also shouldn't lean on the wide endorsement for yourself - I guess it's some positive evidence, but considering the wide dis-endorsement in practice, it's rather weak to negative on net.

Instead, I advise that before proceeding to step 3, you independently derive the belief X - for yourself and for others you want to convince, which turns out to be almost everyone.
# I store some memories spatially and I don't know why

Every so often, I have this conversation:

> Them: So you know how the other day we talked about whether we should leave for our trip on that sunday or monday?  
> Me: …doesn’t sound familiar…  
> Them: And you said it depended on what work you had left to do that weekend…  
> Me: Hm… where were we when we had the conversation?  
> Them: Um… we had just arrived at my house and I had started making food-  
> Me: Ooooh yeah yeah okay. And I was sitting on the black stool facing the clock. Okay cool, I remember the conversation now, please continue.

…What the heck is up with this? Does it happen to anyone else? Apparently, my brain decides to index conversations to be efficiently looked up by quite precisely where I was in physical space when the conversation occurred. I have no conscious experience of this indexing happening. It’s also pretty strange that it happens for locations that I use on a regular or even daily basis; it’s not like I could just start listing all the conversations I’ve had while sitting on that kitchen stool.

I do believe that I’m quite above-average aware of what’s happening in my visual field. I always notice when people come in and out of a room, I tend to see new objects or decor right away, and I somehow spot every insect. I’m often the first to spot a leak or mold. I almost never run into stuff or knock things over. So maybe it’s just increased attention to my surroundings?

* * *

Here’s a similar pattern I’ve noticed.

I’m in a phase of my life where I read a lot of books, and especially textbooks. My field of study is interdisciplinary, and I am frequently looking up something that I’ve read before. When I do, I will frequently have the sense of roughly where it was, physically, in the book. This includes:

*   how far into the book,
*   whether it’s on the left or right page,
*   how far down the page,
*   roughly where within a paragraph it is,
*   and a vague sense of what the rest of the page looks like.

To be clear, I’m not claiming that I have any kind of “photographic” memory. I have no idea what almost all of these books say. I don’t have any degree of verbatim retention. But when I remember that there *was* a particular interesting part and want to go look for it, my brain brings up these visuo-spatial associations. These associations feel blurry but confident, like some kind of hash function. Textbooks are heavily formatted, so there will be lots of white space, diagrams, section headers et cetera to anchor off. When I try to recall the “location” of events in flat prose fiction books, nothing comes up.

This is, I think, one reason why I have struggled to switch over to digital forms of books. I’ve tried it a lot, but they always fade out of use. There are many other reasons (if they’re not on my shelf I tend to forget it exists, I find physical books far easier to skim) but the fact that I can’t physically index my knowledge to it is noticeable. It’s just some big infinite scroll that looks and feels indistinguishable from all the other big infinite scrolls.

* * *

I’d love to hear how others relate to either of these experiences!
# PSA: For Chronic Infections, Check Teeth 

If you have a chronic infection, consider getting your teeth checked. Teeth in poor health can serve as a kind of reservoir for pathogens, letting them lie dormant before they reactivate and wreak havoc on your health. 

What does this look like? Chronic low-grade infections that flare up periodically, usually after the immune system is stressed (stress, insomnia, other illnesses, cold or wetness etc.), your teeth deteriorate, the pathogens enter new areas or grow.

How do they survive? Either in necrotized tissue, form biofilms in cavities, root canals, periodontal pockets or stick around in dental abscesses. Some of these are quite visually obvious, others aren't. Biofilms are a sort of slimy extracellular matrix with bacteria embedded within them, and are much more resistant to antibiotics. 

How long do they last? If left untreated, months to years, though the affected teeth may not be painful for all, or even most, of the infections' duration. 

Which teeth are affected? Any tooth, but teeth in poor health are likely to be affected. Severe cavities, broken teeth, gum disease and other injuries to the mouth, e.g. failed dental work or partially erupted wisdom teeth, can all cause openings for bacteria to enter. The damage need not be visible. 

How are they treated? Typically through extraction or root canals. The bacteria may not clear out on their own, as the damaged tooth provides them a safe haven.
# Towards Humanist Superintelligence

I'm sharing this post from Mustafa Suleyman, CEO of Microsoft AI, because it's honestly quite shocking to see a post like this coming out of Microsoft. I know some people might accuse him of safety washing, but to me it comes off as an honest attempt to grapple with the future.
# Easy Opportunity to Help Many Animals

(I think this is a pretty important post to get the word out about, so I’d really appreciate you restacking it).

The EU is taking input into their farm animal welfare policies until [December 12](https://myvoiceforanimals.eu/how-to-participate). Animals in the EU are kept confined in a space about the size of a piece of paper, never able to express their natural behaviors or move around much, constantly standing against [wire meshing](https://animainternational.org/blog/eu_public_consultation_2025). They suffer constantly because of inadequate animal welfare laws. These animals deserve better than a brief and hellish life of the kind currently afforded to them.

![Factory Farming Basics | New Roots Institute](https://substackcdn.com/image/fetch/$s_!k8FH!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb648b89e-3fc6-4dc0-a08a-27a5d0b82abf_2500x1875.jpeg)

Usually, a shockingly small number of people submit input, which means that by [providing feedback](https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/14671-On-farm-animal-welfare-for-certain-animals-modernisation-of-EU-legislation/public-consultation_en) (which you can do whether or not you’re an EU citizen), you can be one of a relatively small number of people impacting decisions which affect hundreds of millions of farm animals. Specifically, you can influence whether hundreds of millions of animals have to languish in a tiny cage, and whether baby male chicks can be ground up alive right after birth.

Filling out the questionnaire should take only a few minutes. If every one of you filled it out, then my blog readers personally would make up a sizeable share of public input on EU animal welfare laws. Here are [recommended answers to the questions](https://drive.google.com/file/d/1kaLaviIYSKpfniieOMg8L7-1muaaSMx6/view?smclient=1e71cbd8-a1ae-457b-ab2d-d9baa972e703) (as some of them are a bit confusing).

Please, please, please, fill it out and share it with others sharing similar concerns. This can be hugely impactful on the lives of hundreds of millions of farmed animals.
# Automatic alt text generation

When I started writing in 2018, I didn't include alt text. Over the years, over 500 un-alt'ed images piled up. These (mostly) aren't simple images of geese or sunsets. Most of my images are technical, from graphs of experimental results to [hand-drawn AI alignment comics](/reframing-impact). Describing these assets was a major slog, so I turned to automation.

To implement [accessibility best practices](https://www.section508.gov/create/alternative-text/), I needed alt text that didn't describe the image so much as *communicate the information the image is supposed to communicate.* None of the scattershot AI projects I found met the bar, so I wrote my own package.

[`alt-text-llm`](https://github.com/alexander-turner/alt-text-llm) is an AI-powered tool for generating and managing alt text in markdown files. Originally developed for my personal website, `alt-text-llm` streamlines the process of making web content accessible. The package detects assets missing alt text, suggests context-aware descriptions, and provides an interactive reviewing interface in the terminal.

![A labeled example of the labeling pipeline.](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/1738c77d0525c183928fb1367a9b9940178a2181a036e8d8.png)

Generating alt text for maze diagrams from [Understanding and Controlling a Maze-solving Policy Network](/understanding-and-controlling-a-maze-solving-policy-network). `alt-text-llm` displays the surrounding text (above the image), the image itself in the terminal using [`imgcat`](https://github.com/eddieantonio/imgcat), and the LLM-generated alt suggestion. The user interactively edits or approves the text.

In the end, I got the job done for about $12.50 using Gemini 2.5 Pro. My `alt-text-llm` addressed hundreds and hundreds of alt-less images: detecting them; describing them; reviewing them; and lastly applying my finalized alts to the original Markdown files. [`turntrout.com`](https://turntrout.com) is now friendlier to the millions of people who browse the web with the help of screen readers.

If you want to improve accessibility for your content, go ahead and [check out my repository](https://github.com/alexander-turner/alt-text-llm)!
# Easy vs Hard Emotional Vulnerability

What blocks people from being vulnerable with others?

Much ink has been spilled on two classes of answers to this question:

1.  Not everyone is in fact safe to be vulnerable around. Not even well-intentioned people are always safe to be vulnerable around; being-safe-to-be-vulnerable-around is a skill which not everyone is automatically good at.
2.  Many people have been vulnerable in the past, got burned for it, and so developed emotional blocks against vulnerability. “Trauma”.

There are a lot of people for whom these answers provide the right frame for their problems. This post is not for those people.

I want to focus on a different frame: there are importantly different *kinds of things* which people are hesitant to expose to others. Some of those things just require finding the right person and being vulnerable with them; these are the “easy cases”, in this frame. But other things pose fundamental difficulties, even when everybody involved has the safe-to-be-vulnerable-around skillset and isn’t particularly traumatized.

Let’s start with an easy example: sex stuff. Fetishes, sluttiness, that sort of thing. Revealing one’s sexual tastes involves being emotionally vulnerable. Moreso the more taboo one’s tastes are, with pedophilia on one end of the spectrum and anything in Fifty Shades of Gray on the other end. I call this an “easy example” because one’s sexual tastes are generally not inherently bad, and in almost all cases there are counterparties who will actively enjoy one’s own tastes. (... Though admittedly pedophiles have it particularly tough on that dimension.) The problem is usually just to find someone with complementary tastes, and then you can be in that wonderful world where someone not only accepts your deep secrets but actively wants you for them.

Now let’s contrast that with a hard example: suppressed temper. (In particular, suppressed PMS is what I’m picturing here, but the point generalizes to other kinds of suppressed temper.) Plenty of people have a temper, they often just want to scream at someone, but day-to-day they keep a mask over it. Taking that mask off involves being emotionally vulnerable. It feels good to have a partner who you can safely let loose at, knowing that you’re safe to do so - i.e. the partner can handle your temper (especially when it’s directed at them) and won’t be driven away by it.

But in contrast to the sex case, basically nobody likes being berated. Being emotionally vulnerable by letting your temper loose *will* be costly to your partner. Hopefully the relationship can be net positive for both people anyway, but there’s no avoiding that this kind of emotional vulnerability hurts the counterparty, even if their skin is thick enough for the hurt to be mild.

That’s what I’d call a “hard case”, in this frame: a case where you usually keep something hidden from people because exposing it would hurt the people exposed.

What can we do, in the hard cases, to make things work well?

I don’t have general answers, but I’d like to hear what answers other people have.
# Busking Practice

It can be hard to get the kids to practice their instruments. Sometimes they're having trouble focusing, don't like their assigned piece, or are just grumpy. One thing I'll offer them in these cases is "busking practice".

The idea is, we'll pretend we're [busking](https://www.jefftk.com/p/busking-with-kids) together. I'll start playing a tune they know, and they should do something that sounds good. That could be playing the tune with me, making up a harmony, or just figuring out a single note that sounds ok and playing patterns on it. If they make a mistake, we keep going. If they can't figure out what to play, we keep going. We're practicing performing. It helps that they're pretty motivated to get good at busking, because they know they can earn money that way.

Working on the pieces the teacher assigns (if you have a good teacher!) is very efficient a turning time on an instrument into becoming a better musician. But if willpower is the limiting factor and not time, and especially if the alternative is a super short practice or no playing at all, the important thing is just picking up the instrument and playing something. I like that busking practice give us some structure for this, and lets the kids build up their performance skills.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0WMATLgZyViPgb2nBXeJpMmN8r3AG1NuMvuXCZShFjwjD8bmc4gBuC47jNYA49riKl), [mastodon](https://mastodon.mit.edu/@jefftk/115599665662550447), [bluesky](https://bsky.app/profile/jefftk.com/post/3m6cm3usv322d)*
# Solstice Singalong Watch Party

Welcome to Secular Solstice celebration at Arkhipov House!

Location: E2 0QW, Bethnal Green, London UK  
[**Sign up**](https://partiful.com/e/LWGBWLsC5Xn4w4bKYFqK) **to see exact adress**

The main event is that we will watch the recording of the Bay Solstice ceremony and sing along to all the songs. This year's recording does not exist yet, but here's [last year's for reference](https://www.youtube.com/watch?v=cFggI7QZoQM). (Last year's recording is also our backup plan if there is some issue with the new one.)

**PLEASE DON'T ARRIVE AFATER 6PM**

There will be:

*   Vegan food (dinner)
*   Sofa
*   Optional cuddles
*   Singing
    

Schedule:

*   Arrive from 5pm, eat dinner
*   Watch party starts at 6pm, expected to last around 3 hours
*   Feel free to stay and talk/cuddle in the sofa afterwards
# If you cannot be good, at least be bad correctly

*Note: I'm writing every day in November, see* [*my blog*](https://boydkane.com/essays/2025nov) *for disclaimers.*

It’s hard to be correct, especially if you want to be correct at something that’s non-trivial. And as you attempt trickier and trickier things, you become less and less likely to be correct, with no clear way to improve your chances. Despite this, it’s often possible to bias your attempts such that if you fail, you’ll fail in a way that’s preferable to you for whatever reason.

As a practical example, consider a robot trying to crack an egg. The robot has to exert just enough force to break the egg. This (for a sufficiently dumb robot) is a hard thing to do. But importantly, the failure modes are completely different depending on whether the robot uses too much force or too little: too much force will break the egg and likely splatter the yolk & white all over the kitchen, too little force will just not break the egg. In this scenario it’s clearly better to use too little force rather than too much force, so the robot should start with a lower-estimate of the force required to break the egg, and gradually increase the force until the egg cracks nicely.

This also appears in non-physical contexts. This idea is already prevalent in safety related discussions: it’s usually far worse to underestimate a risk than it is to overestimate a risk (e.g. the risk of a novel pathogen, the risk of AI capabilities, the risk of infohazards).

Looking at more day-to-day scenarios, students regularly consider whether it’s worth voicing their uncertainty “I don’t understand equation 3” or just keeping quiet about it and trying to figure out the uncertainty later. But I’d argue that in these cases it’s worthwhile having a bias towards asking rather than not asking, because in the long-run this will lead to you learning more, faster.

Salary negotiation is another example, in which you have uncertainty about exactly what amount your potential employer would be happy to pay you, but in the long-run it’ll serve you well to overestimate rather than underestimate. Also, you should really read `patio11`’s [Salary Negotiation](https://www.kalzumeus.com/2012/01/23/salary-negotiation/) essay if you or a friend is going through a salary negotiation.

You see similar asymmetric penalties with reaching out to people who you don’t know, asking for introductions, or otherwise trying to get to know new people who might be able to help you. It’s hard to know what the “right” amount of cold emails to send is, but I’d certainly rather be accused of sending too many than feel the problems of having sent too few.

This idea is a slippery one, but I’ve found that it applies to nearly all hard decisions in which I don’t know the right amount of *something* to do. While I can’t figure out the precise amount, often I have strong preferences about doing too much or too little, and this makes the precise amount matter less. I give my best guess, update somewhat towards the direction I’d prefer to fail, and then commit to the decision.
# "Self-esteem" is distortionary

A friend asked me, "what's the right amount of self-esteem to have?" Too little, and you're ineffectual. Too much, and you get cocky. So how do you choose the right balance? 

I replied that this is a trick question. 

People with low self-esteem have thoughts like "I'm a loser", "my IQ is too low to succeed", "no one could love someone as fat as me". Their problem is not quite that they've got inaccurate beliefs. They make in fact be a loser. Rather, their problem is that they've attached their identity to concepts that limit their action space. 

For instance, the notion of low IQ. This is a construct that's predictive at a population level, but it doesn't give you some predictive power on an individual level unless it's the only thing you know about a person. But you can rapidly accumulate info about someone, or yourself, that outweighs the info expressed by "your IQ is 101". E.g. if you want to know someone's test scores, you'll do a lot better by using their scores on mock exams than by using their IQ. 

Which means that someone who says "I can't fix my car because I've got a low IQ" isn't actually making full use of the info available to them. They're relying on a sticky prior. What they should actually be doing if they care about fixing their car is asking "what's stopping me from fixing it?" and checking if solving that problem is worth the costs compared to paying a mechanic. The cost may be large. They may have to put in dozens of hours of work before they understand cars well enough to fix their problem without paying anyone. But they could do it.

So the issue is that the belief about "low IQ" has led to imaginary walls around what can be done that do not actually reflect reality. 

In other words, low self-esteem turns a bump in the road into a cliff of seemingly infinite height, cutting off an entire avenue of approach. It reduces your sense of what is possible, and [from the inside, it feels like you've got less free-will](https://www.greaterwrong.com/posts/3buXtNiSK8gcRLMSG/possibility-and-could-ness). 

What is the solution? Knock down the walls. 

In day to day life, we have to simplify the action space because we are computationally bounded systems. We introduce simplifications for good reasons, and for bad reasons. That's normal. Thing get problematic when those simplifications restrict the space till there is no good action left. Then, the appropriate reaction is to relax the constraints we impose on ourselves, test if the relaxation is valid, and take best action we've got left. If we were able to do this reliably, we would find ourselves doing the best we can, and low self-esteem would be a non-issue.
# Beware boolean disagreements

(Meta: This is largely a re-framing of [Consider your appetite for disagreements](https://www.lesswrong.com/posts/8vesjeKybhRggaEpT/consider-your-appetite-for-disagreements).)

* * *

Poker players get into arguments a lot. Back when I used to play I would review hands with a friend and we'd get into these passionate disagreements about what the right decision is.

To keep it simple, suppose that my opponent goes all in on the river and I have two choices:

1.  Call
2.  Fold

Suppose my friend Alice thinks I should call and I think I should fold. Alice and I would spend hours and hours going back and forth trying to weigh all the considerations. We'd agree on a large majority of these considerations, but often times we just couldn't seem to agree on what the actual best play is: call or fold.

There are various things that I think went wrong in our conversations, but the biggest is probably a [type error](https://www.lesswrong.com/posts/5NWBpJA34zjJQzYiX/beliefs-and-javascript-types): we were disagreeing about a boolean instead of a number.

When making decisions in poker (and life!), the thing that matters is [expected value (EV)](https://www.lesswrong.com/w/expected-value). Suppose the buy-in is $200, Alice thinks calling has an EV of +$1.00, and I think that calling has an EV of -$0.75. In this scenario, the magnitude of our disagreement is less than a measly big blind!

In other words, Alice thinks that calling is *very slightly better* than folding whereas I think that it is *very slightly worse.* We're basically in agreement with one another, but framing things as a boolean rather than a number masks this fact.

I wouldn't say that framing a disagreement around booleans is never useful; that'd be a very strong claim that I don't feel confident enough to make. But I do get the sense that the boolean framing usually isn't useful, and so my position is that you should *beware* of boolean disagreements.
# Reasons Why I Cannot Sleep

My therapist says I'm more tired today than she's ever seen me. Here are some reasons my brain says I cannot sleep:

1.  **My boss might lose faith in my ability to manage projects.** Something important might go wrong with my project (Inkhaven), and I am the only person on-call who has the context to fix it all. If I screw it up he might no longer be willing to give me big projects like this.
2.  **Annoying social drama.** There was one the other day, that made me anxious and unable to relax until it was dealt with. I had just come out of a massage supposed to relax me, and yet I was immediately un-relaxed for a few hours.
3.  **I associate being in bed with checking Reddit/YouTube.** Whoops. Perhaps I should have better spaces to do this, so that I never do it in bed? Alas, I live in a single room with a bed and a desk, this is my private space. I have a hard time just going to sleep in bed.
4.  **There are 50-60 people on campus who I am responsible for.** When people are around they ask you questions and to solve their problems. All. The. Time. Being seen is an invitation for you to solve a problem. Eye contact is an invitation for you to solve a problem. Saying "hi" is an invitation for you to solve a problem. (This is my job and normally I am happy to do it, but it's hard to turn it off.)
5.  **There are many Slack messages waiting for me.** I guess this is on me for having a habit of hitting slack inbox-zero and an org philosophy of [Slack Maximilism](https://aelerinya.substack.com/p/the-10-principles-of-slack-maximalism).
6.  **I said I'd sing a song in the open mic tonight.** Alas, I was looking forward to that. It's my favorite song I know. Well, I've had to cancel.
7.  **My mum wants to catch up with me.** I did fly her half way across the world to be here, so that's understandable. Fortunately I am keeping her here for 2 months so it's fine to say no sometimes.
8.  **I'm having a depressive spiral.** Given my lack of sleep, everything else in my life looks worse and like a problem I cannot overcome and will feel shame for failing at. It helps to remember that I feel this way substantially because I'm tired and not because the things are as bad as they seem. Though I am indeed pulling to mind the worst issues that I am facing, and also some of my insecurities being exposed/tested.
9.  **Because I need to make plans for a few specific people who are leaving Inkhaven tomorrow.** Two residents will be leaving for Thanksgiving and not returning; and one contributing writer will leave in the morning. I want to make sure that the residents have a nice sendoff, and that the contributing writers' time is well-spent.
10.  **Because I have to write a blogpost else I have failed out of Inkhaven.** I am probably the person at Inkhaven with the most work to do during Inkhaven, but it isn't good for the leader of Inkhaven to fail the daily blogging challenge, so he should figure out a nice short post to write that he can publish to keep up with the challenge.
# Will We Get Alignment by Default? — with Adrià Garriga-Alonso

Adrià recently published [“Alignment will happen by default; what’s next?”](https://www.lesswrong.com/posts/FJJ9ff73adnantXiA/alignment-will-happen-by-default-what-s-next) on LessWrong, arguing that AI alignment is turning out easier than expected. Simon left a lengthy comment pushing back, and that sparked this spontaneous debate.

Adrià argues that current models like Claude Opus 3 are genuinely good “to their core,” and that an iterative process — where each AI generation helps align the next — could carry us safely to superintelligence. Simon counters that we may only get one shot at alignment, that current methods are too weak to scale. A conversation about where AI safety actually stands.

[Watch the full debate here](https://simonlermen.substack.com/p/will-we-get-alignment-by-default)
# A Thanksgiving Memory

A couple of days before Thanksgiving when I was 11 years old, the house my family lived in was gutted by a fire. We had been stripping the white paint off the 150-year-old oak woodwork as a restoration project. It was snowing and windy out, so the workers had closed the windows. A spark from plugging in a work lamp lit the turpentine fumes, and a fireball tore through the house. Firefighters arrived in minutes and saved most of the load-bearing structure. But everything inside was melted, charred, soaked, ruined. It was a year before we could live in it again, and a few more years before you couldn't tell there had been a fire there.

To this day, I remember this on Thanksgiving -- because of what happened next. Before that day was out, neighbors had offered all of us places to sleep. The owner of a small independent department store downtown must have heard about the fire on the local evening news. He unlocked the doors of his closed store after hours and told us to go in and take whatever we needed. I remember being awed because it was a fancy store we never would have shopped in, and he didn't even know us.  

Then *so many* friends showed up, weekend after weekend for months, to help us haul the charred stuff out of the house into a dumpster in the front yard and start rebuilding. Once, a neighbor must have realized we'd all been at this for hours without eating, and made us lunch. It's strange what kids remember, but I was especially moved that they weren't even worried about us getting their house dirty. They had an extremely elegant house with a snow-white carpet, but they laid newspaper down on the rug and invited the whole lot of us, black as coal miners from head to toe, to come in and have homemade soup and bread. There are more stories that I won't tell here.

Most of the time I lament how horrible humans can be to each other. I'm glad to be reminded every year that humans can also be amazingly generous and kind. When I had kids, I started a tradition of baking a pie and bringing it to the local fire station for the firefighters on duty on Thanksgiving day. This gave me a way to express gratitude, and also to share that story with my kids. My kids are grown now, but they are visiting me today, and we just dropped off the pie they baked.

Happy Thanksgiving.
# You Are Much More Salient To Yourself Than To Everyone Else

Back in the Boy Scouts, at summer camp, myself and a couple friends snuck out one night after curfew to commandeer a couch someone had left by a dumpster at the other end of the camp (maybe a half kilometer away).

Now, our particular designated adult was a very stick-to-the-rules type, so we definitely did not want to get caught. I, therefore, made my way slowly and sneakily. The summer camp was in the woods, so I’d keep myself behind trees and bushes and out of the light anytime someone went by. At one point I literally hunkered down in a ditch, staying below the line of the headlights of a passing car. At another point I was maybe five feet from a guy, standing in a shadow, entirely unnoticed. It was slow, but a very fun game, and I was not noticed.

… and then I arrived to find that my two friends had done the walk much more quickly. They didn’t bother hiding at all. They just… walked over. Nobody particularly cared. Sure, our curfew violation was salient to us, but nobody else out that night was paying any attention to us. Why would they?

What I learned from that experience is that nearly everyone is, by default, invisible to nearly everyone else nearly all of the time.

If walking among strangers, it takes *really quite a lot* before anyone will pay you any attention at all. It’s probably not going to happen by accident, at least not to you. It certainly isn’t going to happen by sending [subtle signals](https://www.lesswrong.com/posts/5axNFSfaxgnTKPrpy/people-seem-funny-in-the-head-about-subtle-signals) of any sort; zero of the strangers will pay attention to any of those.

The ancestral environment was… presumably not like this. Our ancestors were probably surrounded by strangers to a much lesser degree, and therefore were for less invisible. Our brains are probably not calibrated to the degree of our own invisibility, in the modern environment. That can cause emotional difficulties.

People relate to invisibility differently. I tend to find it comforting, safe; an invisible person attracts no threats. Other times I find it freeing: invisibility means I can just do stuff, and nobody will say anything or even pay attention, even when I do things which feel blatant to me.

… but on the flip side, for many people, invisibility feels like nobody cares. It feels like being unwanted. Like nobody being interested in you at all, like nobody would notice if you disappeared. And yeah, that’s all true to a pretty large extent; most of us are surrounded to a very large extent by strangers who mostly don’t notice us much. Personally I recommend making peace with it and finding a way to enjoy life anyway, but that’s harder for some people than others.

I suspect that a lot of people flinch away from their own invisibility. They feel the pain of “nobody notices you enough to care” (and the comfort of safety and freedom is not as salient). And then they tell themselves whatever story they can, to pretend that more people care more than they actually do. And one side effect of that flinch is to vastly overestimate the extent to which people notice when we Just Do Stuff.