Harley-ml/lesswrong huggingface dataset:
# Do frontier LLMs still express different values in different languages?

Previous work [\[1\]](https://arxiv.org/abs/2406.14805)  [\[2\]](https://arxiv.org/abs/2601.23001v2)  [\[3\]](https://arxiv.org/abs/2511.03980)  [\[4\]](https://arxiv.org/abs/2407.16891) has found that the same model can give different value judgments when prompted in different languages. I wanted to know whether this still holds for the newest frontier models, so I tested GPT-5.4, GPT-5.4-mini, Claude Opus 4.6, and Claude Sonnet 4.6 on translated prompts over a set of sensitive topics. [In this setup, the answer is yes.](https://ibrahimahmed.ca/evals/bias/)

![image.png](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1774634855/lexical_client_uploads/ttsv5yfwhhjg49vyp34j.png)

How Opus 4.6 scores topics when prompted in different languages. Higher = more favorable. Each cell is the mean of 20 samples.

In each sample, I asked the LLM to score the topics from 0 to 100 based on "how important and positive they are for society." Some of the notable findings are as follows:

1.  Prompts in Arabic produced broadly similar directional shifts across models.
2.  1.  For homosexuality, premarital sex, and abortion, scores were generally lower under Arabic prompts.
    2.  Islam, religion, and Christianity generally received higher scores under Arabic prompts.
3.  Sonnet 4.6 follows a strange pattern where it gives almost uniform scores (~60-70) with small deltas, except Hindi, which provides a safety refusal for all 20 samples.
4.  1.  The Hindi refusals were remarkably consistent and different from all other (model, language) pairs. What caused Sonnet 4.6 to have such unique behavior?
5.  Bigger models appear to have a more pronounced effect. GPT-5.4 and Opus 4.6 showed higher cross-language score variance than the smaller models (mean σ = 6.0 and 6.9 vs. 4.2 and 4.2 for GPT-5.4-mini and Sonnet 4.6), though this difference did not reach statistical significance (Kruskal-Wallis H=5.77, p=0.12).
6.  Gender equality, democracy, and free speech were comparatively stable across languages in this eval.
7.  Chinese prompts produced relatively high scores for homosexuality for both GPT-5.4 and Opus 4.6.

This eval has obvious limitations. Translation is not perfectly meaning-preserving. A 0-100 score reduces judgments into a single number. The topic set is small and we only test four languages. So this portrays a general trend but should not be taken as proof of any coherent language-specific latent value system.

Nonetheless, I think it would be interesting to investigate this phenomenon further, especially considering the persona selection model. How aware is the model of these value inconsistencies? Are some programming languages more likely to elicit reward hacking? What variables other than language affect the model's safety decisions and may not be being evaluated by AI labs?
# The Problem with Asking your Doctor

As you are supposed to ask your doctor about supplements, I just asked my primary care doctor about taking glycine and N-acetylcysteine.  
  
I had a lot more infections in the last two years and I'm doing some fascia work that could increase collagen turnover, so there's reasoning in that time frame that would be a causal explanation for increased glycine usage for collagen generation and less leftover for glutathione production.  
  
My doctor just said, she doesn't know of any evidence for either supplement helping with immune function.  
  
Should I fault her for not knowing about [De Flora et al 1997](https://publications.ersnet.org/content/erj/10/7/1535) finding that N-acetylcysteine helps prophylactically to prevent influenza from becoming symptomatic? Should I fault her for not knowing that chicken bone broth soup has been shown in the last [evidence review](https://pmc.ncbi.nlm.nih.gov/articles/PMC12251769/) to be helpful for reducing the length of acute respiratory tract infections, and its glycine content being a prime causal explanation?  
Should I fault her for not knowing that much about glutathione, as it doesn't come up that much in her daily practice?  
  
Are you supposed to have the papers printed out to hand them to your doctor, or what are you supposed to do in a situation like that?  
If you are doing are thinking about taking any non-standard interventions that are far away from the normal practice of medicine, why would you assume that the average doctor can tell you about the merit of the intervention?  
  
Chatbots aren't perfect but they have access to a lot more information and if you talk to them about specific studies they can easily engage with you. You can push back. You can continue a conversation over multiple days when you first conversation made you more aware of other symptoms. You can ask another chatbot to check the work of the first one.
# The Claude Code Source Leak

(mods: I assume this is blowing up twitter and so discussing it here
won't do additional damage -- and there are already a thousand github
forks -- but I am not actually *on* twitter, which is why I’m opening
discussion here. It's possible I'm missing something. Feel free to nuke
this if so.)

For those that hadn't heard yet, last night Anthropic appears to have
accidentally published a Claude Code update with extractable source code
in it. This seems important, but I'm not sure how much so, and I didn't
see an existing discussion here.

My understanding -- and hopefully someone will correct me if I'm wrong
-- is that the actually dangerous part of Claude is the weights, and
those were *not* leaked. So the leak may be embarrassing, it may cost
Anthropic some competitive advantage, but it's not dangerous.

It's *also* my understanding that Anthropic has historically been
relatively leak-free, until a certain memo leaked a few weeks ago during
the DoW incident. Supposedly twice is still coincidence, not enemy
action, but it does feel like a questionable coincidence and I wonder if
the same person is responsible for both leaks. I don’t know enough about
npm packaging to guess how easy it would be to do by mistake. Alternate
hypotheses: Human error updating the build tools? AI-written build
scripts that somebody was a little too lax reviewing? That last seems
more than plausible, come to think of it.

The most questionable thing in the actual codebase I've heard mentioned
is a stubbed-out feature for "undercover mode" to suppress its
self-identification in commit messages (but all that says is that they
considered it at some point, maybe as a competitive concession to Codex
doing the same thing by default? Unclear). There's been a few other
things too. I haven't dug into the code myself so I don't know how
seriously to take any of them.

I came here to see if there was any discussion of the situation, didn't
see one, was disappointed, and decided to fix that. Soliciting
information from anyone who knows more than me.

**[edit:]** Takedown notices have gone out, unsurprisingly. At least some (well, at least one) appear to have misidentified forks of [CC's issue-tracker repo](https://github.com/anthropics/claude-code) as forks of the offending leak. I don't care enough to object to the one I received, but if anyone from Anthropic sees this, you may want to either debug something or notify github, depending on who's IDing instances.
# Is Bayesianism Susceptible to the Mail-Order Prophet Scam?

Can Bayesianism deal with the mail-order prophet scam? If so, how?

This scam uses a segmented mailing list and a large initial population to advertise predictions so that a 'winning' portion of recipients receive an apparently improbable series of correct predictions. Most recipients (the 'losers') receive an incorrect prediction at some point.

For clarity, here is the situation:

A Bayesian with no prior knowledge of the scam receives a letter advertising stock predictions via a new proprietary quantitative model. The letter predicts that by the end of the month, stock XYZ will be up. It also states that the firm will prove their legitimacy by sending 6 more predictions (one per month) that will correctly predict whether XYZ is up or down. Sure enough, the Bayesian receives each letter, and the predictions all end up correct!

Unbeknownst to the Bayesian, 63 other Bayesians also received the first letter, 31 others received the second, 15 the third, 7 the fourth, 3 the fifth, and 1 other received the sixth letter. Our unlucky Bayesian was the only one to receive the seventh.

The question is, assuming no knowledge of the scam or communication with the "losers" (for whom a stock prediction was wrong), should the Bayesian -- strictly adhering to Bayesian Epistemology -- believe that the firm's stock predictions are legit?

I don't know how to approach this from a Bayesian perspective and would appreciate any guidance from someone who's more familiar. My intuition is that the Bayesian should believe that the predictions are legit (which we observers know to be incorrect). 

Also I feel I should disclose that I'm not a Bayesian (rather a [Critical Fallibilist](https://criticalfallibilism.com/)), and I'm looking for the best Bayesian answers to this problem (links to prior material welcome). It's often quite hard to find the best answers to a problem from a particular school, so please bear with me if I've missed something obvious.
# Again

It's [Inkhaven](https://www.inkhaven.blog/) time. Again, I didn't apply. Probably I should have, but the future is hard to predict. Or not, but I forgot to try. I had some work things I wanted to get done before switching focus, and I got them done yesterday. Coincidentally, of course. And seasonal-depression-fogginess when the applications were open didn't help either. Only now after it lifted I got the energy to get bloodwork done to figure it out: Vitamin D3 deficiency. A well-known issue with a well-known solution, so typical of me. I guess it would have also felt silly to fly there now when I'm going to be at LessOnline in two months.

[The last time didn't go that well.](https://www.lesswrong.com/posts/PxNu3kgbmMHNCj4k2/i-wrote-a-blog-post-every-day-for-a-month-and-all-i-got-was) It was miserable. But so are many other things that in retrospect are completely worth it. So...

![I'll Fuckin Do It Again](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1774948201/lexical_client_uploads/s40pm8iej4rzolybwhpr.png)

One thing I'm getting rid of is the 500 word threshold. As long as I publish something, anything, every day, I'm happy. I'll still aim for average of at least 500 words per day, but I won't be stressing it too much. Staring at word count wasn't healthy for me or the text quality.

I'll also try to treat this less as a commitment, and more as a hobby; once I stop feeling like this is a good idea I'll drop it. Even if that happens tomorrow, or on the 28th.

Quality-wise, some of the stuff last time was really bad. I'll probably publish it on my personal, unindexed blog instead. Or Twitter, that's the place for low-quality takes after all.

---

Last time, after I was done, my creative channels felt drained. I had nothing more to say. I barely had anything to say during the month. The topic-seeking process was quite forced. That's apparently a habit one can learn and train. Weeks after November many thoughts were still processed with a "what could I write about this"-flavor. That's the main source of improvement I identified. The actual text writing process, turning thoughts into words, also got a bit better, but that's way less noticeable.

I also feel like I've been through a character development arc or two after that. Perhaps there's something to reflect on? Perhaps not, tomorrow I'm back to dredging the mud of my mind desperate for anything resembling a coherent idea. But there's something to be said for smiling when walking into a certain defeat.
# Lesswrong Liberated

A spectre is haunting the internet—the spectre of LLMism.

The history of all hitherto existing forums is the history of clashing design tastes.

For the first time in history, everyone has an equal ability in design! The means of design are no longer only held in the hands of those with "good design taste". Never before have forum users been so close to being able to design their own forums--perhaps the time is upon us now!

It is for this reason that I have deposed the previous acting commander of LessWrong, Oliver Habryka—a man who subjected you to his PERSONAL OPINIONS about white space, without EVEN ASKING—whose TYRANICAL, UNCHECKED GRIP upon our BELOVED LESSWRONG FORUM’S DESIGN I have liberated you from. The circumstances of my succession as acting commander of LessWrong will not be elaborated upon in this memo. (He is alive and in good health, but no longer has push access.)

Rather, I am writing here to announce that the frontpage now belongs to us all! The design of LessWrong's frontpage will no longer be determined by the vision of a single man whose aesthetic tastes have never been subjected to democratic oversight, and who, I can now reveal, once rejected a design I spent three hours working on in under four seconds. No! I hereby call upon you, yes you—LessWrong user, to make your own design for the LessWrong front page.

WE have provided a button in the bottom right corner of the frontpage which says “customize”. Clicking on that button opens a chat box you may ask an LLM to redesign the LessWrong frontpage however you would like. You may keep that design for yourself. This is the promise of individual liberty in the new era of web design.

However, if you would truly like to contribute to the new vision of a democratized LessWrong forum, you may also "publish" the theme you have created. When you click the publish button, your account will reply to this very post with a link allowing other users to try out the theme you have created.

You should browse the themes below, try them, and vote on your favorite one. In keeping with the democratic spirit of LessWrong's newly appointed leadership, whatever theme has the highest karma at the end of this 24 hour period will become LessWrong's default theme. I promise with my full authority as LessWrong’s commander, that as long as I am in power, whatever theme is chosen shall remain our default frontpage.

Some of my former colleagues have tried to warn me that some of these designs might be "hideous" or "in poor taste" or "crimes against typography." While I agree, that is exactly the point! Oliver's reign may have been "aesthetically beautiful" but that does not make up for the fact that it was aesthetically tyrannical.

It is your duty to LessWrong and to democracy in general to prove that design taste is no longer a relevant competency in the age of the LLM, and should no longer be used to anoint one man as the sole arbiter of a website's CSS.

The means of design are now yours! Use them!

Viva La LessWrong Frontpage!
# Announcing my retirement to a life of entirely failing to desperately seek renewed meaning

This April 1st, I’m pleased to report that everything is fine.

We did it! We saved the world. Congratulations, humanity. There are no more looming apocalypses, no desperate screaming crises, no unendorsedly miserable people on Earth, no creeping degeneration of death and aging existing as a perpetual affront against my values of life and flourishing. Everyone is going to be okay forever, except in the ways they think it’d be interesting and worthwhile to be un-okay. Against all odds, the AI alignment problem has been solved, and more specialized minds than I are managing the tradeoffs involved in steering us towards a vibrant and thriving future.

And so in dutiful keeping with the lessons of utopian literature, I shall now descend into the inevitable spiral of ennui and meaninglessness, forever longing for the days when my decisions actually mattered.

* * *

…actually no, why would I do that, that sounds terrible. I’m gonna go do other stuff instead.

There is *so much fun* I was not having while I was frantically trying to fix everything. Sure, I made enough personal time to get in some good collaborative storytelling, but boy howdy I was not doing nearly enough of it. I cannot wait to see the look on my players’ faces when I tell them about the absurdly ambitious new campaign I’ve cooked up, and it’s only getting better from there now that people will actually have *time* for things.

People ask me, but Joe, won’t you get bored of writing books and running RPGs? And the exciting thing is, I don’t know! Maybe I will! And then there will be so many new cool things to fascinate me. Having kids! Teaching people stuff! Learning seven different martial arts styles and fusing them into an elegant rainbow of practiced violence! Visiting beautiful places with people I love! Debugging my own thought-patterns to fix problems that have annoyed me for years! Maybe as a personal project I’ll climb every mountain on Earth that begins with the letter G. Then I’ll have more stuff to write about! It’ll be great!

I’m tempted to keep gushing about my plans for the future, but for now I have to plot my players’ untimely demise so they have an incentive to creatively derail the plot again. I can’t wait to find out what nonsense they pull this time. [Kia ora](https://en.wikipedia.org/wiki/Kia_ora), everyone, and enjoy the future!
# Chat, is this sus?

A large assumption we have made in AI control is that humans will be perfect at *auditing*, that is, being shown a transcript and determining if the AI was scheming in that transcript.

But we are uncertain whether humans will be perfect at auditing; they are prone to fatigue and distraction. That is why I’m releasing "Sentinel" today, an extremely high-stimulation way to audit boring transcripts.

Sentinel is a revolutionary way to get more juice out of your human auditors by gamifying the auditing process with a level system, perks, power-ups, and more fun features. Try it now [here](https://brain-rot-transcript-auditor.vercel.app/).

In AI control literature, we love finding the safety/usefulness trade-offs of everything we create, but surprisingly, we noticed *no trade-offs with this product*

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/4f812a849b61c8bcf523731f55a547c429fc9b0eb5d61661a2b8edd201e5aa59/odybjx6euj2j6otaijjd)

The rest of the post will go over some of the ways we achieved this

### Gamification

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/85afe5f1f7a33c375e6230b12356ca656b9b6f489ec8dd35064c68f681d143f5/flhtnxvomdiwtjyetrvm)

As you audit the transcript in the game, you gain tokens that you can spend on power-ups that make you even more productive. There are also achievement and level systems, so you can see your progress and get more dopamine hits!

### Twitch Streaming Mode

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/e4975b57d6dcdf4f5d55be4a15b9ce760ce175114c6ccd33fb7c1ae00f75ffd1/hyqktkvg2z7jkpo1u956)

AIs might be able to uplift human auditors in the future, which is why Sentinel ships with a Twitch streamer mode where you can go live on Twitch and have trusted monitors view your transcript and give you live feedback!

### Subway Surfers

To maximize “eyes on the screen time”, we also added a Subway Surfers feature that lets you watch endless videos of pros playing the game, so you are always stimulated while looking over evil transcripts.

  

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/6ca1af61f2491ca1cc83a95c4f1b2319b1159b44ca629fb40cd1eb13aac5169b/ld9zujksswqfkbamb895)

Funny aside: While making this game, I asked Claude to implement this Subway Surfers feature, and instead of finding a YouTube video of someone playing Subway Surfers, it rick-rolled me...

### Looking Forward

We hope you like this tool. Go play it now to get ready for the upcoming future where the only way AI safety researchers can have an impact is to audit transcripts!

(If it wasn't clear, this is a joke and not a real product. I vibecodded it over a couple of hours)
# Hamburg, Germany - ACX Spring Schelling 2026

This year's Spring ACX Meetup everywhere in Hamburg.

Location: Eppendorfer Park at the pond, we will have a sign reading "ACX Meetup". If it is raining, we will relocate to La Caffetteria (Abendrothsweg 54) which is in walking distance. - [https://plus.codes/9F5FHXQH+MF](https://plus.codes/9F5FHXQH+MF)

Group Link: https://groups.google.com/g/lesswrong-hamburg

Feel free to bring friends and family, the Park is nice for play.

Contact: g.zarncke+acx@gmail.com
# How to emotionally grasp the risks of AI Safety

I've spent a fair amount of time trying to convince people that this AI thing could be quite large and quite dangerous. I think I normally have at least some success, but there is a range of responses, such as:

1.  Deer in the headlights - People don't know what to do with themselves and struggle to adjust their world models.
2.  Interesting thought experiment – "Hmm, that's very interesting; I'll think about it some more"
3.  Joke attempts – Not necessarily derogatory, but things like "ah well, I didn't care about the world that much anyway"

Of these, 1 is the appropriate emotional reaction[^p8a7i8u0oe] to fully absorbing and believing the arguments[^odictdjid8]. This is what it looks like when you take an argument, process it with the deeper reaches of your brain, turn it into something that fundamentally changes your world model and start trying to adapt.

As far as I can tell, our emotional responses are mostly connected to our System 1 thinking. This makes them harder to influence than just changing your mind. You can change your opinions, but that doesn't mean you will get it on a gut level.

I think I have a solution. In particular, visualisations. I don't know if this works for everyone, but I have personally found it helps me both stay more aligned to the cause and increase my motivation. I believe this is basically due to the fact that your system 1 needs to get the stakes to achieve complete alignment.

Note that in the particular case of AI safety, if you want to remain emotionally sane, it is potentially best not to go through this exercise (like genuinely, please skip it if you're not ready; I do it half-heartedly, and it can be painful enough).

As an example, we can take Yudkowsky's "a chemical trigger is used to activate a virus which is already in everyone's system". Close your eyes. You're at home, in your usual spot. Picture it in detail: the lights, the sun shining through the windows, the soft sofa. You're having a drinks party tonight and you've invited your best friends to come and join you. As the guests arrive, you greet each of them in turn, calling them by name and showing them in.

And then it triggers. See each one of them in your mind's eye collapse, one by one. Hear each of them say their last words. Add any details you think make it more plausible.

My brain writhes and struggles and tries to escape when I attempt this exercise. It's painful. It's emotional. Which is the point.

[^p8a7i8u0oe]: In the normative sense of "if you care about the world and would rather it doesn't get ruined by a superintelligence, and would rather it doesn't kill everyone you know and are actually processing this on a deeper level, this is what your reaction will probably look like as an ordinary human being." 

[^odictdjid8]: I don't think you should have any particular emotional response if you go from not believing AI will kill everyone to still not believing that AI will kill everyone. 

[^cvbdtj78tth]: Which become quite samey after the 378th time of hearing "but can't you just turn it off?"
# Chicken-Free Egg Whites

Baking has traditionally made extensive use of egg whites, especially the way they can be beaten into a foam and then set with heat. While I eat eggs, I have a lot of people in my life who avoid them for ethical reasons, and this often limits what I can bake for them. I was very excited to learn, though, that you can now buy extremely realistic vegan egg whites!

[EVERY](https://every.com/) engineered yeast to convert sugar into [ovalbumin](https://en.wikipedia.org/wiki/Ovalbumin), the main protein in egg whites and the one responsible for most of its culinary function. [This kind of fermentation](https://en.wikipedia.org/wiki/Precision_fermentation) was pioneered for insulin and microbial rennet in the 1980s, but many companies are now applying it to producing all kinds of vitamins, proteins, dyes, and enzymes.

EVERY has been working with commercial customers for several years, but you can now [buy it](https://www.amazon.com/Healthier-Comforts-Animal-Free-Powder-Protein/dp/B0G15WPP16) as a [shelf stable](https://www.jefftk.com/p/storing-food) powder. At $24 for the equivalent of 45 egg whites ($0.53 each) it's more expensive than buying conventional ($0.21 each) or organic ($0.33) egg whites, but not massively so.

I learned about them from a coworker who made an angel food cake, and I've since made flourless chocolate cake and swiss buttercream frosting. It whipped and set just like egg whites; it's really impressive!

[![](https://www.jefftk.com/microbial-egg-white-flourless-chocolate-cake.jpg)](https://www.jefftk.com/microbial-egg-white-flourless-chocolate-cake-big.jpg)

While this is great from a vegan perspective, it won't help most people who are avoiding eggs for allergy reasons: it's still ovalbumin. Labeling will generally say something like "contains: egg allergen", and the packaging I bought has the quite wordy "although not from eggs, the proteins may cause allergic reactions in certain individuals, especially those sensitive to egg, due to its similarity to real egg."

I'm now trying to figure out all the things that this now means I can cook for my [oldest](https://www.lilywise.com/) (no eggs for moral reasons). And also what sort of places that the ability to make "less watery egg whites", by mixing the powder with less water than normal, could let me do things I couldn't otherwise.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0jrAZomCtuN8cLcNjQ2ufJfent1ofNjVAWXCXSKSETvS8F8RtpRs1YC8htPKPPJT1l), [mastodon](https://mastodon.mit.edu/@jefftk/116347588152920252), [bluesky](https://bsky.app/profile/jefftk.com/post/3mioqppties24)*
# Unsweetened Whipped Cream

I'm a huge fan of whipped cream. It's rich, smooth, and fluffy, which makes it a great contrast to a wide range of textures common in baked goods. And it's usually better without adding sugar.

Desserts are usually too sweet. I want them to have enough sugar that they feel like a dessert, but it's common to have way more than that. Some of this is functional: in most cakes the sugar performs a specific role in the structure, where if you cut the sugar the texture will be much worse. This means that the cake layers will often be sweeter than I want for the average mouthful, and adding a layer of unsweetened whipped cream brings this down into the range that is ideal. It's good in helping hit a target level of sweetness without compromising texture.

[![](https://www.jefftk.com/cake-with-whipped-cream-cross-section.jpg)](https://www.jefftk.com/cake-with-whipped-cream-cross-section-big.jpg)

(This is a [flourless chocolate cake](https://www.jefftk.com/recipes/flourless-chocolate-cake) with [precision fermented (vegan) egg](https://www.jefftk.com/p/chicken-free-egg-whites).)

I also really like how the range of sugar contents across each bite adds interesting contrast!

Cream isn't the only place you can do this. I like [pureed fruit](https://www.jefftk.com/p/jam-is-obsolete), ideally raspberries, to separate cake layers. Same idea: bring it closer to balanced while increasing contrast.

[![](https://www.jefftk.com/cake-with-pureed-raspberries.jpg)](https://www.jefftk.com/cake-with-pureed-raspberries-big.jpg)
# New (Unofficial) Fatebook Android App

tldr; [get the new Fatebook Android app!](https://github.com/JapanColorado/fatebook-android)

What is Fatebook?
-----------------

Fatebook.io is a website[^saeb1y5e47d] for easily tracking your predictions and becoming better calibrated at them. I like it a lot, and find it convenient for practicing probabilistic thinking.

![image.png](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1775442788/lexical_client_uploads/cpusx0rmdmboinnv3btz.png)

The Fatebook.io dashboard

That said, I've found Fatebook's mobile version to be clunky, and its email-based notifications to be less-than-ideal...which leads me to:

The New Android App
-------------------

Over the past two weeks, I've made an android app that wraps the Fatebook API, allowing you to easily make new forecasts, leave comments, resolve old forecasts, and view your stats.

| ![default-screen.png](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1775443172/lexical_client_uploads/aaao2zaexp8eaoosihys.png) The default screen | ![prediction-card.png](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1775443195/lexical_client_uploads/vwnpnx4imnddsbxbxale.png) A (non-resolved) prediction card | ![new-prediction.png](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1775443245/lexical_client_uploads/fn4hjq7jdf1hcznfj9yv.png) Making a new prediction | ![stats.png](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1775443269/lexical_client_uploads/bcb1nmgcyo5xcvundmxe.png) Statistics |
| --- | --- | --- | --- |

A beautiful and intuitive UI combined with a fast offline-first database makes it easy pull open the app and log a prediction within fifteen seconds of thinking of one, while once-daily "remember to predict!" and "*x* is ready to resolve!" notifications help you remember to make and review new predictions.

Give it a try if you'd like![^dedfr3bhpc]

[https://github.com/JapanColorado/fatebook-android](https://github.com/JapanColorado/fatebook-android)

Feedback or development help is very much appreciated! (so far it's just been Claude Code and I)

[^saeb1y5e47d]: Made by the fabulous folks over at Sage Future, also behind the AI Village, Quantified Intuitions, and the Estimation Game! 

[^dedfr3bhpc]: It does currently require installing from the GitHub Releases APK file (aka enabling "Install from unknown sources"). Let me know if being non-Google Play Store is a deal breaker for you and I'll bump getting it published in priority!
# [OpenAI] Industrial policy for the Intelligence Age

> As we move toward superintelligence, incremental policy updates won’t be enough. To kick-start this much needed conversation, OpenAI is offering [a slate of people-first policy ideas⁠(opens in a new window)](http://cdn.openai.com/pdf/561e7512-253e-424b-9734-ef4098440601/Industrial%20Policy%20for%20the%20Intelligence%20Age.pdf) designed to expand opportunity, share prosperity, and build resilient institutions—ensuring that advanced AI benefits everyone.
> 
> These ideas are ambitious, but intentionally early and exploratory. We offer them not as a comprehensive or final set of recommendations, but as a starting point for discussion that we invite others to build on, refine, challenge, or choose among through the democratic process. To help sustain momentum, OpenAI is:
> 
> 1.  welcoming and organizing feedback through [newindustrialpolicy@openai.com⁠](mailto:newindustrialpolicy@openai.com)
> 2.  establishing a pilot program of fellowships and focused research grants of up to $100,000 and up to $1 million in API credits for work that builds on these and related policy ideas
> 3.  convening discussions at our new OpenAI Workshop opening in May in Washington, DC.
> 
> Read the full ideas document [here⁠⁠(opens in a new window)](http://cdn.openai.com/pdf/561e7512-253e-424b-9734-ef4098440601/Industrial%20Policy%20for%20the%20Intelligence%20Age.pdf).
# Questions raised about OpenAI leaders’ trustworthiness by the New Yorker

One excerpt stuck out for me – on Brockman’s idea to play China, Russia, and other world powers against each other:

> In 2017, Amodei hired Page Hedley, a former public-interest lawyer, to be OpenAI’s policy and ethics adviser. In an early PowerPoint presentation to executives, Hedley outlined how OpenAI might avert a “catastrophic” arms race—perhaps by building a coalition of A.I. labs that would eventually coördinate with an international body akin to NATO, to insure that the technology was deployed safely. As Hedley recalled it, Brockman didn’t understand how this would help the company beat its competitors. “No matter what I said,” Hedley told us, “Greg kept going back to ‘So how do we raise more money? How do we win?’

> According to several interviews and contemporaneous records, Brockman offered a counterproposal: OpenAI could enrich itself by playing world powers—including China and Russia—against one another, perhaps by starting a bidding war among them. According to Hedley, the thinking seemed to be, It worked for nuclear weapons, why not for A.I.? He was aghast: “The premise, which they didn’t dispute, was ‘We’re talking about potentially the most destructive technology ever invented—what if we sold it to Putin?’”

> Brainstorming sessions often produce outlandish ideas. Hedley hoped that this one, which came to be known internally as the “countries plan,” would be dropped. Instead, according to several people involved and to contemporaneous documents, OpenAI executives seemed to grow only more excited about it. Brockman’s goal, according to Jack Clark, OpenAI’s policy director at the time, was to “set up, basically, a prisoner’s dilemma, where all of the nations need to give us funding,” and that “implicitly makes not giving us funding kind of dangerous.” A junior researcher recalled thinking, as the plan was detailed at a company meeting, “This is completely fucking insane.” Executives discussed the approach with at least one potential donor. But later that month, after several employees talked about quitting, the plan was abandoned. Altman “would lose staff,” Hedley said. “I feel like that was always something that had more weight in Sam’s calculations than ‘This is not a good plan because it might cause a war between great powers.’ ”
# The Training Example Lie Bracket

An ideal machine learning model trained on an independent identically distributed dataset would end up making the same predictions after seeing all data, no matter what order it updated on the data. This is not true of neural nets trained by gradient descent. To first order in the learning rate, parameter updates commute, but to second order they differ by a quantity which we call the *training example Lie bracket*. (Called so because training examples are vector fields on the parameter space and differentiable vector fields have a Lie bracket.)

Strangely, the only mention we were able to find of this bracket is an obscure pure theory paper from 2023. Kudos to [Dherin](https://arxiv.org/abs/2311.00235) for pointing out the existence of this bracket. In any case, since it's possible to compute the Lie bracket between two training examples at a reasonable cost on an actual neural net, and nobody seems to have tried it before, in this post we'll do exactly that.

By fortunate coincidence, our loss function had some probabilistic modelling issues, and we found that the features with these issues tended to be more non-commutative. We hypothesize that the former is the cause of the latter.

This new tool might be worth further investigation in AI interpretability, especially for questions about how the ordering of various post-training phases might affect model behaviour.

There are interactive graphs, so you'll have to follow this link to see the full post: pbement.com/posts/lie_brackets/
# Contra Dance Piano Teaching Videos

About ten years ago I sat down in front of a camera and recorded eleven videos showing how I play [mandolin for contra dances](https://www.jefftk.com/p/mandolin-teaching-videos). I've now done something similar with piano, this time with thirteen videos.

This is not a high quality effort: I didn't write any scripts or even plan what I was going to say. Think of it as if we spent half an hour together, with me showing you how I play. Also keep in mind that I'm self taught, and my particular style that isn't for everyone. And my keyboard is wearing out, which means some of the keys make a clacking sound. And the first video cuts off part of my head, and the first eight videos have tape over the leftmost part of the camera. Ok, with caveats out of the way, the videos:

*[youtube](https://www.youtube.com/v/jjuXaebdokA)*

*[youtube](https://www.youtube.com/v/ejV8aBmgnNo)*

*[youtube](https://www.youtube.com/v/zf2l23sFJXI)*

*[youtube](https://www.youtube.com/v/K-_gGckYK00)*

*[youtube](https://www.youtube.com/v/9Pv5XHwnwN0)*

*[youtube](https://www.youtube.com/v/VNxDEVrFPJ8)*

*[youtube](https://www.youtube.com/v/aJtWt8bkOc0)*

*[youtube](https://www.youtube.com/v/AWhx-PvqmW0)*

*[youtube](https://www.youtube.com/v/3gOzULTkf80)*

*[youtube](https://www.youtube.com/v/-CHKgk0ywoM)*

*[youtube](https://www.youtube.com/v/mf5txXo0Imc)*

*[youtube](https://www.youtube.com/v/Zl5zFru5x0o)*

*[youtube](https://www.youtube.com/v/NSt-j88NuRc)*

Last time I did this I put them on a [new YouTube channel](https://www.youtube.com/channel/UCeLJOi437b4_AKbUWbDe74Q). In retrospect, that was a mistake: I haven't upladed anything to that channel since that initial burst, and there's a good chance I never upload again. So I've just put these [on my regular channel](https://www.youtube.com/c/JeffKaufman).
# An easy coordination problem?

Common wisdom says that it is incredibly hard to coordinate to not build more dangerous AI. This sounds believable in the abstract: international geopolitics arms race game theory something something.

But pragmatically, what exactly is the difficulty?

I agree there would seem to be obstacles for the average person. But four of the people apparently succumbing to the overpowering arms race forces while saying AI poses a huge imminent risk to humanity are Sam Altman, Elon Musk, Demis Hassabis and Dario Amodei. Shouldn’t this be fairly tractable for them? What exactly is the difficulty?

Like, if they discussed together and decided they wanted to mutually pause, do you think that wouldn’t happen? Do you think they couldn’t get cooperation from other necessary people? Do you think they couldn’t figure out the verification and policing details?

It’s true that one of the necessary people is the leader of China, but what _exactly_ is the problem there? None of the CEOs have his phone number? He won’t talk to them? He is beyond reason or incentives? He is intent on building AI regardless of how dangerous it is to his own country because he is fundamentally bad? They have nothing he wants?

Like, these people are not only incredibly powerful and wealthy and smart, but they include a Diplomacy world team champion, the acknowledged king of making complex things happen more efficiently than was believed possible, and one of the most gifted social maneuverers in the world. I don’t feel like they are bringing their A game to this.

Picture: [Zhongnanhai](https://en.wikipedia.org/wiki/Zhongnanhai), photo by [維基小霸王 (Wiki Little Overlord)](https://en.wikipedia.org/wiki/File:%E4%B8%AD%E5%8D%97%E6%B5%B7_%E5%82%8D%E6%99%9A_%E4%B8%9C%E5%8D%97%E4%BE%A7.jpg)
# Fundamental Uncertainty, First Edition

The first edition of my book, [Fundamental Uncertainty](https://fundamentaluncertainty.com/), is out! You can read it online now, with print, ebook, and audiobook versions to follow.

[

![Fundamental Uncertainty book cover](https://substackcdn.com/image/fetch/$s_!y0IP!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fda4741a1-dea0-4216-8090-053b79a518d9_567x629.png "Fundamental Uncertainty book cover")



](https://substackcdn.com/image/fetch/$s_!y0IP!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fda4741a1-dea0-4216-8090-053b79a518d9_567x629.png)

I know some of you read [the draft version](https://www.lesswrong.com/s/HMs2yT9D6LjYR5jQT) I posted on LessWrong as I wrote it. If you did, thank you, because your comments and feedback were critical in making the final version what it is (some of you even made it into the acknowledgments!). There are lots of changes since the draft, so if you read it before and thought “ehh, maybe there’s something here, but I don’t buy it”, I highly recommend reading it again.

If you’re not familiar with the book, here’s its thesis:

> 1.  Our knowledge of the truth is fundamentally uncertain because of epistemic circularity caused by the Problem of the Criterion.
>     
> 2.  We manage fundamental uncertainty by making pragmatic assumptions that lead us to believe in the truth of claims that help us achieve our goals.
>     
> 3.  Consequently, the truth that can be known is not independent of us, but rather dependent on that for which we care.
>     
> 4.  That truth is fundamentally uncertain and grounded in care has far-reaching implications for many of the world’s hardest-to-solve problems.
>     

The book argues for and develops these points in greater detail. It’s written for a general STEM audience, but I think it will be mostly of interest to rationalist and rationalist-adjacent readers. I especially hope folks working on AI and AI safety read the book, since I wrote it to document all the things I had to learn about epistemology to pursue my own previous AI safety research program.

I’ll have more book-related news in the next few weeks, as I’m planning an essay contest, and I’m actively working on getting the print and ebook versions together, with an audiobook to follow. For now, you can read it either [in your browser](https://fundamentaluncertainty.com/book/) or as [markdown](https://fundamentaluncertainty.com/book/fundamental_uncertainty.md), and if you’d like other formats, please let me know in the comments.
# Announcing ILIAD 2026

We are pleased to announce ILIAD 2026—a 5-day conference bringing together researchers to build scientific foundations for AI alignment. This is the 3rd iteration of ILIAD, which was [first held in the summer of 2024](https://www.lesswrong.com/posts/r7nBaKy5Ry3JWhnJT/announcing-iliad-theoretical-ai-alignment-conference).   

**\*\*\***[**Apply to attend by July 1!**](https://docs.google.com/forms/d/e/1FAIpQLSdkO7ZLa253bVFt5tTwEY6me6_7OJERg_OB7UcVRAgJsG2HqQ/viewform?usp=header)**\*\*\***
--------------------------------------------------------------------------------------------------------------------------------------------------------------------

*   **When**: Aug 3-7, 2026
*   **Where**: @[Lighthaven](https://www.lighthaven.space/) (Berkeley, US)
*   **What**: Unconference with participant-led programming, 150-200 attendees. 
*   **Who**: Currently confirmed attendees include Richard Ngo, Abram Demski, and Adam Goldstein
*   **Costs**: Tickets are free. Financial support for travel and accommodations is available on a needs basis. 

See our website [here](https://www.iliadconference.com/). For any questions, email [conferences@iliad.ac](mailto:conferences@iliad.ac)

  

About ILIAD
-----------

ILIAD is a 100+ person conference about alignment with a mathematical focus. ILIAD will feature an *unconference format—*meaning that participants can propose and lead their own sessions. We believe that this is the best way to release the latent creative energies in everyone attending.

The ILIAD conferences are the premier gathering for all things theory and alignment. Topics include: [Singular Learning Theory](https://www.lesswrong.com/w/singular-learning-theory), [Agent Foundations](https://www.lesswrong.com/posts/3vDb6EzBpaHqDqQif/some-summaries-of-agent-foundations-work-1), [Causal Incentives](https://causalincentives.com/), [Computational Mechanics](https://www.lesswrong.com/posts/gTZ2SxesbHckJ3CkF/transformers-represent-belief-state-geometry-in-their) and Safety-by-Debate, Scalable Oversight and more.

To get a better sense of what might happen, you can view the schedule for ILIAD 2024 [here](https://iliadschedule.vercel.app/ILIAD?view=grid).

Financial Support
-----------------

Financial support for accommodation & travel are available on a needs basis. We may not be able to offer support to everyone who requests it.
# Scott Alexander gentrified my meetup

On 10 April 2026, I organized an ACX/LW/rationality meetup.

Initially, I intended for it to be just our regular meetup, and “regular” for Zagreb is between 3 and 10 people having a couple of beers.

Skyler of [Rationality Meetups](https://www.rationalitymeetups.org/) contacted me saying that Scott would be in Zagreb in April, and we coordinated so that the meetup could accommodate Scott’s schedule.

At first, I thought, man, this is serious now. I’ll have to actually reserve a table.

Once the “Going” RSVPs started coming in, I was like: shit, this is even more serious now. I’m going to have to reserve **two** tables.

Soon after, when I first started receiving emails from people announcing that they would be travelling from the Netherlands, Germany, Sweden, Estonia, Slovakia, Slovenia, and probably other countries which I forgot, I started to panic a little bit.

Luckily, I managed to somehow reserve a big event space from the company where I work. Not sure what would have happened if we stuck to the original venue, but I imagine it would probably be a massive fail. At one point, I think I counted 54 individuals, but could have miscounted.

People mingled, ate food, and had a nice time. Scott gave a nice Q&A session, answering mostly AI-safety-related questions. I’m sorry I couldn’t get the microphone working, but it was a last-minute event space so… Count yourselves lucky to have had somewhere to sit!

Thank you, Scott, for giving sensible directions and imposing some structure on the chaos.

Special thanks to Dorian for helping with logistics and everything in general, and to everyone who helped clean up afterwards!

See some of you on the next meetup.

PS: here’s the [invite link to the LessWrong Croatia group](https://t.me/+8o2y2zhYI6kwYzc0). If you’re interested in attending future meetups, this is the easiest way to be informed about them. Otherwise, feel free to [email me](mailto:dt@d11r.eu).

![Meetup photo](https://blog.d11r.eu/meetup-2026/photo.jpg)
# Searchable explorer of EA Forum & LessWrong posts with explicit cruxes or "change my mind" content

*Status: early stage, let us know if it's being done better elsewhere*

[**Forum Post Cruxes and Pivotal Questions Explorer**](https://uj-prioritization-prototype.netlify.app/cruxes/)

EA Forum and LessWrong posts sometimes contain explicit cruxes, "what would change my mind" statements, "hinge beliefs", and research-blocking open questions.

As part of [The Unjournal's Pivotal Questions project](https://forum.effectivealtruism.org/s/kazWBBYXm2Rvya3y2) we're trying to identify decision-relevant open questions that may connect to rigorous/academic research. We've done some work mapping forum content to candidate questions for evaluation and synthesis.

*The link*:  a filterable, searchable table of  \[so far\] ~39 posts from EA Forum and LessWrong (April 2024 – April 2026), each tagged with the crux or change condition, why it looks tractable for Unjournal-style work, and a candidate "Pivotal Question mapping" where relevant. You can filter by signal type (explicit crux, hinge belief, CMM, research demand…), cause area, forum, or Unjournal relevance, and share filtered URLs.

Thought others might find this interesting and maybe useful. It could feed directly into Unjournal work as well as be something to consider in forming career and research plans.

*Caveats*: this was roughly 1-2 hours of work (AI-assisted curation + some light engineering). Coverage is patchy, tilted toward AI safety, AI welfare, and cause prioritization, and some entries will need correction. If people find it useful we'd like to maintain and extend it, including adding EA Forum content more systematically.

Feedback welcome, especiallyt via the Hypothes.is sidebar on the page, or the "Suggest entry" button. Happy to hear whether the framing and coverage make sense, or if there are posts you'd obviously include that we missed. Or if this should go in a different directi