Harley-ml/lesswrong huggingface dataset:
# Are we "being poisoned"?

I would like to revisit some of the concepts Scott explored in his 2020 article "[For, Then Against, High-Saturated-Fat Diets](https://slatestarcodex.com/2020/03/10/for-then-against-high-saturated-fat-diets/)". I'm hoping someone will have some novel/updated insights or new research to share concerning the impacts of the Western diet on health in 2025. 

I'm about to turn 32, and I sense that I'm moving toward becoming a prime candidate for one of the gastrointestinal cancers that (among many others) are on the rise globally among people under the age of 50. Demographically, I am disproportionately at risk due to genetic, socioeconomic, and other factors. And, guys, I don't want to die any time soon. 

This trend in cancer rates among young adults has seen the most dramatic uptick specifically in rich, industrialized Western nations, and the consensus seems to be that diet and lifestyle choices are a factor, albiet one of many, not all of which are clear right now. 

Yesterday, I saw a video on social media of a woman discussing how, since moving to the US, she has started mysteriously gaining weight even though she's eating the same amount of food as before she moved here. 

The comments were full of people with similar stories. Some of them claim that we are all "being poisoned" via our own food systems and the corporations that run them, as evidenced by the prevalence of growth hormones in meat, the presence of microplastics in everything, the preservatives and toxic chemicals in common food staples, the chemical pesticides that are present in the environment, the fields, the water supply, and by extension even in natural foods and animal products. 

The woman's experience is consistent with my own experience of Western vs non-Western diets and their respective impacts on health. Without veering too far off into the anecdotal wilderness, I lived in Turkey for most of 2012 and was a total glutton while there. I ate my weight in delicious Turkish food and sweets every day and exercised very little. I also smoked cigarettes (but have since quit). In spite of my gluttony, I lost so much weight that the US customs agent who stamped my passport on my way back into the country pointed it out to me and congratulated me on it. 

I want to know what, specifically, is driving this problem (doesn't everyone) and what an ordinary person can do to minimize the risk of serious illness. What are we all being exposed to that's wreaking such havoc on our health? How significant is the microbiome theory? Should I try cutting gluten? Start raising my own chickens and growing my own produce on the kitchen window sill? Switch to a vegan diet? Leave the country altogether? Do you know of any promising updated research in this direction that you'd like to share, or have a story that could provide some valuable personal insight? Know of any good blogs covering this?
# Jaan Tallinn's 2024 Philanthropy Overview

to follow up my [philanthropic pledge](https://www.lesswrong.com/posts/ZMRCpfGD4iXs5Mi5Q/jaan-tallinn-s-philanthropic-pledge) from 2020, i've updated my [philanthropy page](https://jaan.info/philanthropy/) with the 2024 results.

in 2024 my donations funded [$51M worth of endpoint grants](https://jaan.info/philanthropy/donations.html#2024) (plus $2.0M in admin overhead and philanthropic software development). this comfortably exceeded my 2024 commitment of $42M (20k times $2100.00 — the minimum price of ETH in 2024).

this also concludes my [5-year donation pledge](https://jaan.info/philanthropy/#donation-pledge-for-2020-2024), but of course my philanthropy continues: eg, i’ve already made over $4M in endpoint grants in the first quarter of 2025 (*not* including 2024 grants that were slow to disburse), as well as pledged at least $10M to the [2025 SFF grant round](https://survivalandflourishing.fund/2025/application).
# What AI safety plans are there?

I'm trying to make a list of every written plan for how to make AGI go well. I am interested in hearing about any plans that I missed, or any lists of plans that have already been compiled.

Here are the plans I've found so far, in no particular order:

- Google https://deepmind.google/discover/blog/taking-a-responsible-path-to-agi/
- OpenAI https://openai.com/index/updating-our-preparedness-framework/
- Anthropic https://www.anthropic.com/news/announcing-our-updated-responsible-scaling-policy
- https://www.lesswrong.com/posts/8vgi3fBWPFDLBBcAx/planning-for-extreme-ai-risks
- https://www.lesswrong.com/posts/bb5Tnjdrptu89rcyY/what-s-the-short-timeline-plan
- https://sleepinyourhat.github.io/checklist/
- https://www.narrowpath.co/
- https://situational-awareness.ai/
- https://gradual-disempowerment.ai/
- https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ or https://intelligence.org/2024/05/29/miri-2024-communications-strategy/ (note: these both touch on MIRI's plan but they are not a complete breakdown of a plan)
- https://aligned.substack.com/p/alignment-mvp
- https://www.lesswrong.com/posts/vZzg8NS7wBtqcwhoJ/nearcast-based-deployment-problem-analysis
- https://drive.google.com/file/d/1LbQgcz_aehcnhFMupCtXqoR-BV5dX5Rv/view
- https://pauseai.info/proposal

It's somewhat arbitrary as to how much detail something needs to qualify as "a plan". I guess I would define a plan as "if we do these things and they all work, then we will be okay." By this definition Google is the only company with an actual plan (as far as I know) but I included the closest thing I could find from OpenAI + Anthropic.
# My Favorite Productivity Blog Posts

I’ve read at least a few hundred blog posts, maybe upwards of a thousand. [Agreeing](https://www.gleech.org/essays) with Gavin Leech, I believe I’ve gained from essays more than any other medium. I’m an intellectually curious college student with a strong interest in both the practical and philosophical. This blog post provides a snippet into the most valuable blog posts I’ve read on the topic of productivity. I encourage you to scan what I’ve gotten valuable out of the blog posts and read deeper if interesting.

*   [**On systems - Living a life of zero willpower**](https://www.neelnanda.io/blog/mini-blog-post-19-on-systems-living-a-life-of-zero-willpower) by Neel Nanda. This post inspired me to sign up for Focusmate. I can confidently say that Focusmate (1) has increased my productivity by >20% in a lasting way and (2) houses more than half of my social life. I’ve met many of my closest friends through the app. The other advice in Neel’s blog post is also great. (If you use Focusmate, feel welcome to favorite [me](https://app.focusmate.com/user/parconley) and lock in a coworking session!)
*   [**How to write good prompts**](https://andymatuschak.org/prompts/) by Andy Matuschak. Empirically, I have this blog post to thank for my >2 year long Anki habit. It was a three-hour read, but it was well worth it (being a full-time student spending 1-2h/day using Anki).
*   [**How to learn at college**](https://web.archive.org/web/20241003162607/https://brennancolberg.com/writing/how-to-learn-at-college) by Brennan Colberg. Led me to skip 70% of my classes last college semester.
*   [**Honesty about reading**](https://www.cold-takes.com/honesty-about-reading/)  by Holden Karnofsky. I skim a lot. I mostly don’t finish books. This blog post by Holden gave me even more permission to continue skimming a lot.
*   [**Internet Search Tips**](https://gwern.net/search) by Gwern. Two things I got from this: (1) I have [single-press hotkeys](https://github.com/parconley/AHK/blob/main/scripts/hotkeys.ahk) for LLMs, and (2) I now use `site:` in search queries. I use these all the time. Having Claude/ChatGPT one click away is life-changing, highly recommend.
*   [**My tech stack**](https://andrewconner.com/tech-stack/) by Andrew Conner. It was either this blog post, another blog post, or a conversation with Andrew that pushed me over the edge into trying out [raindrop.io](http://raindrop.io/). I love raindrop, and use it daily. It’s basically a bookmarks bar that actually works.
# Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

> Recent breakthroughs in reasoning-focused large language models (LLMs) like OpenAI-o1, DeepSeek-R1, and Kimi-1.5 have largely relied on *Reinforcement Learning with Verifiable Rewards* (RLVR), which replaces human annotations with automated rewards (e.g., verified math solutions or passing code tests) to scale self-improvement. While RLVR enhances reasoning behaviors such as self-reflection and iterative refinement, we challenge a core assumption:
> 
> ***Does RLVR actually expand LLMs' reasoning capabilities, or does it merely optimize existing ones?***
> 
> By evaluating models via *pass@k*, where success requires just one correct solution among *k* attempts, we uncover that RL-trained models excel at low *k* (e.g., pass@1) but are consistently *outperformed by base models* at high *k* (e.g., pass@256). This demonstrates that RLVR *narrows the model's exploration*, favoring known high-reward paths instead of discovering new reasoning strategies. Crucially, all correct solutions from RL-trained models already exist in the base model's distribution, proving RLVR enhances *sampling efficiency*, not reasoning capacity, while inadvertently shrinking the solution space.
# Zstd Window Size

At work we've recently been using [zstd](https://en.wikipedia.org/wiki/Zstd) as a better-compressing alternative to [gzip](https://en.wikipedia.org/wiki/Gzip), and overall I've been pretty happy with it. A minor documentation gripe, though, is that the behavior around multithreaded compression is a bit unclear. I understand it's chunking the work and sending chunks to different threads to parallelize the compression process, and this means that I should expect to see better use of threads on larger files because there are more chunks to spread around, but what is the relationship?

When I look in `man zstd` I see that you can set `-B<num>` to specify the size of the chunks, and it's documented as "`generally 4 * windowSize`". Except the documentation doesn't say how `windowSize` is set.

From a bit of poking at the source, it looks to me like the way this works is that `windowSize` is `2**windowLog`, and `windowLog` depends on your compression level. If I know I'm doing `zstd -15`, though, how does `compressionLevel=15` translate into a value for `windowLog`? There's a table in [`lib/compress/clevels.h`](https://github.com/facebook/zstd/blob/dev/lib/compress/clevels.h) which covers inputs >256KB:

| Level | windowLog | chainLog | hashLog | searchLog | minMatch | targetLength | strategy |
| --- | --- | --- | --- | --- | --- | --- | --- |
| <1  | 19  | 12  | 13  | 1   | 6   | 1   | fast |
| 1   | 19  | 13  | 14  | 1   | 7   | 0   | fast |
| 2   | 20  | 15  | 16  | 1   | 6   | 0   | fast |
| 3   | 21  | 16  | 17  | 1   | 5   | 0   | dfast |
| 4   | 21  | 18  | 18  | 1   | 5   | 0   | dfast |
| 5   | 21  | 18  | 19  | 3   | 5   | 2   | greedy |
| 6   | 21  | 18  | 19  | 3   | 5   | 4   | lazy |
| 7   | 21  | 19  | 20  | 4   | 5   | 8   | lazy |
| 8   | 21  | 19  | 20  | 4   | 5   | 16  | lazy2 |
| 9   | 22  | 20  | 21  | 4   | 5   | 16  | lazy2 |
| 10  | 22  | 21  | 22  | 5   | 5   | 16  | lazy2 |
| 11  | 22  | 21  | 22  | 6   | 5   | 16  | lazy2 |
| 12  | 22  | 22  | 23  | 6   | 5   | 32  | lazy2 |
| 13  | 22  | 22  | 22  | 4   | 5   | 32  | btlazy2 |
| 14  | 22  | 22  | 23  | 5   | 5   | 32  | btlazy2 |
| 15  | 22  | 23  | 23  | 6   | 5   | 32  | btlazy2 |
| 16  | 22  | 22  | 22  | 5   | 5   | 48  | btopt |
| 17  | 23  | 23  | 22  | 5   | 4   | 64  | btopt |
| 18  | 23  | 23  | 22  | 6   | 3   | 64  | btultra |
| 19  | 23  | 24  | 22  | 7   | 3   | 256 | btultra2 |
| 20  | 25  | 25  | 23  | 7   | 3   | 256 | btultra2 |
| 21  | 26  | 26  | 24  | 7   | 3   | 512 | btultra2 |
| 22  | 27  | 27  | 25  | 9   | 3   | 999 | btultra2 |

See the source if you're interested in other sizes.

So it looks like `windowSize` is:

*   `≤1`: 524k
*   `2`: 1M
*   `3-8` (default): 2M
*   `9-16`: 4M
*   `17-19`: 8M
*   `20`: 32M
*   `21`: 64M
*   `22`: 128M

Probably best not to rely on any of this, but it's good to know what `zstd -<level>` is doing by default!

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0qoBikYJ7xPoFBFciU8nHhCi9rH2yPqx9UvDXyCMdT8DJuj69hja3YNee45NJcsjJl), [mastodon](https://mastodon.mit.edu/@jefftk/114399107320756518), [bluesky](https://bsky.app/profile/jefftk.com/post/3lnnh54xlok2s)*
# Open Source LLM Pokémon Scaffold

[This is a cleaned-up, open-source version](https://github.com/cicero225/llm_pokemon_scaffold) of the LLM Pokémon Scaffold described in [Research Notes: Running Claude 3.7, Gemini 2.5 Pro, and o3 on Pokémon Red](https://www.lesswrong.com/posts/8aPyKyRrMAQatFSnG/untitled-draft-x7cc). (forked from David Hershey of Anthropic's scaffold [here](https://github.com/davidhershey/ClaudePlaysPokemonStarter/tree/main), all development on top of that was done by my friend, not me)

Since that post, a number of changes have been made to the scaffold. The major ones are:

1.  Instead of using colored squares on the game screenshots, information is printed as text, ex. "Impassable", "Explored", "Check Here"
    1.  Models are seemingly helped by putting relevant information blatantly in the spot they need to see it, rather than indirectly via a legend or instructions or whatever
    2.  ![For some reason it helps if you just write "CHECK HERE" on every unexplored tile.](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/f89cd23c3ec7e93b0d47f01762e90a79cc9d58d278657afd.png)
        
        For some reason it helps if you write "CHECK HERE" on every unexplored tile.
        
2.  Automatically-updating ASCII collision map given to LLM
    1.  Generated by code
    2.  Uses numbers indicating how many moves away each tile is[^84lzgicwgga]
    3.  ![Behold, Pewter City.](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/97fac78acd3fa6d94cc0233a58708cd28fec533a7c370df7.png)
        
        Behold, Pewter City.
        
3.  Improved prompts for "Critique Claude"/"Guide Gemini"/"Oversight o3"
    1.  Prompt 1: Given a bunch of facts about the current game state and instructions on what is trustworthy and what's not, make a summary
        1.  this is an attempt to get the model to grasp reality better, telling it what sources of information it should basically always trust (data from game's RAM), mostly trust (its own knowledge of the game from training), not trust (map labels it made itself), and mostly distrust (its own vision)
    2.  Prompt 2: Look at output from prompt 1 and try to remove inconsistencies
    3.  Prompt 3: OK now talk to the model you're critiquing
4.  Models encouraged to use a "mark_checkpoint" tool to maintain a running list of major checkpoints (Left House, Beat Misty, died to Brock, etc.)
5.  "detailed_navigation" tool which, if called, calls an alternate model that basically rolls around trying to explore + DFS but isn't told what the goal is (but is told to talk to NPCs and exit maps)
6.  Autopathing tool that can travel to known coordinates on the map

All of this helps somewhat but doesn't make LLMs amazing at Pokémon by any means.

[^84lzgicwgga]: This has been updated to give the map in the form of pure coordinates instead, which works better, perhaps for tokenizer reasons.
# MiCARwave

Last year at NEFFA we mostly bought food. We brought a range of cold options, but they weren't very tasty and the kids weren't very excited about them. This time we've mostly eaten food we brought. Why? The miCARwave!

[![](https://www.jefftk.com/micarwave-at-neffa.jpg)](https://www.jefftk.com/micarwave-at-neffa-big.jpg)

We have an old (light!) microwave \[1\] and I recently got a [portable power station](https://www.amazon.com/Anker-Portable-Generator-UltraFast-Optional/dp/B0C5C89QKZ/) (battery). We combine this with a cooler (pasta, rice, cheese, sauce, [freezer pucks](https://www.jefftk.com/p/frozen-lunch-pucks), etc) which gives a lot of options for tasty warm food.

My deal with the kids was that if they wanted to eat purchased food they could, but they would need to pay for half the cost. And if they wanted to buy desserts they were on their own. This seemed to be a pretty good balance: at Lily bought and enjoyed some fries, but other than that the kids were pretty happy eating brought lunch and dinner.

(I previously wrote some about this in my [2025 tour retrospective](https://www.jefftk.com/p/kingfisher-tour-february-2025), and an earlier iteration in my [2024 one](https://www.jefftk.com/p/tour-retrospective-february-2024).)

  
\[1\] Ours is used, but it looks like similar ones are [still made](https://www.amazon.com/Counter-Rotary-Microwave-WCM660W-Westinghouse/dp/B00BGTNY8O?th=1).

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid02fAasQHoWnYXPYBwmjiBihXPs2Gw8tQkLgqHmAKb1Y8EthHv5haVaH6oN7reQZ2PNl), [mastodon](https://mastodon.mit.edu/@jefftk/114407566992205688), [bluesky](https://bsky.app/profile/jefftk.com/post/3lnr7dxzrxs24)*
# After Internet Dependency

ChatGPT recently introduced improved cross-chat memory, and I've found myself using it a lot more sense then. Everything from helping write components with unfamiliar JavaScript libraries, to counseling me through a breakup. So much so that when I face a new problem, my mind turns not to breaking it down myself, but how to structure it as a prompt for o3 to help me solve. As useful as these models are, I think I'm experiencing first hand a new type of dependency. 

This parallels a similar problem I had before LLMs got large enough to be useful. I've always been more of a word-cell than a shape-rotator, so when navigating unfamiliar areas, it's easier to pull out Google Maps than actually understand where things are, and how to get there. So ever since I learned to drive, I would follow Google's unwavering blue line to my destination. And if I ever faced a roadblock, it would seamlessly reroute me along the next fastest path. However like ChatGPT, I found myself completely helpless without this tool. To the point where if my phone died, I would be forced to charge it before I could find my way home.

I'm now finding that instead of just having trouble navigating from point A to point B, I struggle to navigate *life* without the advice of my pocket oracle. I've thought about embracing it, and purchasing Meta glasses to fill in the gaps where ChatGPT doesn't fit, but I've seen where that leads. Instead I'm trying an experiment: doing things the old fashioned way. Instead of generating most of my code, I'm treating AI like a knowledgeable colleague who I don't want to bother too frequently. Rather than using o3 like a cheatcode to my love life, I'm leaning more on my friends.

o3 has no problem one-shotting a boiler-plate frontend, but repeatedly failed to help me code a particle editor using a newer library than existed in it's training data. Also, sitting in my basement staring at ChatGPT.com is not (yet) the best way to maintain friendships and find a new boyfriend.
# The case for multi-decade AI timelines [Linkpost]

So this post is an argument that multi-decade timelines are reasonable, and the key cruxes that Ege Erdil has with most AI safety people who believe in short timelines are due to the following set of beliefs:

1.  Ege Erdil doesn't believe that trends exist that require AI to automate everything in only 2-3 years.
2.  Ege Erdil doesn't believe that the software-only singularity is likely to happen, and this is perhaps the most important crux he has with AI people like [@Daniel Kokotajlo](https://www.lesswrong.com/users/daniel-kokotajlo?mention=user) who believe that a software-only singularity is likely.
3.  Ege Erdil expects Moravec's paradox to bite hard once AI agents are made in a big way.

This is a pretty important crux, because if this is true, a lot more serial research agendas like Infra-Bayes research, Natural Abstractions work, and human intelligence augmentation can work, and also it means that political modeling (like is the US economy going to be stable long-term) matter a great deal more than is recognized in the LW/EA community.

Here's a quote from the article:

> *   I don’t see the trends that one would extrapolate in order to arrive at very short timelines on the order of a few years. The obvious trend extrapolations for AI’s economic impact give timelines to full remote work automation of around a decade, and I expect these trends to slow down by default.
> *   I don’t buy the software-only singularity as a plausible mechanism for how existing rates of growth in AI’s real-world impact could suddenly and dramatically accelerate by an order of magnitude, mostly because I put much more weight on bottlenecks coming from experimental compute and real-world data. This kind of speedup is essential to [popular accounts](https://www.forethought.org/research/preparing-for-the-intelligence-explosion) of why we should expect timelines much shorter than 10 years to remote work automation.
> *   I think intuitions for how fast AI systems would be able to think and how many of them we would be able to deploy that come from narrow writing, coding, or reasoning tasks are very misguided due to [Moravec’s paradox](https://epoch.ai/gradient-updates/movarec-s-paradox). In practice, I expect AI systems to become slower and more expensive as we ask them to perform agentic, multimodal, and long-context tasks. This has already been happening with the rise of AI agents, and I expect this trend to continue in the future.
# Questions for old LW members: how have discussions about AI changed compared to 10+ years ago?

I wasn't on LessWrong 10 years ago, during the pre-ChatGPT era, so I'd like to hear from people who were here back in the day. I want to know how AI-related discussions have changed and how people's opinions have changed.

1.  Are you leaning more towards "alignment is easy" or towards "alignment is hard", compared to many years ago? Is the same true for the community as a whole?
2.  Which ideas and concepts ("pivotal act", CEV, etc.) that were discussed years ago do you think are still relevant, and which ones do you think are obsolete?  
    Is there some topic that was popular 10+ years ago that barely anybody talks about these days?
3.  Are you leaning more towards "creating AGI is easy" or towards "creating AGI is hard", compared to many years ago? Is the same true for the community as a whole?
4.  What are you most surprised about? To put it another way, is there something about modern AI that you could not have predicted in advance in 2015?  
    I am surprised that LLMs can write code yet cannot reliably count the number of words in a short text. Recently, I asked Gemini 2.5 Pro to write a text with precisely 269 words (and even specified that spaces and punctuation don't count as words), and it gave me a text with 401 words. Of course, [there are lots of other examples where LLMs fail in surprising ways](https://www.lesswrong.com/posts/sgpCuokhMb8JmkoSn/untitled-draft-7shu), but this is probably the most salient example for me.  
    If in 2015 I was trying to predict what AI will look like in 2025, I **definitely** would not be able to predict that AI won't be able to count to a few hundred but will be able to write Python code.
# “A victory for the natural order”

> In what many are calling "a victory for the natural order," the final remaining vials of the controversial CHRONO-X anti-aging treatment were ceremoniously destroyed yesterday at Massachusetts General Hospital in Boston.
> 
> "Today marks humanity's return to wisdom," declared Giorgos Hickel, spokesperson for the Foundation Against Increasing Lifespan (FAIL). "Death isn't just natural - it's necessary. The prospect of immortality was never a gift but a curse disguised as progress. We've finally chosen meaningful brevity over longer, purposeless existence."

Keep reading at the link.

Originally posted on April 1st.
# What are the best standardised, repeatable bets?

Suppose that I have some budget set aside for philanthropic funding, say $1,000, but I think there are a big returns to scale, so that it would be >1,000x better if I had $1,000,000.[^e6fp8yyfbuf]

What are my best options for some bets I can make to get some chance of turning that $1,000 into $1,000,000?

I imagine the bets are generally better the more positive their EV is (which in practice means the less negative the EV is), the easier they are to find (like if they're standardised), and the better their tax treatment is, especially if you repeat them multiple times (maybe they can be done by DAFs).

[^e6fp8yyfbuf]: NB: I don't believe this
# Bandwidth Rules Everything Around Me: Oliver Habryka on OpenPhil and GoodVentures

In [this episode](https://youtu.be/uD37AKRx2fg) of our podcast, Timothy Telleen-Lawton and I talk to Oliver Habryka of Lightcone Infrastructure about his thoughts on the Open Philanthropy Project, which he believes has become stifled by the PR demands of its primary funder, Good Ventures.

Oliver’s main claim is that around mid 2023 or early 2024, Good Ventures founder Dustin Moskovitz became more concerned about his reputation, and this put a straight jacket over what Open Phil could fund. Moreover it was not enough for a project to be good and pose low reputational risk; it had to be _obviously_ low reputational risk, because OP employees didn’t have enough communication with Good Ventures to pitch exceptions.  According to Habryka.

That’s a big caveat. This podcast is pretty one sided, which none of us are happy about (Habryka included). We of course invited OpenPhil to send a representative to record their own episode, but they declined (they did send a written response to this episode, which is linked below and read at end of the episode). If anyone out there wants to asynchronously argue with Habryka on a separate episode, we’d love to hear from you. 

Transcript available [here](https://docs.google.com/document/d/1VcLFI1MYv7WZbIUVj1eJ7xGBPXxCBZ_D/edit).

Links from the episode:

[An Update From Good Ventures](https://forum.effectivealtruism.org/posts/foQPogaBeNKdocYvF/linkpost-an-update-from-good-ventures) (note: Dustin has deleted his account and his comments are listed as anonymous, but are not the only anonymous)

[CEA announcing the sale of Wytham Abbey](https://forum.effectivealtruism.org/posts/yggjKEeehsnmMYnZd/announcement-on-the-future-of-wytham-abbey)

[OpenPhli career page](https://www.openphilanthropy.org/careers/)

[Job reporting to Amy WL](https://web.archive.org/web/20250328184228/https://jobs.ashbyhq.com/centreforeffectivealtruism/685ec578-d411-46bc-9410-d9348014205b)

[Zach’s “this is false”](https://forum.effectivealtruism.org/posts/AFMRWvCCgRyys9JXP/has-your-organisation-lost-funding-due-to-the-good-ventures?commentId=uq78YkbedxpmbW6aA)

[Luke Muelhauser on GV not funding right of center work](https://forum.effectivealtruism.org/posts/hCQwysXaAPdQDXygu/lukeprog-s-quick-takes?commentId=NSFmh8zGeQGTmj9Nk)

[Will MacAskill on decentralization and EA](https://forum.effectivealtruism.org/posts/DdSszj5NXk45MhQoq/decision-making-and-decentralisation-in-ea)

[Alexander Berger regrets the Wytham Abbey grant](https://www.openphilanthropy.org/research/our-progress-in-2023-and-plans-for-2024/)

[Single Chan-Zuckerberg employee demanding resignation](https://www.businessinsider.com/chan-zuckerberg-initiative-employee-demands-zuckerberg-resign-report-2020-6) over failure to moderate Trump posts on Facebook

[Letter from 70+ CZ employees](https://www.vox.com/recode/2020/6/17/21294396/mark-zuckerberg-priscilla-chan-philanthropy-education-george-floyd-petition-czi) asking for more DEI within Chan Zuckerberg Initiative.

[OpenPhil’s response](https://docs.google.com/document/d/1EYCMHa6_7Mudb4s1MDvppGMY5BmHEVvryGw9cX_dlQ8/edit?tab=t.0)
# Investigating task-specific prompts and sparse autoencoders for activation monitoring

We’d like to share the most recent paper from the OpenAI Interpretability Team. The question of how activation probes with sparse autoencoders compare with baseline activation probes has gotten a [lot](https://transformer-circuits.pub/2024/features-as-classifiers/index.html) [of](https://arxiv.org/pdf/2502.11367) [attention](https://arxiv.org/abs/2502.16681) [recently](https://www.alignmentforum.org/posts/4uXCAJNuPKtKBsi28/sae-progress-update-2-draft). With this work, we took a step back and asked: how do both these activation probing methods compare with the common baseline of just asking a language model in natural language? And how do they combine? 

We find that a combination of “just asking the model” and activation probing, similar to that introduced in the [representation engineering paper](https://arxiv.org/abs/2310.01405), improves on either prompting or activation probing by itself. This method is highly data efficient and generalizes well, but requires additional inference-time compute for the monitoring task-specific prompt. We also find SAE-based probing methods outperform raw activation probing in most of the monitoring tasks we studied.

Contributors: Henk Tillman, Dan Mossing
# Don't accuse your interlocutor of being insufficiently truth-seeking

I argue that you shouldn't accuse your interlocutor of being insufficiently truth-seeking. This doesn't mean you can't internally model their level of truth-seeking and use that for your own decision-making. It just means you shouldn't come out and say "I think you are being insufficiently truth-seeking".

What you should say instead
---------------------------

Before I explain my reasoning, I'll start with what you should say instead:

### "You're wrong"

People are wrong a lot. If you think they are wrong just say so. You should have a strong default for going with this option.

### "You're being intentional misleading"

For when you *basically* thinking they are lying but maybe technically aren't by some definitions of "lying".

What about if they are being *unintentionally* misleading? That's usually just being wrong, you should probably just say they are being wrong. But if you really think the distinction is important, you can say they are being unintentionally misleading.

### "You're lying"

For when they are lying.

You can also add your own flair to any of these options to spice things up a bit.

Why you shouldn't accuse people of being insufficient truth-seeking
-------------------------------------------------------------------

### Clarity

It's not clear what you are even accusing them of. "Insufficient truth-seeking" could arguably be any of the options I mentioned above. Just be specific. If you *really* think what you're saying is so important and nuanced and you just need to incorporate some deep insight about truth, use the "add your own flair" option to sneak that stuff in.

### Achieving your purpose in the discussion

The most common purposes you might have for engaging in the discussion and why invoking "truth-seeking" doesn't help them:

**You want to discuss the object-level issue**

You just fucked yourself because the discussion is immediately going to center on whether they actually are insufficiently truth-seeking and whether that accusation was justified. You're going to have to gather The Fellowship, take your argument to Mordor, and throw it into the fire of NEVER GO META before you're ever going to be able to discussion the object level again.

**You want to discuss your interlocutor's misconduct**

You again fucked yourself because:

1.  It's not clear what misconduct you are accusing them of.
2.  Because of the ambiguity they are going to try to make it seem like you're accusing them of more than what you intended, and therefore actually it's *you* who isn't being truth-seeking, and you're even accusing them of that *in bad faith*!
3.  Because your statement is about "truth-seeking" instead of the actual misconduct, observers who agree with your interlocutor on the object level but might be sympathetic to your misconduct allegation are going to find it harder to agree with you on the meta issue. You are muddying the object-meta waters instead of tackling the meta-level issue you want to address head-on.

Conclusion
----------

Don't accuse your interlocutor of being insufficiently truth-seeking. Just say they are wrong instead.
# Prioritizing Work

I recently read a [blog post](https://www.talentstuff.com/blog/on-loyalty-to-your-employer) that concluded with:

> When I'm on my deathbed, I won't look back at my life and wish I had worked harder. I'll look back and wish I spent more time with the people I loved.

Setting aside that some people don't have the [economic breathing room](https://www.jefftk.com/p/the-privilege-of-earning-to-give) to make this kind of tradeoff, what jumps out at me is the implication that you're not working on something important that you'll endorse in retrospect. I don't think the author is envisioning directly valuable work (reducing risk from international conflict, pandemics, or AI-supported totalitarianism; improving humanity's treatment of animals; fighting global poverty) or the undervalued less direct approach of [earning money and donating it](https://80000hours.org/articles/earning-to-give/) to enable others to work on pressing problems.

Definitely spend time with your friends, family, and those you love. Don't work to the exclusion of everything else that matters in your life. But if your tens of thousands of hours at work aren't something you expect to look back on with pride, consider whether there's something else you could be doing professionally that you could feel good about.
# Why does METR score o3 as effective for such a long time duration despite overall poor scores?

*Epistemic status: Question, probably missing something.*

Context
-------

See the preliminary evaluation of o3 and o4-mini here: [https://metr.github.io/autonomy-evals-guide/openai-o3-report/#methodology-overview](https://metr.github.io/autonomy-evals-guide/openai-o3-report/#methodology-overview)

This follows up [important work by METR](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) measuring the maximum human-equivalent lengths of tasks that frontier models can perform successfully, which [I predicted would not hold up](https://www.lesswrong.com/posts/BrHv7wc6hiJEgJzHW/reactions-to-metr-task-length-paper-are-insane) (perhaps a little too stridently). 

I'm also betting on that prediction, please provide me with some liquidity: 

Question
--------

o3 doesn't seem to perform too well according to this chart:

![Performance on subset of RE-Bench chart](https://metr.github.io/autonomy-evals-guide/image/openai_o3_report/rebench_bar.png)

But it gets the best score on this chart:

![Models' 50% Time Horizon chart](https://metr.github.io/autonomy-evals-guide/image/openai_o3_report/models_50_time_horizon.png)

I understand that these are measuring two different things, so there is no logical inconsistency between these two facts, but the disparity does seem striking. Would someone be willing to provide a more detailed explanation of what is going on here? I am not sure whether to update that the task length trend is in fact continuing (or accelerating) or interpret the overall poor performance of o3 as a sign that the trend is about to break down.
# London Rationalish meetup - Lincoln's Inn Fields

This month, assuming the weather is nice, we'll be in Lincoln's Inn Fields. Most likely the North East quadrant. If it's not nice we'll go to a nearby pub.

Our reading list for this month is:

1\. [Cynicism as a Fantasy that Perpetuates Itself Through Mockery](https://naturalhazard.xyz/cynicism_fantasy_mockery)  
2\. [Social Dark Matter](https://www.lesswrong.com/posts/KpMNqA5BiCRozCwM3/social-dark-matter)

We'll be showing up from 2, and start to discuss these around 3. If you have articles you want to suggest for future readings, you can do that at [https://redd.it/v3646u](https://redd.it/v3646u).
# What's up with AI's vision

This week I've read 2 pieces of interesting information:

*   Apparently current models (in particular o3) are really good at GeoGuessr ([Testing AI's GeoGuessr Genius](https://www.astralcodexten.com/p/testing-ais-geoguessr-genius)),
*   Apparently they have  very had time understanding what they see on screenshots from Pokemon Red, even when stuff is clearly marked ([Research Notes: Running Claude 3.7, Gemini 2.5 Pro, and o3 on Pokémon Red](https://www.lesswrong.com/posts/8aPyKyRrMAQatFSnG/untitled-draft-x7cc)).

This is a puzzling to me. I find it much easier to find staircase in Pokemon Red than to figure out where s photo as taken.

Reading about the first example from Scotts post. I guess there are not that many featureless plains in the world. But can it really tell types of grass growing there? But have issues identifying doors / staircases in Pokemon Red? Seems weird.

Do we know what's up with all that?
# SimpleStories: A Better Synthetic Dataset and Tiny Models for Interpretability

The dataset and model suite TinyStories is widely used as a toy setup for mech interp research. Searching for the term on this forum currently yields 23 results. "[A List of 45+ Mech Interp Project Ideas from Apollo Research’s Interpretability Team](https://www.alignmentforum.org/posts/KfkpgXdgRheSRWDy8/a-list-of-45-mech-interp-project-ideas-from-apollo-research)" lists making an improved version of it as a desideratum, because the dataset is "very formulaic, small, and has unusual unicode characters in it".  
  
The improved version, [the SimpleStories dataset and model suite, is now out](https://huggingface.co/datasets/SimpleStories/SimpleStories). We are looking forward to see what the interp community ends up doing with it, and hope that it is better than TinyStories in all interp use cases. For feedback, we keep [a community doc](https://docs.google.com/document/d/1XRb-EDDw-h6c-L6pKltI9A8zJmufE71I2_4wx8d5rMg/edit?usp=sharing).
# What is your favorite podcast?

I'm looking for podcast recommendations from the LessWrong community. If you have a favorite podcast, please share:

1.  **What is the podcast?**
2.  **What specifically do you like about it?**
3.  **What concepts or topics does it teach particularly well?**
4.  **Why do you consider its content trustworthy or reliable?**
5.  **Who would you recommend this podcast to?** (e.g., target audience, specific interests)

**Please keep each top-level answer limited to** ***one*** **podcast recommendation.** This will allow others to vote on individual suggestions effectively.

If you want to recommend multiple podcasts, please create a separate top-level answer for each one.
# PSA: Before May 21 is a good time to sign up for cryonics

Cryonics Institute and Suspended Animation now have an arrangement where Suspended Animation will conduct a field cryopreservation before shipping the body to Cryonics Institute, thus decreasing tissue damage occuring in transit. They are raising their prices accordingly, but offering a discount from the new price for people who sign up by May 21 (and arrange funding within another 3 months). See [https://cryonics.org/members/standby/suspended-animation-inc-standby-transport-services-option/](https://cryonics.org/members/standby/suspended-animation-inc-standby-transport-services-option/) for details.

It is thus an especially good time to sign up cryonics if you intend to contract with Cryonics Institute plus Suspended Animation, and live in the United States. If you don't live in the US, don't intend to contract with CI, or intend to contract with CI but not also with SA, then this deadline doesn't mean anything for you, but, if you want to, you could still take this as impetus to get around to signing up.
# Fake AI lawsuits to drive links

Someone sent me this and I thought it fairly interesting:

TLDR: Fake AI generated law companies are searching for unattributed images on the internet, claiming they own them, and sending emails asking the user to add a link to their website. The aim is to increase their ranking in Google search to drive traffic to their (AI generated) website.

Yet another example of how the world is going to get steady wierder as signals we used to use for somebody putting in effort into something no longer apply.
# Blue light, 'Adrenal ASMR': strange experiences I can't find any literature about

Hi, I've found I have an intense psychosomatic reaction to certain bands of blue lighting. I bought a powerful blue lamp on Amazon simply because I liked the color blue, and found it does something interesting to me when I sit in a dark room with this lamp switched on. Not always, because I build a high tolerance to it if I do it too many times back to back, but it's happened often enough.

It feels like a combination of tingling heat, mild adrenaline rush and a feeling of gentle compression on either side of about L1/T12, as if someone has reached right in and is massaging my adrenal glands. I've measured my heart rate in case I simply imagined the sensation. It was in the 90-100 range when my resting is 70-80. Most search results for 'blue light' are negative, but this is pleasant, not like anxiety. It feels almost like an atypical form of ASMR that involves adrenaline instead of sleepiness.

I understand that this could be purely imaginary, but then why is blue light causing me to imagine something so bizarre and so specific? Any ideas? I have some more extensive logs and notes about it if anyone wants them.
# Nonprofit to retain control of OpenAI

The OpenAI Board has an updated plan for evolving OpenAI’s structure.

* OpenAI was founded as a nonprofit, and is today overseen and controlled by that nonprofit. Going forward, it will continue to be overseen and controlled by that nonprofit. 
* Our for-profit LLC, which has been under the nonprofit since 2019, will transition to a Public Benefit Corporation (PBC)–a purpose-driven company structure that has to consider the interests of both shareholders and the mission.
# It's 'Well, actually...' all the way down

Some people (the “Boubas”) don’t like “chemicals” in their food. But other people (the “Kikis”) are like, “uh, everything is chemicals, what do you even mean?”

The Boubas are using the word “chemical” differently than the Kikis, and the way they’re using it is simultaneously more specific and less precise than the way the Kikis use it. I think most Kikis implicitly *know* this, but their identities are typically tied up in being the kind of person who “knows what ‘chemical’ means”, and… you’ve gotta use that kind of thing whenever you can, I guess?

There is no single privileged universally-correct answer to the question “what does ‘chemical’ mean?”, because the Boubas exist and are using the word differently than Kikis, and in an internally-consistent (though vague) way.

The Kikis are, generally speaking, much more knowledgeable about the details of chemistry. They might hang out around chemists, whose intellectual ancestors invented the precise definition of the word “chemical”. Or they (we) might just be the kind of person to bank on their knowledge of things-like-chemistry. It’s easy to assume that the Kikis must be right that “avoiding chemicals” is silly, since they seem to know so much more.

But the Boubas are pointing to a real thing that Kikis seem weirdly inclined to assume away: The modern world does *intentional large-scale chemical engineering*, and this is *meaningfully different* in its scale and quickly-shifting nature than nearly anything that came before 1661. You can’t necessarily use the same analogical and cultural reasoning you would have used in 1660 to tell you what’s safe and what isn’t, in a world where everything you touch was chemically engineered.

Which is not to say that the Boubas are right to avoid all (Bouba-defined) chemicals in their food! But it is to say that reversed stupidity is not intelligence: You *should* be less confident in your understanding of the long-term consequences of eating Red Dye Number 40 than in your understanding of the long-term consequences of eating non-GMO wheat. Pretending otherwise is silly. At the same time, risks do not inherently outweigh benefits! Red Dye Number 40 is very pretty! I’d guess that food preservatives have saved or enabled millions of lives! I haven’t looked it up; I’m going to live with the risk that I’m wrong, and so are you.
# Chess - "Elo" of random play?

I'm interested in a measure of  chess-playing ability that doesn't depend on human players, and while perfect play would be the ideal reference, as long as chess remains unsolved, the other end of the spectrum, the engine whose algorithm is "list all legal moves and uniformly at random pick one of them," seems the natural choice.

 I read that the formula for Elo rating $E$ is scaled so that, with some assumptions of transitivity of winning odds, $p_{\textrm{victory}} \approx \dfrac{1}{1 + 10^{\Delta E/400}},$  so it's trivial to convert probability to Elo rating, and my question is roughly equivalent to "What is the probability of victory of random play against, say, Stockfish 17?"  If the Elo is close to 0[^p2obaqo0e5], that makes  the probability around $10^{-9}$ (estimating Stockfish 17's Elo to be 3600). Eyeballing the y-intercept of [this plot](https://chess.stackexchange.com/a/23507) of lc0's Elo vs. number of games of self-play, it looks something like 150–300 (lots of uncertainty). Does that sound reasonable?

I understand the probability of victory against the best modern engines is probably too small to accurately measure directly, so you would have to construct weaker/noisier engines and step down in increments of a few hundred Elo. Has anyone done this?

[^p2obaqo0e5]: I don't see an a priori reason to expect it would be.
# Which journalists would you give quotes to? [one journalist per comment, agree vote for trustworthy]

Let’s create a list of which journalists LessWrongers trust, so as to gave a guide if people get contacted.

Agree votes are for more good. 

Upvotes aren’t helpful because i think heavily downvoted comments get hidden (and i expect many journalists to be underwater). If you want to use upvotes, I suggest they are for if someone is a journalist or not.

Please add each journalist as a separate entry. I will delete any entries that include multiple people. If you’d prefer not to add someone yourself, feel free to DM me.

The title question is a proxy for the thing I mean:

*   Are they trustworthy?
*   Do they publish articles that reflect the quotes you give
*   Do they avoid cutting up quotes?
*   Are they truth seeking?