trentmkelly/LessWrong-43k huggingface dataset:
Endometriosis is an incredibly interesting disease
by Abhishaike Mahajan — 2025-06-14T22:14:04.694Z
Link: https://www.owlposting.com/p/endometriosis-is-an-incredibly-interesting

Introduction

There are several diseases that are canonically recognized as ‘interesting’, even by laymen. Whether that is in their mechanism of action, their impact on the patient, or something else entirely. It’s hard to tell exactly what makes a medical condition interesting, it’s a you-know-it-when-you-see-it sort of thing.

One such example is measles. Measles is an unremarkable disease based solely on its clinical progression: fever, malaise, coughing, and a relatively low death rate of 0.2%~. What is astonishing about the disease is its capacity to infect cells of the adaptive immune system (memory B‑ and T-cells). This means that if you do end up surviving measles, you are left with an immune system not dissimilar to one of a just-born infant, entirely naive to polio, diphtheria, pertussis, and every single other infection you received protection against either via vaccines or natural infection. It can take up to 3 years for one's ‘immune memory’ to return, prior to which you are entirely immunocompromised.

There’s a wide range of such diseases, each one their own unique horror. Others include rabies (trans-synaptic transmission), ebola (causes your blood vessels to become porous), tetanus (causes muscle contractions so strong that they can break bones), and so on.

Very few people would instinctively pigeonhole endometriosis as something similarly physiologically interesting, or at least I wouldn’t have. But via a mutual friend, I recently had a chat with Shilpa Pothapragada, a Schmidt Fellow studying at the Wyss Institute at Harvard. She studies better ways to diagnose endometriosis, and, as a result of the fascinating conversation, I now consider the disease one of the strangest conditions I’ve ever heard of.

Honestly, prior to my discussion with Shilpa, I didn’t even know what endometriosis even was, only that it was painful to have and affects women. To judge whether I was simply deeply ignorant, or the disease genuinely didn’t have much mindshare, I took an informal poll amongst a dozen friends outside of the life-sciences. Even amongst cisgender women (!), knowledge of what endometriosis was astonishingly sparse — most people could only say something like ‘that’s a uterus condition, right?’, and a sum total of zero people actually knew what the disease entailed.

So I decided to write this essay in an attempt to fix that knowledge gap amongst the small ...
Field Notes from Shipping Real Code with Claude
by creatorrr — 2025-06-14T16:36:35.362Z
Link: https://diwank.space/field-notes-from-shipping-real-code-with-claude

Vibe Coding Isn’t Just a Vibe

Shimmering Substance - Jackson Pollock

Think of this post as your field guide to a new way of building software.

Let me take you back to when this all started. Andrej Karpathy tweeted about "vibe-coding"—this idea of letting AI write your code while you just vibe. The developer community had a good laugh. It sounded like the ultimate developer fantasy: kick back, sip coffee, let the machines do the work.

Then Anthropic released Sonnet 3.7 and Claude Code, and something unexpected happened. The joke stopped being funny because it started being… possible? Of course, our trusty friend Cursor had been around awhile but this new interface finally felt like true vibe coding.

Vibe-coding is more like conducting an orchestra. You’re not playing every instrument—you’re directing, shaping, guiding. The AI provides the raw musical talent, but without your vision, it’s just noise.

There are three distinct postures you can take when vibe-coding, each suited to different phases in the development cycle:

AI as First-Drafter: Here, AI generates initial implementations while you focus on architecture and design. It’s like having a junior developer who can type at the speed of thought but needs constant guidance. Perfect for boilerplate, CRUD operations, and standard patterns.
AI as Pair-Programmer: This is the sweet spot for most development. You’re actively collaborating, bouncing ideas back and forth. The AI suggests approaches, you refine them. You sketch the outline, AI fills in details. It’s like pair programming with someone who has read every programming book ever written but has never actually shipped code.
AI as Validator: Sometimes you write code and want a sanity check. AI reviews for bugs, suggests improvements, spots patterns you might have missed. Think of it as an incredibly well-read code reviewer who never gets tired or cranky.

The Three Modes of Vibe-Coding: A Practical Framework

After months of experimentation and more than a few production incidents, I’ve settled on three distinct modes of operation. Each has its own rhythm, its own guardrails, and its own use cases.

Mode 1: The Playground

Lighter Fluid

When to use it: Weekend hacks, personal scripts, proof-of-concepts, and those “I wonder if…” moments that make programming fun.

In Playground Mode, you embrace the chaos. Claude writes 80-90% of the code while you provide j...
Training Superior Sparse Autoencoders for Instruct Models
by Haoran Ye — 2025-06-14T16:35:39.092Z

Resource
Link

Paper
https://arxiv.org/abs/2506.07691

Code
https://github.com/Geaming2002/FAST

SAEs
Llama-3.1-8B-Instruct_SAEs🤗,Llama-3.2-3B-Instruct_SAEs🤗,Llama-3.2-1B-Instruct_SAEs🤗,Qwen2.5-7B-Instruct_SAEs🤗,Qwen2.5-3B-Instruct_SAEs🤗,Qwen2.5-1.5B-Instruct_SAEs🤗,Qwen2.5-0.5B-Instruct_SAEs🤗

💡 TL;DR

In this paper, we discover problems in previous SAE training approaches for instruct model :

📚 Suboptimal dataset selection affecting SAE performance.

✂️ Semantic discontinuity caused by block training truncating samples mid-content.

Therefore, we propose Finetuning-aligned Sequential Training (FAST)💪, a novel training method specifically tailored for instruct models. The results demonstrate:

Token Reconstruction Performance 📉: FAST shows token better reconstruction performance. On Qwen2.5-7B-Instruct, FAST achieves a mean squared error of 0.6468, significantly outperforming baseline methods with errors of 5.1985 and 1.5096.

Feature Interpretability 🎯: FAST yields a higher proportion of high-quality features. For Llama3.2-3B-Instruct, 21.1% scored in the top range, compared to 7.0% and 10.2% for BT(P) and BT(F).

Novel Discovery 🔍: Intervening on the activations of special tokens via the SAEs leads to improvements in output quality, suggesting new opportunities for fine-grained control of model behavior, enabling broad adoption and future research.

Find the details in our post below👇

🔍Motivation: Why Traditional SAE Training Falls Short

Imagine reading a novel where every few pages, the story abruptly jumps to a completely different book—confusing📚✂️, right? This is essentially what happens with traditional Sparse Autoencoder (SAE) training methods for large language models!

Block Training (BT) has become the default approach for SAE training, where datasets (usually pretraining datasets) are concatenated into fixed-length blocks (Joseph Bloom and Chanin, 2024; Bricken et al., 2023). While this works reasonably well for base models—which are accustomed to processing random text chunks during pretraining—it creates significant problems for instruct models that have been fine-tuned to understand complete, coherent instructions.

Consider a typical 8,192-token training block: BT might stitch together 2,048 tokens from one sample with 6,144 tokens from another, creating jarring semantic discontinuities. For instruction-tuned models ...
Foresight Institute AI safety RFPs in automation, security, multi-agent, neuro
by Allison Duettmann — 2025-06-14T16:29:20.189Z

Foresight Institute is seeking project proposals across four AI Safety categories:

Automating research and forecasting

Security technologies for AI-relevant systems

Safe multi-agent scenarios

Neurotech for AI safety

This Request for Proposals builds up on our existing AI Safety Grants in these four categories by specifying the types of projects we would like to see more of in each category.

As with the prior grants program, we plan to continue to fund ~$5M in grants annually, and accept applications quarterly, with the next deadline on June 30th.

[Apply Now]

We seek proposals in the following areas:

Automating Research and Forecasting

Open-Source AI Research Agents: tools that automate key parts of scientific research – like reading papers, generating hypotheses, or designing and executing experiments – through open-source agents that can be adapted across domains.
Automated Forecasting Systems: systems that use AI to generate, compare, and collaborate on forecasts on critical developments – such as AI capabilities, regulation, or biosecurity risks – and present them in ways that builders and decision-makers can act on.

[Read more]

Security Technologies for AI-Relevant Systems

AI-Augmented Vulnerability Discovery and Formal Verification: tools that use AI to automate red-teaming, detect vulnerabilities, and formally verify critical systems.
Provably Secure Architectures and Privacy-Enhancing Cryptography: develop provable guarantees for system behavior and scalable cryptographic infrastructure to support trustworthy AI deployment.
Decentralized and Auditable Compute Infrastructure: infrastructure that distributes trust, increases transparency, and enables secure AI operation in adversarial environments.

[Read more]

Safe Multi-Agent Scenarios

AI for Negotiation and Mediation: concrete demonstrations of AI systems that help humans find common ground and reach beneficial agreements in complex negotiations.
Pareto-Preferred Coordination Agents: autonomous agents that can identify, negotiate, and enforce mutually beneficial arrangements between humans and other AI systems.
AI-Enhanced Group Coordination: AI systems that enhance collective intelligence and enable more effective group coordination around shared preferences.

[Read more]

Neurotech for AI Safety

Brain-Aligned AI Models: proposals

...
A Very Simple Case For Giving To Shrimp
by Bentham's Bulldog — 2025-06-14T15:31:32.017Z

Crossposted from my blog.

(I think this is an important article, so I’d really appreciate you sharing and restacking it—especially with people who are very rich).

I’ve

elsewhere

written

more

words

about

why

you

should give to shrimp welfare than are contained in a

typical Andy Masley post

(which, by conservative estimates, is at least 10 billion). I’ve talked about what makes pain bad. I’ve done rough Fermi calculations of the amount of agony averted. But here I’m going to try a different tack. I think people probably already care about shrimp enough that they should give to shrimp welfare.

Imagine that you discovered that there were 150 lobsters about to be boiled alive. There’s a machine designed to anesthetize them—so that they don’t have to struggle and writhe in agony for many minutes. Sadly, the machine is broken. To fix it, you would have to spend one single penny.

Question: is that a good use of a penny?

The obvious answer is yes! It’s worth spending a single penny so that 150 lobsters don’t have to experience painful deaths. But lobsters are no more important than shrimp and the Shrimp Welfare Project can anesthetize 150 shrimp before death per penny they receive (by purchasing stunners which stun thousands of animals per dollar raised). So if you think that it’s worth spending a penny to anesthetize 150 lobsters so they don’t have to experience being boiled alive, you should also support funding the

Shrimp Welfare Project

.

If you had the opportunity to keep doing this over and over again—to have each penny that you give stop 150 lobsters from experiencing the pain of being boiled—that would be even better. So similarly, you should give some money to shrimp welfare! If you give, say, 1,000 dollars, you can prevent ~15 million shrimp from experiencing extreme agony!

Consider another case: suppose that there were 100 shrimp suffocating to death in a bucket. You could put them back into the water, but doing so would take about a minute of your time. Assume that they’d be killed in a few minutes if they went back into the water, but you’d spare them from painful and slow suffocation. Question: should you spend that minute?

If the answer is yes, then I think you should donate to shrimp welfare.

Let’s assume you value your time at 15 dollars an hour, being fairly conservative. This means that you value each minute of your time at 25 cents. So you’re willing ...
Why we’re still doing normal school
by juliawise — 2025-06-14T12:40:54.747Z

Cross-posted from Otherwise.
Caveats: My oldest child is 11, and I don’t have parenting experience beyond elementary school. We’re lucky that our local public school is a good fit for our kids, and we’d be looking harder at alternatives if that weren’t the case.

See also: Raising children on the eve of AI

Current schools are preparing children for something like the present or the past. I think it’s very likely that AI will shake up work, economics, and daily life considerably by the time my children are adults. The way the schools are preparing them essentially for 2019 feels pretty absurd to me.

In my dream world, my kids would be doing something more flexible than school, with some able tutor to guide them at the right level, and plenty of outdoor time and visiting places in the real world. So why are we not doing that?

School is where the other children are

One of the key benefits homeschoolers point to is that they have more free time, because a classroom is inefficient for any given child. But what my older kids typically like to do with marginal free time is be with other kids, and school is their best opportunity for that. Even after school and on vacations, most of the other neighborhood kids are in some kind of program or camp. If you’re not in school, there’s a much higher barrier to hanging out with other kids. I realize there are homeschooling groups and stuff (and there’s a space near us that seems unusually good for this), but we’d have to spend a good bit of time driving them to these.

During pandemic-era remote school, my kid’s first grade was assigned to draw things they loved. One of the things she drew was her classmates, represented as little heads on a screen. I found that very sad.

Classmates on a laptop

We don’t want to quit our jobs

In a world where neither of us had the desire or opportunity to work on altruistic projects, Jeff might have been a stay-at-home dad. (I wouldn’t want to be home full-time, at least with little kids.) But we value the work both of us do.

In theory we could hire some kind of tutor or homeschooling facilitator, though I expect someone who could do a good job would cost considerably more than any nanny we’ve had.

Free public school makes parenting much less costly than it would otherwise be.

It also might just not be the right combination of people. One of my kids dislikes learning-type activities with me much more than ...
What Caused the Fertility Collapse?
by Zero Contradictions — 2025-06-14T07:15:42.839Z
Link: https://expandingrationality.substack.com/p/what-caused-the-fertility-collapse

Note: I didn't write this essay, nor do I own the blog where it came from. I'm just sharing it. The essay text is displayed below this line.

I recently put out a post on fertility collapse, The Paradox of Low Fertility, in which I claimed that the fertility collapse is mostly due to modern birth control. Some people disagreed with this explanation. One person, Peter Rabbit, said:

Your theses regarding the pill is a bit outdated. I would recommend arcotherium’s piece overviewing the four phases of demographic transition (in the West). A quick rebuttal is to point to France’s history.

I responded:

Yeah, I’ve read it. He underestimates the importance of birth control. I don’t think I mentioned the pill in the essay. Birth control is a more general notion, which includes many things, including the pill, condoms, abortion, the rhythm method, etc. E.g. in Japan, after WW2, they relied heavily on abortion to limit fertility. The point I’m making is that people have an increased ability to limit reproduction. Women also have an increased ability to delay marriage. There are multiple factors involved (which I described) but birth control is very important. There was a big collapse in fertility after the birth control pill was introduced in the West.

I am going to expand on what I said there, and respond to some of the points in Arctotherium’s essay, which mostly ignores birth control as a cause of low fertility.

His essay begins with the paragraph:

Human net fertility is complicated. Some things that matter on the margin: winning elections, baby simulators in health class, war, housing costs, religion (both type and intensity), women’s education, population density, racial diversity, STD-induced infertility, baby bonuses, antinatal propaganda campaigns and sterilizations, and status-messaging in soap operas. The full list is much longer. But most of these factors are just not that important—a few percent here, a few percent there, and with sharply diminishing returns.

Birth control is conspicuously absent, as if the birth control pill, latex condoms, IUDs and safe abortions had no effect on fertility whatsoever. You could argue that birth control is not the primary cause of low fertility, but to completely ignore it seems a bit strange.

Some people might say “Birth control has existed for a long time, and thus it couldn’t be the cause of this recent change”.

Of course, b...
Relocation triggers
by denkenberger — 2025-06-14T06:36:32.209Z

What would you do if a nuclear weapon were used in Ukraine? What about if there were evidence of a disease as transmissible as COVID-19 but more fatal, potentially enabled by

AI

?

While our community aims to reduce the chances of catastrophe, we may not always succeed. If the risk becomes very high in your location you might consider the possibility of relocating. Most people don’t have the luxury to consider moving, but many EA Forum readers do. For most people relocating, it will be a personal decision to increase the safety of themselves and their loved ones. But in some cases, this could be an altruistic decision, for example if your work is likely to be valuable post-catastrophe by helping to rebuild or mitigate further harm.

We think it's a good idea to think through what you would do ahead of time so that you can act quickly. Sentinel has put together a number of different potential

triggers

. It also has a fantastic weekly newsletter mailing list that will be sent as an emergency message if there is a particularly concerning trigger (sign up

here

- you may also be interested in applying to join their emergency response team). Of course it's going to depend on your particular circumstances, but you may want to move out of a high risk

city

or country.[1] If you’re able to travel far, you might consider Southern Hemisphere locations that are more likely to be resilient to

nuclear

winter

and islands that are more resistant to

pandemics

. There may be more temporary or permanent immigration options than you realise, for example, it has recently become easier to work remotely in New Zealand. If the risk level drops with no catastrophe, as we all hope it will, you could return to your original location. This would mean some people relocating could end up reflecting on this time as more like a working holiday.

^

For nuclear, depending on the trigger, this could include nuclear weapon states plus the rest of NATO.
Memory Decoding Journal Club: Neocortical synaptic engrams for remote contextual memories
by Devin Ward — 2025-06-14T02:26:41.113Z

Join Us for the Memory Decoding Journal Club!

A collaboration of the Carboncopies Foundation and BPF Aspirational Neuroscience

This time, we’re diving into a groundbreaking paper:
"Neocortical synaptic engrams for remote contextual memories"

Authors: Ji-Hye Lee, Woong Bin Kim, Eui Ho Park & Jun-Hyeong Cho

Institutions: University of California, Riverside, Department of Molecular Cell and Systems Biology.

Presented by: Dr. Randal Koene

When? June 17th, 2025 – 3:00 PM PDT | 6:00 PM EDT | 10:00 PM UTC

Where? Video conference:

https://carboncopies.org/aspirational-neuroscience

Register for updates:

https://aspirationalneuroscience.org/register-with-us/

Once registered, you'll receive event invites & updates!

#Neuroscience #MemoryResearch #Amygdala #JournalClub #BrainScience #Carboncopies #AspirationalNeuroscience
How concerned are you about a fast takeoff due to a leap in hardware usage?
by MichaelDickens — 2025-06-14T01:15:03.629Z

I am imagining a scenario like:

A company spends $10 billion training an AI.

The AI has fully human-level capabilities.

The company thinks, wow this is amazing, we can justify spending way more than $10 billion on something like this.

They don't bother with any algorithmic improvements or anything, they just run the same training but with $1 trillion instead. (Maybe they get a big loan.)

The $1 trillion AI is superintelligent.

The $1 trillion AI kills everyone.

Thus there is no period of recursive self-improvement, you just go from human-level to dead in a single step.

This scenario depends on some assumptions that seem kinda unlikely to me, but not crazy unlikely. I want to hear other people's thoughts.
How could I tell someone that consciousness is not the primary concern of AI Safety?
by Lysandre Terrisse — 2025-06-13T22:44:34.319Z

In the PDF version of the Dive into Deep Learning book, at page 27, we can read this:

Frequently, questions about a coming AI apocalypse and the plausibility of a singularity have been raised in non-technical articles. The fear is that somehow machine learning systems will become sentient and make decisions, independently of their programmers, that directly impact the lives of humans. To some extent, AI already affects the livelihood of humans in direct ways: creditworthiness is assessed automatically, autopilots mostly navigate vehicles, decisions about whether to grant bail use statistical data as input. More frivolously, we can ask Alexa to switch on the coffee machine.

Fortunately, we are far from a sentient AI system that could deliberately manipulate its human creators. First, AI systems are engineered, trained, and deployed in a specific, goal-oriented manner. While their behavior might give the illusion of general intelligence, it is a combination of rules, heuristics and statistical models that underlie the design. Second, at present, there are simply no tools for artificial general intelligence that are able to improve themselves, reason about themselves, and that are able to modify, extend, and improve their own architecture while trying to solve general tasks.

A much more pressing concern is how AI is being used in our daily lives. It is likely that many routine tasks, currently fulfilled by humans, can and will be automated. Farm robots will likely reduce the costs for organic farmers but they will also automate harvesting operations. This phase of the industrial revolution may have profound consequences for large swaths of society, since menial jobs provide much employment in many countries. Furthermore, statistical models, when applied without care, can lead to racial, gender, or age bias and raise reasonable concerns about procedural fairness if automated to drive consequential decisions. It is important to ensure that these algorithms are used with care. With what we know today, this strikes us as a much more pressing concern than the potential of malevolent superintelligence for destroying humanity.

If you have been interested about the alignment problem and AI safety, you should probably already know that the second sentence of the first paragraph is wrong. Indeed, both this sentence and the first sentence of the second paragraph are mentionin...