trentmkelly/LessWrong-43k huggingface dataset:
Meal Replacements in 2025?
by alkjash — 2025-01-06T15:37:25.041Z

I'm considering meal replacements for 1-2 meals a day, and recall Soylent and Mealsquares were popular meal replacements ~5 years ago when I visited the Berkeley rationalist scene. I haven't found any recent posts discussing the newer options like Huel or about long-term effects.

Does anyone have informed opinions on these things?
AI safety content you could create
by Adam Jones — 2025-01-06T15:35:56.167Z
Link: https://adamjones.me/blog/ai-safety-content/

This is a (slightly chaotic and scrappy) list of gaps in AI safety literature that I think would be useful/interesting to exist. I’ve broken it down into sections:

AI safety problems beyond alignment: Better explaining non-misalignment catastrophic AI risks.
Case studies of analogous problems: Historical lessons from nuclear weapons, surviving the resource curse, etc.
Plans: End-to-end plans for achieving AI safety.
Definitions: Accessible explanations of often-used AI safety terminology.

If you think there are articles that exist covering the topics described, please verify the articles you are thinking about do meet the criteria, and then tell me.

Communication of catastrophic AI safety problems outside alignment

I’ve previously written about how alignment is not all you need. And before me, others had written great things on parts of these problems. Friends have written up articles on parts of the economic transition, and specifically the intelligence curse.

Few people appear to be working on these problems, despite them seeming extremely important and neglected - and plausibly tractable? This might be because:

there is little understanding of these problems in the community;
the problems don’t match the existing community’s skills and experiences;
few people have started, so there aren’t obvious tractable in-roads to these problems; and
there aren’t organisations / structures for people to fit in to work on these problems

Tackling the first two issues might be done by spreading these messages more clearly. Corresponding semi-defined audiences:

The existing community. Funders or decision makers at AI policy orgs might be particularly useful.
People who would be useful to add to the community. These might be experts who could help by working on these problems, or at least beginning to think about them (e.g. economists, politics and international relations scholars, military/field strategists). I suspect there are many things where people from these fields will see obvious things we are missing.

There is downside risk here. We want to be particularly careful not heating up race dynamics further, particularly in messaging to the general public or people likely to make race-y decisions. For this reason I’m more excited about spreading messages about the coordination problem and economic transition problem, than the power distribution problem (see my problem naming for more c...
Childhood and Education #8: Dealing with the Internet
by Zvi — 2025-01-06T14:00:09.604Z

Related: On the 2nd CWT with Jonathan Haidt, The Kids are Not Okay, Full Access to Smartphones is Not Good For Children

It’s rough out there. In this post, I’ll cover the latest arguments that smartphones should be banned in schools, including simply because the notifications are too distracting (and if you don’t care much about that, why are the kids in school at all?), problems with kids on social media including many negative interactions, and also the new phenomenon called sextortion.

Table of Contents

How Many Notifications?.

Ban Smartphones in Schools.

Antisocial Media.

Screen Time.

Cyberbullying.

Sextortion.

How Many Notifications?

Tanagra Beast reruns the experiment of having a class tally their phone notifications. The results were highly compatible with the original experiment.

The tail, it was long.

Ah! So right away we can see a textbook long-tailed distribution. The top 20% of recipients accounted for 75% of all received notifications, and the bottom 20% for basically zero. We can also see that girls are more likely to be in that top tier, but they aren’t exactly crushing the boys.

What if you asked only about notifications that would actually distract?

There was even more concentration at the top. The more notifications you got, the more likely you were to be distracted by each one.

Here are some more highlights.

Which apps dominate? Instagram and Snapchat were nearly tied, and together accounted for 46% of all notifications. With vanilla text messages accounting for an additional 35%, we can comfortably say that social communications account for the great bulk of all in-class notifications.

There was little significant gender difference in the app data, with two minor apps accounting for the bulk of the variation: Ring (doorbell and house cameras) and Life 360 (friend/family location tracker), each of which sent several notifications to a few girls. (“Yeah,” said girls during our debriefing sessions, “girls are stalkers.” Other girls nodded in agreement.)

Notifications from Discord, Twitch, or other gaming-centric services were almost exclusively received by males, but there weren’t enough of these to pop out in the data.

The two top recipients, with their rate of 450 notifications per hour (!), or about one every eight seconds, had interesting stories to tell. One of these students had a job after school,

...
Latent Adversarial Training (LAT) Improves the Representation of Refusal
by alexandraabbas — 2025-01-06T10:24:53.419Z

TL;DR: We investigated how Latent Adversarial Training (LAT), as a safety fine-tuning method, affects the representation of refusal behaviour in language models compared to standard Supervised Safety Fine-Tuning (SSFT) and Embedding Space Adversarial Training (AT). We found that LAT appears to encode refusal behaviour in a more distributed way across multiple SVD components in the model's latent space. Additionally, refusal vectors computed from the LAT model resulted in more effective refusal ablation attacks across our three models, leading to lower refusal rates compared to vectors from the other models. Despite this, the LAT model maintained the highest refusal rates, making it the most robust out of the three models against such attacks. However LAT's better encoding of refusal behaviour could be exploited and result in more successful refusal attacks.

Our paper has been published in the proceedings of the Building Trust Workshop at ICLR 2025. See our paper on arXiv here.

Introduction

Latent Adversarial Training (LAT) is a recently proposed technique that introduces perturbations in a model's hidden layers, providing defences against threats like adversarial attacks and trojans without requiring specific failure examples. We investigated how LAT affects the representation of refusal behaviour by comparing it with supervised safety fine-tuning (SSFT) and embedding space adversarial training (AT) against an attack that compromises the model's ability to refuse harmful requests.

We compared these approaches by computing a "refusal direction" from contrasting pairs of harmful and harmless instructions from the AdvBench and Alpaca datasets, respectively. Our results show that LAT changes how refusal behaviour is encoded in the latent space, concentrating it more in the first two SVD components which account for a greater proportion of variance compared to the reference models. This altered representation results in a more effective ablation attack vector when generated from the LAT model. While LAT shows slightly improved robustness against attack vectors generated from both LAT and the reference models, the evidence for this improvement is not conclusive. We speculate that LAT, by exploring a broader space through perturbations, enables the model to learn a more comprehensive refusal feature, allowing for the computation of a more effective refusal vector. This...
Alternative Cancer Care As Biohacking & Book Review: Surviving "Terminal" Cancer
by DenizT — 2025-01-06T07:43:52.773Z

Introduction

I’ll write a series of posts in which I'll introduce alternative cancer care. I’ll explain why it can be a rigorous form of biohacking rather than mere quackery. I’ll review books popular in the alternative cancer care world like: Surviving Terminal Cancer by Ben Williams and How to Starve Cancer by Jane McLelland, both written by cancer survivors who overcame terminal stage 4 cancer. I’ll review the historical development of combination chemotherapy. I’ll review some famous cases e.g. the ‘spontaneous remission’ of Joe Tippens and the virologist Beta Halassy curing herself via self-experimentation. And I’ll review select phase 1/2 clinical trials. Throughout, I’ll criticize the FDA and the medical establishment for being too risk-averse, for having the wrong framework for clinical trials and having a frustratingly closed-minded epistemology.

This is also my first post on this website so I welcome any (additional) feedback. And last, I don’t have a formal background in biology, and this is the result of my independent research. My perspectives are “strongly held views, loosely held” and as such nothing here should be taken as medical advice.

What is Alternative Cancer Care?

Traditional cancer care is defined by two major themes: (1) Oncologists stick to the Standard Of Care (SoC) and (2) if that fails patients can be recruited into clinical trials that are carefully regulated. These clinical trials tend to test the efficacy and safety of the addition / replacement of one of the agents in the SoC. For example they try to answer questions like, for patients with Stage 4 small cell lung cancer with biomarker X, does drug A when taken in combination with already approved drug B extend progression free survival (PFS) in a statistically significant way? PFS refers to the length of time in which the cancer of the patient doesn’t progress i.e. gets worse.

I define alternative cancer care through a negative, as any treatment that deviates from this framework.

Summary of My Perspective.

Cancer is not a single disease; there is instead both inter-tumoral and intra-tumoral heterogeneity. Thus, it's better to think of it as a collection of diseases, even within a single tumor. These collections of diseases have independent vulnerabilities as well as independent capabilities to develop resistance to treatment.
As a result, cancer will be cu

...
Estimating the benefits of a new flu drug (BXM)
by DirectedEvolution — 2025-01-06T04:31:16.837Z

Introduction

H5N1 is a looming threat, making regular headlines. The CDC has identified 66 U.S. human cases as of January 4, 2024. Scott Alexander has a recent post outlining the situation.

There are several FDA-approved prescription antiviral flu drugs. The newest, using a novel mechanism of action called "cap-snatching," is baloxavir marboxil (BXM, brand name Xofluza). It is on-patent and substantially more expensive than oseltamivir (OST, brand name Tamiflu, now available as a generic).

I built a Guesstimate model to model the impact of using Xofluza at various stages of illness. A few facts about it:.

It is primarily based on data from the largest-to-date published clinical trials for BXM, as well as CDC flu modeling and some other flu-related academic literature.
In Guesstimate, users can simulate different flu and patient characteristics, substituting their assumptions for mine.
It models two flu types: normal-risk and higher-risk. Normal-risk flu is modeled on clinical trial results and CDC/academic flu modeling. Higher-risk flu traits are multipliers of those modeled for normal-risk flu and are based on case fatality rates for H5 and H7 flu. This is speculative, likely not very accurate for if a pandemic happened.
It is not medical advice, I am not a medical doctor.
I leave out financial cost and focus on patient health. Xofluza costs about $180 out of pocket according to Goodrx, which says it's covered by "some Medicare and insurance plans".
Elizabeth Van Nostrand offers a bounty for substantive bugs ($50/bug, up to $250 total, at her discretion).

Quantified example

Assumptions (the defaults you'll find if you click the link above):

85-year-old US male (10.5% chance of death in that year, 5.65 year life expectancy by the actuarial table) considering taking prophylactic BXM after his wife tests positive for influenza
He has comorbidities giving a 7x hospitalization risk if he becomes ill and a 2x disease duration multiplier
The chance his wife has higher-risk flu is about 1 in 500,000
Higher-risk flu lasts 5-24x longer, has a 200-4000x hospitalization and mortality risk, and is 1% as infectious as normal-risk flu

The model estimates the following benefits of taking prophylactic BXM over no prophylactic anti-flu drug and OST-only if hospitalized:

His chance of getting sick is only 24% of what it would be without medication (reduced by 76%)
His chance of hospitalization

...
Measuring Nonlinear Feature Interactions in Sparse Crosscoders [Project Proposal]
by Jason Gross — 2025-01-06T04:22:12.633Z

TL;DR

Problem: Sparse crosscoders are powerful tools for compressing neural network representations into interpretable features.
However, we don’t understand how features interact.

Perspective: We need systematic procedures to measure and rank nonlinear feature interactions.
This will help us identify which interactions deserve deeper interpretation.
Success can be measured by how useful these metrics are for applications like model diffing and finding adversarial examples.

Starting contribution: We develop a procedure based on compact proofs.
Working backwards from the assumption that features are linearly independent, we derive mathematical formulations for measuring feature interactions in ReLU and softmax attention.

Introduction

Training a sparse crosscoder (henceforth simply crosscoders) can be thought of as stacking the activations of the residual stream across all layers and training an SAE on the stacked vector.
This gives us a mapping between model activations and compressed, interpretable features.
We’re excited about this approach to automated interpretability, as it paves the way for accounting for feature interaction in addition to feature discovery.

In this draft project proposal, we describe the linear interaction assumption, and provide a compact-proofs-based explanation for why we’d like to measure feature interaction.
We are sharing concrete metrics for measuring feature interaction in ReLU and softmax attention.
We close with a discussion of empirical projects that we’re excited about, and some interesting takeaways.
Please reach out to us if you have feedback or would like to collaborate!

Linear Interaction Assumption

Crosscoders would be all we need if we could make the following two assumptions about how neural networks process information:

Linear Representation Hypothesis: Features are encoded explicitly and linearly in the residual stream.
For example, if a model needs to work with geometric areas, this hypothesis suggests the area value would be directly encoded, rather than linearly storing just the radius and computing or nonlinearly extracting area=πr2 when needed.

Linear Interaction Assumption: Features are independent.
This assumption suggests that the nonlinear computation in the model distributes over features in some approximately linear way.

If the linear representation hypothesis breaks down, then crosscoders ...
"We know how to build AGI" - Sam Altman
by Nikola Jurkovic — 2025-01-06T02:05:05.134Z
Link: https://blog.samaltman.com/reflections

We are now confident we know how to build AGI as we have traditionally understood it. We believe that, in 2025, we may see the first AI agents “join the workforce” and materially change the output of companies. We continue to believe that iteratively putting great tools in the hands of people leads to great, broadly-distributed outcomes.
We are beginning to turn our aim beyond that, to superintelligence in the true sense of the word. We love our current products, but we are here for the glorious future. With superintelligence, we can do anything else. Superintelligent tools could massively accelerate scientific discovery and innovation well beyond what we are capable of doing on our own, and in turn massively increase abundance and prosperity.

More context in this Bloomberg piece.

What’s the threshold where you’re going to say, “OK, we’ve achieved AGI now”?
The very rough way I try to think about it is when an AI system can do what very skilled humans in important jobs can do—I’d call that AGI. There’s then a bunch of follow-on questions like, well, is it the full job or only part of it? Can it start as a computer program and decide it wants to become a doctor? Can it do what the best people in the field can do or the 98th percentile? How autonomous is it? I don’t have deep, precise answers there yet, but if you could hire an AI as a remote employee to be a great software engineer, I think a lot of people would say, “OK, that’s AGI-ish.”
Now we’re going to move the goalposts, always, which is why this is hard, but I’ll stick with that as an answer. And then when I think about superintelligence, the key thing to me is, can this system rapidly increase the rate of scientific discovery that happens on planet Earth?
Is "hidden complexity of wishes problem" solved?
by Roman Malov — 2025-01-05T22:59:30.911Z

Can the problem described in the hidden complexity of wishes (at least partially) be considered solved? I believe that current LLMs are perfectly capable of answering a question like "How do I save grandma from a burning house?" without suggesting any unexpected maximums in an underspecified utility function.
However, I still think that the existence of LLMs capable of answering such questions is not sufficient proof of solving the problem (we do not have a robot capable of performing tasks of a similar level of difficulty to the 'saving grandma' task, with safety properties comparable to those that a human firefighter can provide when performing 'saving grandma' task). But can it at least be considered evidence supporting progress in that direction? This seems like a topic worth discussing, and I expect it has been discussed somewhere. If so, please link to it in answers.
A Ground-Level Perspective on Capacity Building in International Development
by Sean Aubin — 2025-01-05T20:36:54.308Z

I've been enjoying the blog/podcast Statecraft, which interviews powerful professionals in government and how they approach important problems, because its exposing me to many unfamiliar perspectives. In the spirit of Statecraft, but only being able to leverage my limited personal network, I wanted to interview my friend Abisola, who's done Capacity Building work as part of International Development efforts in Nigeria and Tanzania.

The implicit understanding I’ve gotten from reading GiveWell reports on International Development is that more direct interventions, like Unconditional Cash Transfers and Anti-Malaria Nets, are favoured over higher level Capacity Building interventions, like those done Oxfam or Partners in Health. This is because:

Is too hard to measure the effectiveness of
Even if you could measure it, it's probably going to be less effective in the long run, because higher levels of abstraction means more steps, which means more opportunities problems

Abi’s experiences helped me understand the incentives and structures at play in International Development.

My questions are in bold, with Abi's replies in unformatted text.

First some background. Where did you go and who were the organizations involved? Why do these orgs exist and what motivates them at a conceptual level?

I went to Tanzania to work with an organization we’ll pseudonymously call SEED to improve processes in Monitoring & Evaluation (M&E) via a Canadian organization we’ll pseudonymously call HELPER.

At a practical level, HELPER recruits professionals to volunteer in Africa, the Caribbean, and Northern Canada. HELPER uses the Capacity Building framework of International Development (ID), wherein they help organizations acquire skills they will use independently after HELPER is done.

At a conceptual level, HELPER exists for Canadian professionals to “engage” in development work. The Canadians gain experience working in another environment, while the requesting organization gains skills essential to the growth of their organization.

SEED is an agricultural cooperative. They aim to do a lot, with activities such as:

Training marginalized Tanzanian youth about farming methods.
Facilitating international internships for Tanzanian students to the USA, Europe, and Israel.
Supporting entrepreneurs with business coaching.
Providing agriculture focused business consulting to o

...
Why Linear AI Safety Hits a Wall and How Fractal Intelligence Unlocks Non-Linear Solutions
by Andy E Williams — 2025-01-05T17:08:06.734Z

1. The Non-Linear Challenge in AI Safety

Over the past decade, AI safety and alignment efforts have largely focused on incremental methods: refining RL-based guardrails, imposing regulatory oversight, and adding more researchers to tackle newly identified risks. Such approaches work well if each additional resource—a new policy, an extra auditor—can monitor a corresponding fraction of AI systems. This assumption underpins linear oversight: more resources yield a proportional (linear) increase in safety coverage.

Yet modern AI risk isn’t static or singular. Multi-agent systems and emergent synergies are exploding in complexity. Each new AI model or “agent” can introduce exponentially more interactions and failure modes, making purely linear expansions of oversight insufficient. Current methods barely keep pace with individual large language models, let alone the combinatorial challenges of “emergent behaviors” that arise when multiple models coordinate or compete.

2. Fractal Emergence: Why Complexity Grows Faster than Oversight

Underpinning these challenges is what some call the fractal intelligence hypothesis (Williams, 2024a). It suggests that intelligence—whether human or AI—tends to evolve in “gear shifts,” each new layer of organization creating exponential gains in problem-solving capacity. Examples include:

Individual cognition → Collective (group) cognition → Networks-of-networks (“intelligence-of-intelligences”).
A single neural net (first-order) → Multiple nets sharing semantic representations (second-order) → Hypergraph-level integrations (third-order), and so forth.

Such fractal expansion means that whenever we try to contain or monitor a given layer of AI, a new, higher-order arrangement can emerge, compounding complexity. If we only rely on linear solutions (like more red-teamers or manual audits), we are always a step behind these higher-order synergies.

3. Why Decentralized Collective Intelligence (DCI) May Provide the Non-Linear Jump

Because linear expansions of oversight break down under exponential complexity, we must consider a qualitatively different strategy. Decentralized collective intelligence (DCI) proposes distributing oversight and problem-solving across many agents—but in a way that leverages semantic interoperability to achieve non-linear gains.

Shared Semantic Foundation
Proponents of DCI emphasize a portable, interoperable Conceptual Spac

...