trentmkelly/LessWrong-43k huggingface dataset:
8 PRIME IDENTITIES
- An analisis
by P. João — 2025-04-17T11:36:54.678Z

What is this about?

This project seeks to reduce the cognitive complexity involved in self-evaluating moments, goals, habits, and tasks among incarcerated individuals. I propose a discontinuous self-assessment system grounded in a model of eight intersecting identities.

With some parameters he has thus defined probabilistic identities, one can formulate questions for special moments in life to help organize events, routines, and tasks.

This approach approaches the atomic model where, even if we cannot precisely pinpoint an electron’s layer, through specific questions we can predict which area our focus lies in.

The apparent dichotomy serves as an analytical tool rather than an absolute division. In practice, these categories merge.

1. NAMES, TOTEMS, AND EVOLUTIONARY RELATIONSHIPS

Insects and cool colors for x “indoor” identities (interior metaphors)
Animals and warm colors for y “outdoor” identities (exterior metaphors)

3. HUMAN FUNCTIONS AND THEIR DECOMPOSITION

He starts from the universal objective of combining useful information. He decomposes this “mega-vector” into x/y (input/output) and four areas:

Elemental (genes ↔ environment)
Individual (personal memory ↔ environment)
Informational (memory ↔ memory)
Social (informational ↔ collaboration)

Each area generates two SubVectors: input focus (x) and output focus (y), totaling eight vectors.

2. 8 SUBVECTORS (8 identities)

Each identity and relationship with a human function in random order combining information.

Relationship to evolutionary psychology:

Each subvector aligns with adaptive modules that evolved to process useful information in different contexts (nutrition, defense, social cooperation, etc.).

4. 3 LEVELS OF QUESTIONS

For any activity, we ask:

Informational vs. Social

When sharing information, am I generating ideas for myself or collaborating with others?
Example: Writing in a journal vs. discussing a project with a team.

Individual vs. Informational

Within the personal domain, am I reviewing my memories or applying what I know?
Example: creating art vs. solving a problem.

Elemental vs. Individual

Am I working with basic, shared processes or with unique personal experiences?
Example: Hunt vs. creating art.

5. INITIAL KEY QUESTION

How do we decompose our mega-vector?

x and y (Internal vs. External)

Are you changing yourself or the environment?
Example: Eating (internal) vs. Hunting (external).

...
8 LATENT VALUES -
A simplified construction from MaxEnt Informational Efficiency in 4 questions
by P. João — 2025-04-17T11:04:07.424Z

What is this about?
This project seeks to reduce the cognitive complexity involved in self-evaluating moments, goals, habits, and tasks among incarcerated individuals. I propose a discontinuous self-assessment system grounded in a model of eight intersecting identities.

We often experience complex things (like learning) that can’t be understood just by looking at their pieces. This framework uses ideas from information theory (how systems handle uncertainty) to give us five simple questions for examining our own thinking and behavior.

1. The Mega Vector: Information Efficiency

The “Mega Vector”: We aim to maximize useful information under our own limits (time, energy, attention), MaxEnt principle, That is, what exists in common in every act would be to maximize the information available in the universe.:

Goal: Ask simple questions to decompose vector and see yours areas. So focus on what really matters and improve our ability to generate and use information.

2. The 3 Levels Questions

These help break down any activity by asking where your area focus lies:

Elemental vs. Individual?

Are you working with basic, shared processes (needs common of specimen, genetics) or with your own personal experiences and memories?

Example: Exercising follows common biological rules (elemental), creating art expresses your unique ideas (individual).

Individual vs. Informational?

Within personal focus, are you connecting your own memories with common specimen needs, or comparring the information, the memories?

Example: Creating art (individual) vs. using them to solve a problem (informational).

Informational vs. Social?

When using information, are you generating ideas for yourself or collaborating and communicating with others?
Example: Doing math (informational) vs. discussing a project with a team (social).

2. The initial Key Question

These helps break down any activity by asking where your main focus lies:

Decompose the “Mega Vector”

Inward vs. Outward?

Are you changing yourself or changing your surroundings?
Example: Eating fuels your body (x), while Hunting changes the environment (y).

IDENTITIES

We start from the universal objective of combining useful information. We decompose this “mega-vector” into x/y and four levels areas:

Elemental (environment ↔ genes)
Individual (Elemental ↔ memories)
Informational (Individual ↔ memories)
Social (Informational ↔ Collaborati

...
Automating Mechanistic Interpretability via Program Synthesis
by Edy Nastase — 2025-04-17T10:58:46.748Z

I have been researching for a while, and it seems to me that there isn't that much progress on "automating" MI using Program Synthesis. The only source I could find is a paper from Max Tegmark's lab. However, this paper has been about for quiet a while, and not that much progress has been happening since then (or maybe I am unaware of it).

The inspiration of using Program Synthesis comes from Stephen Casper suggesting that program synthesis could lead to the process of automating MI. In the post itself, he mentions some papers that transform RL policies into programs, while the rest are some obscure methods. However, none of them are actually trying to reverse parts of neural networks into "programs"...

Now, I was thinking about doing some research on this myself, and see if I could end up with a simple prototype. Something like this could represent some of the initial steps:

Select a domain area/data that could be used for experimenting. I think previous MI problems train the networks on arithmetic, operations with integers/Booleans etc. and then try to reverse engineering from there.
A method of identifying circuits aka networks within the network is probably necessary to understand what different parts of the network.
Given a circuit, I need to find a way of identifying whether this could be reversed or not into a program. Maybe if I define a DSL (Domain Specific Language) for the given problem domain, I could try different combinations of sub-graphs inputs/outputs until time runs out? Or it can be a type-search where only specific operations between types are allowed.
From that point, if the prototype is ok, I could start thinking about integrating the system with something like Dream Coder , which essentially "discovers the DSL" given training data and a base language (which could be just the defined DSL, but simplified)

What do you guys think? Why isn't there much research in this direction? What is missing from the above proposed plan?
Understanding and overcoming AGI apathy
by Dhruv Sumathi — 2025-04-17T01:04:53.853Z
Link: https://dhruvsumathi.substack.com/p/waves-not-mountains-overcoming-apathy?r=b10d1

Crossposting from my substack.

Note for LW: This post is about the apathy and nonchalance I've been seeing in many peers regarding AGI. There seems to be massive cognitive dissonance (and it isn't straightforward to overcome) – how can we on the one hand talk about things like AIs becoming smarter than humans across all domains, and on the other just carry on with our everyday lives? I've been searching for ways to explain this kind of apathy, and for ways to convey how serious this is to people that don't think about it much (e.g. my parents). This is my first attempt at doing that, and I'd appreciate any and all thoughts.

–––

Note: I use a combination of real and fictional names for the characters in section 1.

The revolution will not be televised
The revolution will not be brought to you
By Xerox in four parts without commercial interruptions
The revolution will not show you pictures of Nixon blowing a bugle
And leading a charge by John Mitchell, General Abrams, and Spiro Agnew
To eat hog maws confiscated from a Harlem sanctuary
The revolution will not be televised

Gil Scott-Heron

1. Tsunami

When the Earth starts to... settle, God throws a stone at it. And believe me, He's winding up.

Ultron

Waves

It’s Tuesday morning. I’m enjoying my quiet drive to work. The sun is beaming warmth onto my arms through the car window. To my left, the bay is a deep turquoise, a color that I’ve come to associate with San Francisco’s best days.

My phone screams at me. I jolt and pick it up to see what’s happened. “Tsunami Warning.” My stomach tightens. I’ve never seen one of these before. Earthquake off the coast of Eureka, it says. Chance of tsunami. Get to high ground.

Hmm. Whatever. I put my phone down and continue driving.

Something feels off and a sense of unease wells up inside my chest. I’ve seen The Impossible, so I’m a little worried. I pick up the phone and call my brother, Chirag. He lives in San Francisco.

“Hey, did you get the tsunami warning?”

“Yup, that was weird.”

“Yeah, strange. What should we do?”

“I’m not sure, I think it’s fine though. We’ll see.”

“Okay, just be careful.”

I put the phone down and continue my drive, feeling slightly eased by Chirag’s reaction (or lack thereof). I pull into the office parking lot and park alongside my coworker’s car. She’s just getting out. I open my door, climb out onto the little side-step, and call out.

“Hey Dhruv,” she says calmly.

“Did you see th...
ALLFED emergency appeal: Help us raise $800,000 to avoid cutting half of programs
by denkenberger — 2025-04-16T21:47:40.687Z

SUMMARY:

ALLFED is making an emergency appeal here due to a serious funding shortfall. Without new support, ALLFED will be forced to cut half our budget in the coming months, drastically reducing our capacity to help build global food system resilience for catastrophic scenarios like nuclear winter, a severe pandemic, or infrastructure breakdown. ALLFED is seeking $800,000 over the course of 2025 to sustain its team, continue policy-relevant research, and move forward with pilot projects that could save lives in a catastrophe. As funding priorities shift toward AI safety, we believe resilient food solutions remain a highly cost-effective way to protect the future. If you’re able to support or share this appeal, please visit

allfed.info/donate

.

FULL ARTICLE:

I (David Denkenberger) am writing alongside one of my team-mates, as ALLFED’s co-founder, to ask for your support. This is the first time in Alliance to Feed the Earth in Disaster’s (ALLFED’s) 8 year existence that we have reached out on a forum with a

direct funding appeal

outside of Marginal Funding Week/our annual updates. I am doing so because ALLFED’s funding situation is serious.

At ALLFED, we are deeply grateful to all our supporters, including the Survival and Flourishing Fund, which has provided the majority of our funding for years. At the end of 2024, we learned we would be receiving far less support than expected due to a shift in SFF’s strategic priorities toward AI safety.

Without additional funding, ALLFED will need to shrink. I believe the marginal cost effectiveness for

improving the future and saving lives of resilience is competitive with AI Safety

, even if timelines are short, because of potential AI-induced catastrophes.

ALLFED’s funding situation

Without new funding ALLFED will need to cut half of our budget in the coming months, the outcome of which will be making significant cut backs to our team in June. This will reduce by about half our capacity to produce research, support governments, and develop practical interventions that could reduce the risk of global catastrophic food system failure during catastrophic events.

The case for ALLFED’s work, and why we think maintaining full current capacity is valuable

Today we are launching an urgent

2025 Appeal

to ask for your help. With your support, we can continue advancing this work and pr...
Prodromes and Biomarkers in Chronic Disease
by sarahconstantin — 2025-04-16T21:30:02.978Z

Midjourney

Thanks to Renaissance Philanthropy for their support of my research on this topic.

Why Prodromes?

A lot of chronic diseases, particularly autoimmune and neurodegenerative disorders, develop over the course of years, as the patient transitions from “healthy” to “sick”.

This may involve the loss of neurons (as in neurodegenerative diseases like Alzheimer’s or Parkinson’s) or the loss of other kinds of cells (like the insulin-producing cells of the pancreas in type 1 diabetes). Once cells and tissues are permanently damaged or lost, it can be difficult or even impossible to return to the pre-disease state. We may use drugs to treat symptoms (such as insulin in diabetes, to substitute for the insulin the pancreas isn’t producing, or L-dopa in Parkinson’s, to substitute for the dopamine the brain isn’t producing) but true recovery from the disease isn’t tractable.

On the other hand, we might have a better chance of discovering effective treatments if we start earlier.

Often, chronic diseases have a “presymptomatic”, “preclinical”, or “prodromal” stage, when it’s too early to diagnose the full disease, but the patient may have some mild symptoms, or there may be lab findings characteristic of developing disease.

We’re already used to this in conditions like cancer (where it’s routine to operate on precancerous growths of the skin, bowel, or cervix) or heart disease (where it’s routine to give cholesterol-lowering medications based on lab findings alone, before the patient has reported any symptoms.)

Preventative therapy is even beginning to be used in some new diseases, like type 1 diabetes. T1D is an autoimmune disease that destroys the islet cells of the pancreas, and preclinical T1D can be detected by the presence of certain autoantibodies that direct the patient’s immune system to attack their own pancreas. In 2022, the first T1D-preventing drug, teplizumab, was FDA-approved for patients who have diabetes autoantibodies and abnormal blood sugar but no symptoms yet.

Prodromes and Biomarkers

What does it look like to have pre-Parkinson’s? Pre-multiple sclerosis? Pre-rheumatoid arthritis?

In many cases we don’t have a standard definition of a prodromal or preclinical phase of disease, we just know that there are certain symptoms and findings that are more likely to occur in people who go on to get a disease diagnosis.

And for pretty much all chronic diseases, we...
The Practical Imperative for AI Control Research 
by Archana Vaidheeswaran — 2025-04-16T20:27:32.319Z

As we continue to develop increasingly capable AI systems, many of us have noted the striking contrast between two areas of research. On one hand, we see tremendous energy devoted to making AI systems more capable - better at following instructions, solving problems, and generally doing what we want. On the other hand, we see comparatively less attention paid to a question that seems increasingly important: how do we ensure these systems don't do what we don't want, especially if they're actively trying to subvert our safeguards?

This is the core challenge of AI control - developing techniques that remain robust even against an AI system that might be actively working to circumvent them.

Why Control Matters (Even If You're Not Worried About AGI)

The control problem isn't just relevant to far-future concerns about superintelligence. It matters today because we're already deploying systems with access to sensitive data, critical infrastructure, and decision-making capabilities. Consider a few scenarios:

An AI system with access to confidential company data that might attempt to exfiltrate that information
AI coding assistants that could insert subtle backdoors into security-critical software
Autonomous systems that could learn to circumvent monitoring when deployed in high-stakes environments

The common thread is that we can't simply rely on "asking nicely" - we need techniques that remain robust even against a system that might be actively trying to subvert them.

What makes control particularly interesting is its focus on zero trust. Unlike alignment research that assumes we can train AI systems to want the right things, control research asks: "What if the system wants the wrong things? How can we still ensure safety?"

The Current Research Landscape

The formal study of AI control is surprisingly recent. Redwood Research's 2023 paper "AI Control: Improving Safety Despite Intentional Subversion" laid out the core methodology for systematically evaluating control techniques. Since then, there's been growing interest in developing and testing control protocols.

Current research typically involves a "red team/blue team" dynamic where:

Blue teams develop monitoring systems and control protocols
Red teams attempt to circumvent those safeguards
Both sides iterate to develop more robust techniques

This approach has yielded insights into monitoring systems, protocol design, and adver...
Mass Exposure Paradox
by max-sixty — 2025-04-16T20:18:00.492Z

Why are so many health-related anxieties related to exposures that we all encounter?

Here's a model that offers some explanation — I'm calling it the mass exposure paradox. It arises from two opposing consequences of universal exposure:

Mass exposure to something harmful would generate highly significant evidence for its harm. When billions of people use or consume something for decades without a clear epidemic of harm, it becomes strong evidence against large effect sizes. Genuinely harmful exposures — such as leaded gasoline or cigarette smoking — leave unmistakable signals.

Mass exposure boosts memetic fitness. When everyone is exposed, narratives of harm can gain cultural traction, sweeping through social networks and broadcast media.

Examples

Cellphones & brain cancer: pervasive use since the 1990s; no solid tumor spike, but fears linger.

Aspartame: in foods since the 1980s, extensively studied; fears persist despite reassuring evidence.

Vaccines: centuries of use; exceptional safety record despite rare adverse events; we still have large anti-vax movements

Other examples: GMOs, WiFi/5G, seed oils, "toxins", microplastics.

In each case, mass exposure provided a natural experiment demonstrating safety, while its memetic fitness allows anxiety to spread and persist.

Mass Exposure as Meta-Evidence

The very ubiquity of an exposure strongly implies it's safe. If billions of people have been exposed for decades yet we still debate whether any real effect exists, the effect size must be negligible or nonexistent.

Genuinely hazardous substances which had widespread exposure (lead, asbestos, smoking) show clear, undeniable population-level harm without cherry-picking studies. We can generally rely on intensive research scrutiny finding the real dangers of widespread exposures if they exist.

In contrast, if you're worried about harm from exposures, it's much more likely to come from something you're specifically exposed to — something in your town, related to your behavior, or affecting people with your genes — because we don't have nearly as much evidence on those things.

Memetic Fitness Fueled by Universal Relevance

Why do these low-evidence fears flourish and persist? Precisely because exposure is universal:

Shared context: Everyone is exposed, making anxieties easy to share. Economies of scale in media mean that it's worthwhile to, say, do

...
GPT-4.5 is Cognitive Empathy, Sonnet 3.5 is Affective Empathy
by Jack — 2025-04-16T19:12:38.789Z

This was the thing my friend told me I should make an account here to post, I hope it is appropriate.

Introduction

The thrust of this hypothesis is that Anthropic's research direction favors the production of affective empathy while OpenAI's direction favors cognitive empathy.

There are a variety of methods employed to make LLMs appear more friendly and personable, but the actual manner in which these manifest differ. The two major directions for developing LLM personability that I am delineating here:

1. Affective, where an LLM simply behaves in a way that is empathetic without necessarily understanding why a user may be feeling some way.

2. Cognitive, when an LLM develops a sophisticated world model that predicts how the user is feeling without necessarily directing the LLM to act in such a way.

By enhancing these aspects with RLHF there is the promise of ensuring that the affective empathetic response provides the necessary signals to reassure a user, while providing the cognitive empathetic response a stronger bias towards actually being nice. While both can result in similar behaviour and levels of personability there does appear to be significant cost trade offs, and I would hope to spark a broader discussion on how these might produce different failure modes.

The reason I believe this to be important is the broader strokes of the research direction at Anthropic and OpenAI. Which insofar as can be inferred by the limited publicly available data at hand, seem to favor affective and cognitive empathy respectively. While this is speculative, I believe the bones are still valid regardless of the specifics of the lab's methods. What are the risks of having only one type of empathy in our LLMs? What is the risk of having both?

Affective Empathy of Activation Steering

Affective empathy allows for someone to reactively minimize the surprise at the emotional state of another entity in low-information environments. Importantly, the response is often without significant awareness of why, with post-hoc rationalizations tagged on. In short, you don't need to be "smart" to experience high levels of affective empathy.

To start with, I want to make the case that most of the gains from Sonnet 3.5 appear to be the result of activation steering, which was first properly showcased in the now famous Golden Gate Claude. It was cute watching the Claude-lik...