trentmkelly/LessWrong-43k huggingface dataset:
GPT-4.1 Is a Mini Upgrade
by Zvi — 2025-04-16T19:00:03.181Z

Yesterday’s news alert, nevertheless: The verdict is in. GPT-4.1-Mini in particular is an excellent practical model, offering strong performance at a good price. The full GPT-4.1 is an upgrade to OpenAI’s more expensive API offerings, it is modestly better but costs 5x as much. Both are worth considering for coding and various other API uses. If you have an agent or other app, it’s at least worth trying plugging these in and seeing how they do.

This post does not cover OpenAI’s new reasoning models. That was today’s announcement, which will be covered in full in a few days, once we know more.

Introducing GPT-4.1, 4.1-mini and 4.1-nano

That’s right, 4.1.

Here is their livestream, in case you aren’t like me and want to watch it.

On the one hand, I love that they might finally use a real version number with 4.1.

On the other hand, we would now have a GPT-4.1 that is being released after they previously released a GPT-4.5. The whole point of version numbers is to go in order.

The new cheat sheet for when to use GPT-4.1:

Will Brown: it’s simple, really. GPT-4.1 is o3 without reasoning, and GPT-4.1-mini is o4-mini without reasoning. o4-mini-low is GPT-4.1-mini with just a little bit of reasoning. o1 is 4o with reasoning, o1-mini is 4o-mini with a little bit of reasoning, o3-mini is 4o-mini with reasoning that’s like better but not necessarily more, and o4 is GPT-4.5 with reasoning.

if you asked an openai employee about this, they’d say something like “that’s wrong and an oversimplification but maybe a reasonable way to think about it”

I mean, I think that’s wrong, but I’m not confident I have the right version of it.

They are not putting GPT-4.1 in ChatGPT, only in the API. I don’t understand why.

Sam Altman: GPT-4.1 (and -mini and -nano) are now available in the API!

These models are great at coding, instruction following, and long context (1 million tokens). Benchmarks are strong, but we focused on real-world utility, and developers seem very happy.

GPT-4.1 family is API-only.

Greg Brockman: New model in our API — GPT-4.1. It’s great at coding, long context (1 million tokens), and instruction following.

Noam Brown: Our latest @OpenAI model, GPT-4.1, achieves 55% on SWE-Bench Verified *without being a reasoning model*. @michpokrass and team did an amazing job on this! (New reasoning models coming soon too.)

The best news is, Our Price Cheap, combined with the 1M ...
Doing Prioritization Better
by arvomm — 2025-04-16T18:46:41.797Z
Link: https://forum.effectivealtruism.org/posts/ZPdZv8sHuYndD8xhJ/doing-prioritization-better-2

Or on the types of prioritization, their strengths, pitfalls, and how EA should balance them

The cause prioritization landscape in EA is changing. Prominent groups have shut down, others have been founded, and everyone is trying to figure out how to prepare for AI. This is the first in a series of posts examining the state of cause prioritization and proposing strategies for moving forward.

Executive Summary

Performing prioritization work has been one of the main tasks, and arguably achievements, of EA.
We highlight three types of prioritization: Cause Prioritization, Within-Cause (Intervention) Prioritization, and Cross-Cause (Intervention) Prioritization.
We ask how much of EA prioritization work falls in each of these categories:

Our estimates suggest that, for the organizations we investigated, the current split is 89% within-cause work, 2% cross-cause, and 9% cause prioritization.
We then explore strengths and potential pitfalls of each level:

Cause prioritization offers a big-picture view for identifying pressing problems but can fail to capture the practical nuances that often determine real-world success.
Within-cause prioritization focuses on a narrower set of interventions with deeper more specialised analysis but risks missing higher-impact alternatives elsewhere.
Cross-cause prioritization broadens the scope to find synergies and the potential for greater impact, yet demands complex assumptions and compromises on measurement.
See the Summary Table below to view the considerations.

We encourage reflection and future work on what the best ways of prioritizing are and how EA should allocate resources between the three types.

With this in mind, we outline eight cruxes that sketch what factors could favor some types over others.
We also suggest some potential next steps aimed at refining our approach to prioritization by exploring variance, value of information, tractability, and the reliability of the different methods.

Since cross-cause prioritization work (and to a lesser extent cause prioritization) is presently rare, and has considerable benefits, the EA community may well be radically misallocating its prioritization efforts.

A diagram where the left pie chart represents an altruistic space of interest, three regions represent causes that partition the space, and dots represent interventions within that

...
Kamelo: A Rule-Based Constructed Language for Universal, Logical Communication
by Saif Khan — 2025-04-16T18:44:00.139Z

Introduction

Natural languages are messy, ambiguous, and often inefficient for transmitting structured ideas. Kamelo is a proposal for a constructed language designed to address these issues by building words from logical, compositional units. This post outlines the foundations of Kamelo—a rule-based, expandable language using fixed character sets and hierarchical categories to represent meaning with minimal ambiguity or memorization.

Kamelo is not intended to replace natural languages but rather to serve as a meta-language: a bridge for logical communication between humans, AIs, and across cultures, especially in low-bandwidth or assistive contexts. This proposal is relevant to LessWrong’s audience as it touches on rationality, AI alignment, and communication efficiency.

Motivation and Design Goals

Logical Construction: Every word is built from layered semantic categories—no arbitrary mappings.
No Memory Dependence: You can understand a word’s meaning by parsing its parts, not memorizing vocabulary.
Minimal Ambiguity: Sentence-level communication inherits meaning clearly from word-level rules.
Scalable: Works for both common and rare concepts using multi-level, logical trees.
Human-AI Symbiosis: Useful in alignment protocols, translation layers, or accessible UI design.

Core Mechanics of Kamelo

Alphabet Fixed 5-symbol phoneme set: ka, me, lo, ti, su (All words are built from these like a base-5 prefix tree)

Word Structure (Example: "apple")

Level	Encodes	Example Segment
L1	Word type	ka = Noun
L2	Noun subtype	ka = Proper noun
L3	Domain	su = Species
L4	Biological class	me = Plant
L5	Subclass	ti = Fruit
L6–L8	Meaning specificity	su-ka-ka-me (apple)

Each level is chosen from a tree of categories with 5 branches per level. More common distinctions appear earlier (shorter words).

Encoding Example: Apple

ka → Noun
ka → Proper Noun
su → Species
me → Plant
ti → Fruit
su → Family: Rosaceae
ka → Sweet taste
ka → Crunchy texture
me → Tree-grown

Resulting Kamelo word: kakasu meti susukakakakame

This structure is entirely self-descriptive if you know the rules.

Use Cases

Assistive Tech: Minimal phoneme-based speech for those with limited mobility.
AI Protocols: Alignment communication using rule-parsed, auditable intent structures.
Low-bandwidth communication: Works well over noise-prone audio or radio.
Cross-cultural linguistics: Universal

...
Understanding Trust: Overview Presentations
by abramdemski — 2025-04-16T18:08:31.064Z

Since mid-January, I've been working with four students - Hanna Gabor, Roman Malov, Norman Hsia, Paul Rapoport - on the research agenda I articulated last year. The AI Safety Camp project is wrapping up, but most of us plan to continue working on related things and meeting every Thursday to discuss and collaborate. On Thursday April 24th, we will present an overview of what we have thought about over the past few months. If anyone has interest in joining that weekly research group, this would be a good time to attend to get a sense of the group and see if the project is for you. Calendar link: https://calendar.app.google/bFcXqVBx1wGdy8vx7
Telescoping
by za3k — 2025-04-16T17:05:52.392Z
Link: https://blog.za3k.com/telescoping/

Sometimes I try to figure out what to do in a given week, and my first step is to think "what are my long term goals?". Then I realize I have no long-term goals. I sit down and think, "Hmm, what do I want to do with my life?" Eventually I give up, depressed, and go make breakfast or something.

Recently I tried going the other way, with much more success. I call this exercise telescoping. It takes 7 minutes, and is performed as follows:

Open a text document or grab a piece of paper. Grab a kitchen timer with an alarm, or your phone's alarm app.
For one minute, answer: What do I want to happen today?
For one minute, answer: What do I want to happen this week?
For one minute, answer: What do I want to happen this month?
For one minute, answer: What do I want to happen this year?
For one minute, answer: What do I want to happen this month?
For one minute, answer: What do I want to happen this week?
For one minute, answer: What do I want to happen today?

[... continued in blog ...]
Finance and AI Timelines
by DAL — 2025-04-16T16:55:06.957Z

This is in part a response to AI 2027, which I think is rather vague and somewhat unrealistic in projecting the financial aspects of the AI future. As a jumping off point, AI 2027 projects that a year from now, OpenAI’s valuation will have roughly quadrupled to $1 trillion and data center investment will have reached a half trillion dollars a year (about 10% of total US private investment), despite a piddling $26 billion in revenue.

This is a possible future — perhaps, so long as their is technological progress, investors will continue pouring money into AI in the pursuit of a speculative payday when AI is reached. But, I think Wall Street is currently (implicitly) forecasting a different future and that this will have an important influence on events.

Wall Street expects an AI Fizzle

Wall Street valuations right now suggest a future in which AI is a valuable technology, but no more “transformative” than the search engine.

Exhibit A here is everyone’s favorite AI trade — Nvidia. Nvidia has a $2.5 trillion market cap, which is heavily tied to AI-driven demand for the GPUs it designs but does not manufacture. Nvidia’s high valuation represents two bets — that AI will keep driving that demand but also that AI will not itself take over chip design. In a world with AGI (or certainly ASI), Nvidia is worthless. Even super-intelligent AI may well need TSMC or someone else to fabricate its chips, but it clearly does not need the engineers at Nvidia to design them.

The relative valuations of AI companies also likely reflect this dynamic. OpenAI trades at a substantial premium — with about four times the valuation of Anthropic. Perhaps this is because OpenAI is four times more likely than Anthropic to deliver AGI, but there is no obvious reason to make that bet. This is especially true if the route to AGI is to first developing a superhuman coder, given that Claude seems to lead the AI coding pack.

Instead, the easiest way to understand OpenAI’s relative valuation is based on its obvious strengths in branding, name recognition, and consumer adoption. OpenAI has by far the strongest brand in the space, to the point that the ChatGPT brand has become genericized. If the future path to profitability lies in providing a product to consumers whether on a subscription or an advertising model, then OpenAI...
FROM IA CODE TO HUMAN VALUES – 

A construction from MaxEnt Informational Efficiency in 4 questions
by P. João — 2025-04-16T16:53:51.351Z

With information theory, an AI can reduce complex data to 5% and reconstruct it with 95% functional fidelity; these are latent vectors. Maybe we could apply a bit of that to our latent values..

This document introduces an analytical tool that draws on information-theoretic principles Informational Efficiency under MaxEnt Constraints [1, 2] to examine human processes, starting from the hypothesis that it is possible to "break down" a human being into a system of interacting variables — and reconstruct like a vector of process [4, 5, 6].

The goal of this framework is not to compare individuals, but to enable self-evaluation through probabilistic lenses: to answer the question, “Am I more capable today of generating useful information than I was yesterday — and why?”

Binary distinctions such as elemental vs. personal, informational vs social, inward vs. outward, are used not as rigid categories, but as a orientation within a continuous and entangled field of human experience.

Of course, life resists full reduction to any single metric. But perhaps a slightly more precise metaphor can help spark deeper self-insight.

THE UNIVERSAL GOAL: USEFUL INFORMATION AND EFFICIENCY

Is there a variable more universal to human function than information?

Rooted in Maximum Entropy informational efficiency, we view the human being as a system that organizes, transforms, and propagates information—not as a passive physical body subject to chaos, but as an active agent refining its own informational processes. (Just as shadow doesn't exist, only the absence of light, chaos doesn't exist.)

The MaxEnt informational efficiency of a complex system is its ability to reorganize or generate useful information for itself (self‑updating) or for its environment (interaction), by maximizing subjective entropy under cognitive, contextual, or physiological constraints.

In this sense, the human being could be the most refined known entity for combining useful information that we know. Our attention, memory, language, and cultural tools allow for recursive information loops — we not only act, but learn from action; not only think, but think about thinking.

However, due to the multiplicity of variables, extracting useful information can be a challenge. That is why it becomes essential to formulate guiding questions that reduce the number of variables, allowing us to better organize and upd...
AI-enabled coups: a small group could use AI to seize power
by Tom Davidson — 2025-04-16T16:51:29.561Z

We’ve written a new report on the threat of AI-enabled coups.

I think this is a very serious risk – comparable in importance to AI takeover but much more neglected.

In fact, AI-enabled coups and AI takeover have pretty similar threat models. To see this, here’s a very basic threat model for AI takeover:

Humanity develops superhuman AI
Superhuman AI is misaligned and power-seeking
Superhuman AI seizes power for itself

And now here’s a closely analogous threat model for AI-enabled coups:

Humanity develops superhuman AI
Superhuman AI is controlled by a small group
Superhuman AI seizes power for the small group

While the report focuses on the risk that someone seizes power over a country, I think that similar dynamics could allow someone to take over the world. In fact, if someone wanted to take over the world, their best strategy might well be to first stage an AI-enabled coup in the United States (or whichever country leads on superhuman AI), and then go from there to world domination. A single person taking over the world would be really bad. I’ve previously argued that

it might even be worse than AI takeover

. [1]

The concrete threat models for AI-enabled coups that we discuss largely translate like-for-like over to the risk of AI takeover.[2] Similarly, there’s a lot of overlap in the mitigations that help with AI-enabled coups and AI takeover risk — e.g.

alignment audits

to ensure no human has made AI secretly loyal to them, transparency about AI capabilities, monitoring AI activities for suspicious behaviour, and infosecurity to prevent insiders from tampering with training.

If the world won't slow down AI development based on AI takeover risk (e.g. because there’s isn’t strong evidence for misalignment), then advocating for a slow down based on the risk of AI-enabled coups might be more convincing and achieve many of the same goals.

I really want to encourage readers — especially those at labs or governments — to do something about this risk, so here’s a link to our 15 page section on mitigations.

If you prefer video, I’ve recorded an podcast with 80,000 hours on this topic.

Okay, without further ado, here’s the summary of the report.

Summary

This report assesses the risk that a small group—or even just one person—could use advanced AI to stage a coup. An AI-enabled coup is most lik...
Ctrl-Z: Controlling AI Agents via Resampling
by Aryan Bhatt — 2025-04-16T16:21:23.781Z

We have released a new paper,

Ctrl-Z: Controlling AI Agents via Resampling

. This is the largest and most intricate study of control techniques to date: that is, techniques that aim to prevent catastrophic failures even if egregiously misaligned AIs attempt to subvert the techniques. We extend control protocols to a more realistic, multi-step setting, develop novel resample protocols that outperform all existing control protocols, and study many aspects of control techniques in more detail than prior work. In this blog post, we summarize the main takeaways and lessons learned from our work.

Here’s the abstract of the paper:

Control evaluations measure whether monitoring and security protocols for AI systems prevent intentionally subversive AI models from causing harm. Our work presents the first control evaluation performed in an agent environment. We construct BashBench, a dataset of 257 challenging multi-step system administration tasks, and evaluate whether various safety measures can prevent an adversarially constructed AI agent from covertly downloading and executing malicious code in this environment. This multi-step setting introduces new attack and defense dynamics, which we investigate in order to design novel control protocols that prevent safety failures without hindering the ability of non-malicious agents to perform useful work. We introduce a class of control protocols called resample protocols that dynamically take additional samples of certain actions. We find these protocols significantly improve on existing techniques by selectively blocking the AI agent from executing suspicious code and incriminating the agent by generating additional examples of dangerous behavior. We measure the tradeoff between attack prevention and usefulness; our best protocol combines resampling with analysis of previous steps, reducing the success rate of attacks from 58% to 7% at a 5% cost to the performance of a non-malicious agent.

This paper builds heavily on Redwood’s first paper on control evaluations,

AI Control: Improving Safety Despite Intentional Subversion

. If you haven’t read that paper, or are new to AI control, we think you’d get more value out of this post after reading it, along with

The case for ensuring that powerful AIs are controlled

(a more extensive reading list can be found

here

). Buck Shlegeris also talks about our new p...
Gamify life from BayesianMind
by P. João — 2025-04-16T16:17:49.284Z

This post presents an interesting framework for personal improvement, but I see we can enhance it by reconsidering two key aspects:

What Works in the Original System

The author effectively demonstrates how gamification can transform mundane tasks into engaging challenges by:

Assigning point values to activities
Creating a progress system similar to video games
Building consistent habits through regular engagement

Proposed Improvements

1. Eliminating Negative Scoring

The current system subtracts points for undesirable behaviors, but I propose a philosophical shift: nothing should have negative value. Just as darkness is merely the absence of light, "bad" behaviors simply contribute less value rather than negative value.

Benefits of this approach:

Avoids punishment-based motivation which can be discouraging
Creates a purely growth-oriented mindset
Aligns with positive psychology principles

Implementation: Assign very low (but still positive) point values to less beneficial activities while giving significantly higher values to highly beneficial ones. The relative difference remains motivating.

2. Dynamic Confidence-Based XP System

I propose that experience points should be gained when the system identifies variables that impact progress with higher confidence levels.

How it works:

The system tracks correlations between your activities and measurable outcomes
When it discovers a strong correlation (high confidence), you gain XP
This rewards you for finding what truly works for your specific situation
Points become personalized and evidence-based rather than predetermined

This creates a self-improving system where your actions continuously refine your understanding of what works for you personally.

By implementing these two improvements, we create a more psychologically sound and adaptive gamification system that evolves with you rather than imposing fixed external values.
Top OpenAI Catastrophic Risk Official Steps Down Abruptly
by garrison — 2025-04-16T16:04:28.115Z
Link: https://garrisonlovely.substack.com/p/breaking-top-openai-catastrophic

This is the full text of a post from Obsolete, a Substack that I write about the intersection of capitalism, geopolitics, and artificial intelligence. I’m a freelance journalist and the author of a forthcoming book called Obsolete: Power, Profit, and the Race to Build Machine Superintelligence. Consider subscribing to stay up to date with my work.

OpenAI's top safety staffer responsible for mitigating catastrophic risks quietly stepped down from the role weeks ago, according to a LinkedIn announcement posted yesterday.

Joaquin Quiñonero Candela, who took over OpenAI's Preparedness team in July,

announced

on LinkedIn that he has taken on a new role at the company:

I'm an intern! After 11 years since my last commit, I'm back to building. I first transitioned to management in 2009, and got more and more disconnected from code and hands-on work. Three weeks ago, I turned it all upside down, and became an intern in one of our awesome teams that's focused on healthcare applications of AI.

Candela's LinkedIn bio now describes him as the "Former Head of Preparedness at OpenAI."

An OpenAI spokesperson told Obsolete that Candela "was really closely involved in preparing the successor to the preparedness framework" and "will probably be involved in preparedness in some capacity" but is currently "focusing on different areas within the company that he's really excited about."

Candela’s new swag

The spokesperson added that the company recently restructured its safety organization, consolidating "all governance under the Safety Advisory Group" (SAG) — a committee chaired by five-year OpenAI veteran

Sandhini Agarwal

. The SAG uses a rotational leadership structure with one-year terms, designed, they said, to balance "continuity of knowledge and expertise" with "fresh and timely perspectives."

Meanwhile, OpenAI's preparedness work is now distributed across multiple teams, focused on things like capabilities, evaluations, and safety mitigations, the spokesperson said.

Candela's departure from the team comes amidst OpenAI's

contentious attempt

to shed the last vestiges of nonprofit control and follows a

string of scandals

and

high profile exits

in the last year.

It also marks the second major shakeup in the Preparedness team’s short history. In July, OpenAI removed Aleksander Mądry from his role as head of Preparedness — also without a public announcement. ...