trentmkelly/LessWrong-43k huggingface dataset:
A Critique of “Utility”
by Zero Contradictions — 2025-03-20T23:21:41.900Z
Link: https://thewaywardaxolotl.blogspot.com/2025/03/a-critique-of-utility.html

Note: I didn't write this essay, nor do I own the blog where it came from. The essay text is displayed below this line.

Economists and other social theorists often take the concept of utility for granted.

In theory, utility is a measure of value. I will denote the utility of X to Y as U(X, Y) or simply U(X) if Y is clear in the context. For a single individual, we can think of utility as a mathematical function that maps objects or outcomes to points on the continuum. This mapping is called a “utility function”. (In this context, “function” has its mathematical meaning, not its ordinary meaning.)

To think about society, we might want to define utility for a group, not just a single individual. Collective utility is typically defined as the sum of individual utility over the collective. This assumes that utility is defined on the same scale for everyone.

The concept of utility is used to model individual and social decision-making. It is used in game theory to define benefits and costs. It is often used in moral philosophy to define the “goodness” of actions and outcomes. In a utilitarian moral theory, collective utility is the measure of goodness, and we have a moral obligation to act in ways that increase it. In economics, it is often assumed that prices measure the utility of goods and services. GDP is often viewed as a measure of collective utility for a society.

The concept of utility is based on an underlying metaphor: thinking of value as a substance. A substance can be quantified, measured, compared, summed over a collection, exchanged, produced, consumed, etc. Metaphorically, we think of U(X, Y) as defining a quantity of a substance that is present in X. We say “How much value does X have?”, as if value is a substance contained in X. We talk about how much value a company produces, or an entire economy produces, as if value is a substance that we produce and consume.

Does this metaphor make sense? Is value like a substance?

In this essay, I will argue that the concept of utility is a convenient fiction.

Utility can be defined in different ways:

Reproductive utility: U(X, Y) is how much X contributes to the reproductive fitness of Y.
Desiric utility: U(X, Y) is how much Y psychologically values X.
Hedonic utility: U(X, Y) is the net effect of X on Y’s experiences of pleasure and pain.
Energetic utility: U(X, Y) is the amount of work that Y would do for X.
Monetary

...
Intention to Treat
by Alicorn — 2025-03-20T20:01:19.456Z

When my son was three, we enrolled him in a study of a vision condition that runs in my family. They wanted us to put an eyepatch on him for part of each day, with a little sensor object that went under the patch and detected body heat to record when we were doing it. They paid for his first pair of glasses and all the eye doctor visits to check up on how he was coming along, plus every time we brought him in we got fifty bucks in Amazon gift credit.

I reiterate, he was three. (To begin with. His fourth birthday occurred while the study was still ongoing.)

So he managed to lose or destroy more than half a dozen pairs of glasses and we had to start buying them in batches to minimize glasses-less time while waiting for each new Zenni delivery. (The patching protocol was useless without the glasses to go with it.) He would sneak his eyepatch off when we weren't looking and rely on having more parents than the average bear, most of which parents have ADHD diagnoses, to confuse us about whether he was supposed to be patched at any given time. The sensor doodad and its backup were lost or "lost" in the couch cushions over and over until we couldn't find either one again. He was at first extremely prone to falling asleep once the patch was on, which meant he wasn't looking at things with his uncovered eye during the intended patch time. We bribed him with a dollar per day he was cooperative with the patch (an amendment from our original plan to just give him the value of the Amazon credit - between the replacement glasses and the bribery we were down on net. The bribery alone would have ensured that if he'd been more cooperative, but he wasn't.)

They didn't drop us out of the study. In fact, they eventually concluded that their protocol had had the effect they wanted - he's apparently trained away the asymmetry that made it impossible for my father to use a binocular microscope in med school. They didn't make a fuss about the missing sensors, the forgotten days, the napped-through patching hours, the lacunae in glasses compliance.

He was three. They knew he was three. No human three year old is perfectly compliant with a patch protocol and the process of trying to force it to occur perfectly would be nightmarish for within the study, let alone if they popularize this approach as a clini...
Anthropic: Progress from our Frontier Red Team
by UnofficialLinkpostBot — 2025-03-20T19:12:02.151Z
Link: https://www.anthropic.com/news/strategic-warning-for-ai-risk-progress-and-insights-from-our-frontier-red-team

Note: This is an automated crosspost from Anthropic. The bot selects content from many AI safety-relevant sources. Not affiliated with the authors or their organization and not affiliated with LW.

In this post, we are sharing what we have learned about the trajectory of potential national security risks from frontier AI models, along with some of our thoughts about challenges and best practices in evaluating these risks. The information in this post is based on work we’ve carried out over the last year across four model releases. Our assessment is that AI models are displaying ‘early warning’ signs of rapid progress in key dual-use capabilities: models are approaching, and in some cases exceeding, undergraduate-level skills in cybersecurity and expert-level knowledge in some areas of biology. However, present-day models fall short of thresholds at which we consider them to generate substantially elevated risks to national security.

AI is improving rapidly across domains

While AI capabilities are advancing quickly in many areas, it's important to note that real-world risks depend on multiple factors beyond AI itself. Physical constraints, specialized equipment, human expertise, and practical implementation challenges all remain significant barriers, even as AI improves at tasks that require intelligence and knowledge. With this context in mind, here's what we've learned about AI capability advancement across key domains.

Cybersecurity

In the cyber domain, 2024 was a ‘zero to one’ moment. In Capture The Flag (CTF) exercises — cybersecurity challenges that involve finding and exploiting software vulnerabilities in a controlled environment — Claude improved from the level of a high schooler to the level of an undergraduate in just one year.

Figure 1: In less than a year, Claude went from solving less than a quarter to nearly all Intercode CTF challenges, publicly available CTFs that were designed for high school competitions.

We are confident this reflects authentic improvement in capability because we custom-developed additional challenges to ensure they were not inadvertently in the model’s training data.

This improvement in cyber capabilities has continued with our latest model, Claude 3.7 Sonnet. On Cybench — a public benchmark using CTF challenges to evaluate LLMs — Claude 3.7 Sonnet solves about a third of the challenges within five attempts, up from about five pe...
Everything's An Emergency 
by Bentham's Bulldog — 2025-03-20T17:12:23.006Z

Crosspost of an article on my totes amazing blog that you should check out.

“Everything’s an emergency” is the title of a

book that I have not read

. It’s also the title of a

very good substack

by Bess Stillman, wife of the recently deceased Jake Seliger. It also happens to be true.

Every day, about

14,000 children die

. That’s 5 million per year, 10 per minute. Many of these children die in the arms of their parents who love them, but who cannot save them—they lie in distant and desperately poor countries beyond the reach of modern medicine. Most of these deaths are preventable. But they are not prevented.

About

80 billion land animals

are slaughtered every year. That’s about 200 million per day, 9 million per hour, 375,000 per minute, 6,000 per second.

Most of these animals

endured

hellish lives in factory farms

; beaten, caged, abused, forced to live in nightmarish conditions without ever being able to turn around or stretch their wings. Imagine looking out over a vast sea of 6,000 animal carcasses; that is what we produce every second.

Imagine that there was a machine that every second murdered 6,000 dogs. Before killing them, however, it subjected them to cruelty and indignity; it locked them in a cage where they did not see the sun. It sliced off their claws with a knife, and castrated them with no anesthetic. It genetically engineered them so that they became hideous inversions of their natural state—too obese to move without great pain, forced to constantly lie against the painful wire meshing that surrounds them. The dogs in the cages went crazy, routinely biting each other. Injury was routine, extreme, and often lethal.

This is the scale of the horror of factory farming. We are annually eviscerating a population of land animals roughly equal in number to all humans who have ever lived.

Even this horror is but a tiny drop compared to the

suffering of wild animals

. By very conservative estimates, there are 10^17 morally important wild animals. If we assume conservatively this is roughly the number that die in a year, that comes out to around 3 billion deaths a second. Compared to the 3 billion painful deaths per second, everything else looks like a mere drop against this vast ocean of suffering. Given that most animals live short lives, it’s much more likely that at least 30 billion conscious animals die a second, and potentially the number is even in ...
Non-Consensual Consent: The Performance of Choice in a Coercive World

by Alex_Steiner — 2025-03-20T17:12:16.302Z

Crossposted from Substack

Consider the following scenario: You're sitting in a job interview for a position you desperately need to pay rent. The interviewer asks, "Why are you passionate about working in insurance claims processing?" Despite feeling nothing resembling passion, you fabricate enthusiasm, carefully crafting a narrative about your deep interest in risk assessment and customer service.

This exchange represents ironic discourse—both you and the interviewer recognize the fabrication, yet both participate in maintaining it. Neither acknowledges the obvious truth: you need money to survive, they need labor to profit, and the enthusiasm narrative is merely ceremonial cover for this basic transaction.

This ritual exemplifies non-consensual consent—a pervasive societal mechanism forcing individuals to perform consent while removing meaningful alternatives.

The Two-Layered System

"Man is born free, and everywhere he is in chains. One thinks himself the master of others, and still remains a greater slave than they." — Jean-Jacques Rousseau

Non-consensual consent (NCN) is the inverse of "consensual non-consent" (CNC)—a concept most familiar from BDSM and power exchange relationships. CNC begins with an authentic moment of consent—typically formalized through explicit negotiation and boundaries—followed by a mutually desired performance of non-consent. The foundation rests on genuine choice, where participants voluntarily surrender control within pre-defined parameters, ensuring safety and trust while maintaining the illusion of coercion.

In consensual non-consent: You actually have freedom but voluntarily surrender it while pretending you don't have choice. The submissive partner genuinely chooses to participate in a scenario where they act as though they have no choice.

In non-consensual consent: You don't have freedom but must pretend you do have it. You're forced to perform the theater of choice while having no meaningful alternatives.

NCN appears whenever structures require individuals to maintain the fiction of choice while eliminating meaningful alternatives. The insidiousness lies not merely in the lack of options, but in the mandatory performance of having chosen freely.

"Freedom is what you do with what's been done to you." — Jean-Paul Sartre

But what happens when what's been done to you requires pretending nothing was done to you at all?

The Sex work

...
Minor interpretability exploration #4: LayerNorm and the learning coefficient
by Rareș Baron — 2025-03-20T16:18:04.801Z

Epistemic status: small exploration without previous predictions, results low-stakes and likely correct.

Introduction

As a personal exercise for building research taste and experience in the domain of AI safety and specifically interpretability, I have done four minor projects, all building upon code previously written. They were done without previously formulated hypotheses or expectations, but merely to check for anything interesting in low-hanging fruit. In the end, they have not given major insights, but I hope they will be of small use and interest for people working in these domains.

This is the fourth project: checking how the LLC behaves for toy models containing LayerNorm and sharp transitions in the loss landscape (this project from Timaeus).

TL;DR results

LLC behaves as expected. Large drops in loss are mirrored by large spikes in the LLC.

Methods

The basis for these findings is the DLNS notebook of the devinterp library, plus the example code given. Notebooks and graphs can be found here.

Results

There does not seem to be anything peculiar about the progression of the LLC. The sudden drop in loss was mirrored by a sudden rise in LLC, showing a highly compartmentalized loss landscape.

Discussion

This was an attempt to check the validity of LLC for sharply delineated loss landscapes, and to see if any strange results appear when looking at the loss landscape of a LayerNorm model, which contain a feature that the interpretability community dislikes for many reasons. LLC behaved as expected.

I must mention that the project page loss does not one-to-one correspond to the losses registered on the given model. This might be a problem on my end, but I doubt it is a large issue.

Conclusion

This confirmation that LLC acts as it is supposed to even in strange loss regimes is encouraging, indicating that it truly reflects fundamental properties of the network training process.

Acknowledgements

I would like to thank the original Timaeus team for starting this research direction, establishing its methods, and writing the relevant code.
How far along Metr's law can AI start automating or helping with alignment research?
by Christopher King — 2025-03-20T15:58:08.369Z

In METR: Measuring AI Ability to Complete Long Tasks found a Moore's law like trend relating (model release date) to (time needed for a human to do a task the model can do).

Here is their rationale for plotting this.

Current frontier AIs are vastly better than humans at text prediction and knowledge tasks. They outperform experts on most exam-style problems for a fraction of the cost. With some task-specific adaptation, they can also serve as useful tools in many applications. And yet the best AI agents are not currently able to carry out substantive projects by themselves or directly substitute for human labor. They are unable to reliably handle even relatively low-skill, computer-based work like remote executive assistance. It is clear that capabilities are increasing very rapidly in some sense, but it is unclear how this corresponds to real-world impact.

It seems that AI alignment research falls into this. The LLMs clearly have enough "expertise" at this point, but doing any kind of good research takes an expert a lot of time, even when it is purely on paper.

It seems therefore that we could use Metr's law to predict when AI will be capable of alignment research. Or at least when it could substantially help.

My question is what time t does "automatically do tasks that humans can do in t" let us do enough research to solve the alignment problem?

(Even if you're not a fan of automating alignment, if we do make it to that point we might as well give it a shot!)
AI #108: Straight Line on a Graph
by Zvi — 2025-03-20T13:50:00.983Z

The x-axis of the graph is time. The y-axis of the graph is the log of ‘how long a software engineering task can AIs reliably succeed at doing.’

The straight line says the answer doubles roughly every 7 months. Yikes.

Upcoming: The comment period on America’s AI strategy is over, so we can finish up by looking at Google’s and MIRI’s and IFP’s proposals, as well as Hollywood’s response to OpenAI and Google’s demands for unlimited uncompensated fair use exceptions from copyright during model training. I’m going to pull that out into its own post so it can be more easily referenced.

There’s also a draft report on frontier model risks from California and it’s… good?

Also upcoming: My take on OpenAI’s new future good-at-writing model.

Table of Contents

Language Models Offer Mundane Utility. I want to, is there an app for that?

Language Models Don’t Offer Mundane Utility. Agents not quite ready yet.

Huh, Upgrades. Anthropic efficiency gains, Google silently adds features.

Seeking Deeply. The PRC gives DeepSeek more attention. That cuts both ways.

Fun With Media Generation. Fun with Gemini 2.0 Image Generation.

Gemma Goals. Hard to know exactly how good it really is.

On Your Marks. Tic-Tac-Toe bench is only now getting properly saturated.

Choose Your Fighter. o3-mini disappoints on Epoch retest on frontier math.

Deepfaketown and Botpocalypse Soon. Don’t yet use the bot, also don’t be the bot.

Copyright Confrontation. Removing watermarks has been a thing for a while.

Get Involved. Anthropic, SaferAI, OpenPhil.

In Other AI News. Sentience leaves everyone confused.

Straight Lines on Graphs. METR finds reliable SWE task length doubling rapidly.

Quiet Speculations. Various versions of takeoff.

California Issues Reasonable Report. I did not expect that.

The Quest for Sane Regulations. Mostly we’re trying to avoid steps backwards.

The Week in Audio. Esban Kran, Stephanie Zhan.

Rhetorical Innovation. Things are not improving.

We’re Not So Different You and I. An actually really cool alignment idea.

Anthropic Warns ASL-3 Approaches. Danger coming. We need better evaluations.

Aligning a Smarter Than Human Intelligence is Difficult. It’s all happening.

People Are Worried About AI Killing Everyone. Killing all other AIs, too.

The Lighter Side. Not exactly next level prompting.

Language Models Offer Mundane Utility

Arnold Kling spends 30 minutes trying to figure o...
What is an alignment tax?
by Vishakha — 2025-03-20T13:06:58.087Z
Link: https://aisafety.info/questions/8AF1/What-is-an-alignment-tax

This is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback.

The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.

The alignment tax is the extra cost of ensuring that an AI system is aligned, relative to the cost of building an unaligned alternative. The term ‘tax' is used metaphorically here: in the AI safety literature, ‘alignment/safety tax’ or ‘alignment cost’ is meant to refer to all the additional costs of alignment including increased developer time, extra compute, or decreased performance, and not only to the financial cost/tax required to build an aligned system.

In order to get a better idea of what the alignment tax is, consider two extreme possibilities.

The best case scenario is

No Tax

. This means we lose nothing from aligning the system, so there is no reason to deploy an AI that is not aligned, so we might as well align it. The worst case scenario is

Max Tax

. This means that alignment is functionally impossible because an aligned system would take forever to develop, require infinite compute, or be completely useless. So you either deploy an unaligned system, or you don’t get any benefit from AI systems at all.

We expect something in between these two scenarios to be the case.

Paul Christiano distinguishes two main approaches to dealing with the alignment tax.

The first is to have the will to pay the tax, i.e. to ensure that the relevant actors such as corporations and governments are willing to pay the extra costs to avoid deploying a system until it is aligned.

The second is to reduce the tax by differentially advancing existing alignable algorithms or by making existing algorithms more alignable. This means, for any potentially unaligned algorithm, ensuring the additional cost for an aligned version of the algorithm is low enough that the developers would be willing to pay it.
Longtermist Implications of the Existence Neutrality Hypothesis
by Maxime Riché — 2025-03-20T12:20:40.661Z

Crossposted on the EA Forum.

We describe the implications the Existence Neutrality Hypothesis[1] could have for impartial longtermists, and then provide quantitative impact estimates for four idealized interventions when accounting or not for this hypothesis. Under this hypothesis, there is little value lost if humanity does not create a Space-Faring Civilization. The three main implications are: (1) To reduce significantly the longtermist value of reducing Extinction -Risks, such as those caused by nuclear and bioweapons. (2) To make some new AI Safety agendas (e.g., “Plan B”) competitive with existing AI Safety agendas. (3) To update the relative priorities between existing AI Safety agendas and possibly render some of them significantly less attractive.

Sequence:

This post is part 9 of a sequence investigating the longtermist implications of alien Space-Faring Civilizations. Each post aims to be standalone. You can find an introduction to the sequence in the following post.

Summary

The Existence Neutrality Hypothesis (ENH) suggests that humanity not developing a Space-Faring Civilization (SFC) results in minimal marginal value loss, as most cosmic resources would be recovered by other civilizations with comparable utility creation efficiency. Assuming the ENH is about 75% correct (combining the Civ-Saturation Hypothesis[2] as 84% correct and the Civ-Similarity Hypothesis[3] as 90% correct), there are three core implications:

Significantly reduce the priority of Extinction-Risk reduction (e.g., nuclear, bioweapon-risks) by about 75%, since fewer cosmic resources and less value would be lost if humanity fails to create a space-faring civilization.
Increase the importance of alternative AI Safety agendas, such as the "Plan B" agenda, which aims at mitigating negative outcomes from misaligned AIs, for example, by reducing the chance that misaligned AIs achieve space colonization.
Shift the main AI safety target from maximizing the joint probability of alignment and humanity creating a space-faring civilization towards optimizing alignment conditional on humanity achieving space colonization. Shifting the focus from reducing Existential-Risks to Alignment-Risks.

We first qualitatively introduce these implications, and then review some questions related to how substantial they are. Finally, we produce a simple quantitative model estimating and illus...