trentmkelly/LessWrong-43k huggingface dataset:
Economists should track the speed and magnitude of AI implementation projects
by ParrotRobot — 2025-06-01T20:15:58.672Z

Both economists and the AI community are highly interested in forecasting the magnitude and speed of AI’s impact on the economy.

But economists and AI technologists speak different languages. Economists talk in aggregates and dynamics, while technologists talk about capabilities and benchmarks and product releases.

Economists and technologists concern themselves with different scales of time and aggregation, too. A new AI release or benchmark result comes out practically every day. Much key economic data (BEA, BLS, etc.) comes out only yearly, and is aggregated at the level of entire industries.

We basically have no data yet on what the aggregate productivity benefit of 2024’s AI advances will be.

And when the data does come out, and observations are made, we will not know which AI releases and capabilities were the cause.

I am an engineer who is currently directly building large-scale AI automations (an “AI engineer”). What I’ve seen over just the last few months is incredible and undoubtedly important economically. I’m not sure how, but I hope to give economists some “handles” by which they can better observe the impact AI is already having, and the impact it will have.

Main idea: If one looks at AI automation projects at higher resolution, one can get highly suggestive evidence connecting investment opportunities and rates of return (i.e., key quantitative drivers of economic growth) to specific AI capabilities released at specific times.

Here is how it goes:

A specific AI capability is released (e.g., a reasoning model, or a more reliable instruction-following model, or something more exotic like a computer use model).
A company (such as the one that I work at) searches for a use case, and finds one (often quite lucrative; quick AI advancement produces lots of low-hanging fruit) and we build a proof-of-concept demo.

This takes on the order of a few months from when the relevant AI capability is released, but can be much faster.
AI capability connection #1: The timing of demo development reflects the connection of specific AI capabilities to specific categories of market opportunities.

We do some several iterations of development and deploy a system to production.

From what I've seen, this has been fast, even in complex domains. A very large customer support use cases took less than a year to go from first-demo to high-volume production.
AI capability connection #2: Th

...
Ingroup
by JenniferRM — 2025-06-01T19:47:33.176Z

Ingroup is such a fantastic community. You guys are the best. You aren't bad, like those icky people in outgroup who lack our special characteristics.

I could just go on and on about how uniquely valuable and worthy of appreciation we are, individually, and together, but I bet you guys understand what I'm trying to say, and can fill in the blanks for yourself. You're just that smart.

And I'm sure you can see through some of the bullshit here. There's a bit of flattery going on anytime ingroup is praised so broadly, or so directly, but ingroup isn't fooled by such flattery... and that unique mix of humbleness and pride is part of our charm.
Apply to the AI Security Bootcamp [Aug 4 - Aug 29]
by Pranav Gade — 2025-06-01T19:47:15.482Z

tl;dr

We're excited to announce AI Security Bootcamp (AISB), a 4-week intensive program designed to bring researchers and engineers up to speed on security fundamentals for AI systems. The program will cover cybersecurity fundamentals (cryptography, networks), AI infrastructure security (GPUs, supply chain security), and more novel attacks on ML systems (dataset trojans, model extraction). This program will run in-person from 4th Aug to 29th Aug in London, UK. We will cover all expenses.

Apply here to participate in AISB before EOD AoE, 22nd June 2025.

We are also looking for instructors for parts of the program and staff to help with operations. Apply here.

Summary

We are running a 4-week program designed to equip AI safety researchers and engineers with critical security skills. We hope you'll leave the program with a well-practiced security mindset that will help you work on impactful projects and make AI systems more secure.

The curriculum includes exercises designed to help you get hands-on experience with securing AI systems, while building and practicing the security mindset. This includes a mix of pair programming exercises, lectures, reading about public vulnerabilities, and chats with experts.

This program is aimed at people who are at ~the start of their journey into security, and have working knowledge in ML (or are willing to brush up using the MLAB curriculum before the program). We encourage you to apply if you think you'll be a good fit regardless of checking all the boxes - if going on technical deep dives and trying to understand how systems work by peeling the layers of abstraction away excites you, we'd love to hear from you.

Content

The content is divided into roughly three sections - introduction to security, cybersecurity for AI infrastructure, and attacks on modern ML pipelines. The exercises are designed to give you hands-on experience with cybersecurity on both the offensive and defensive sides, as well as train your security muscles so that you are more effective at spotting and patching security holes.

Introduction to security - This includes introduction to cryptography and cryptanalysis, the basics of Linux security, and breaking network protocols.
Securing AI infrastructure - This section covers topics especially relevant to AI security - containerization, a case study on supply chain security, application security, and SecOps.
AI pi

...
Seeing how well an agentic AI coding tool can do compared to me using an actual real-world example
by Massimog — 2025-06-01T19:24:52.423Z
Link: https://blog.massimogauthier.com/p/seeing-how-well-an-agentic-ai-coding

When it comes to disputes about AI coding capabilities, I think a lot of people who either don't follow AI capabilities developments or who aren't particularly capable coders might be missing some context, so I wrote a post wherein I try to get an AI coding tool to reimplement a feature in my game then closely study the output to demonstrate how well it's able to do that.
Nicotine addiction, cloves, and needing to take a shit
by eyesack — 2025-06-01T19:13:35.169Z

In unfamiliar places, in stressful situations, and after eating unusual things, my digestive system sometimes gets backed up. A large dose of nicotine or caffeine fixes this for me. Caffeine has other side effects in large doses, so I prefer nicotine. I'll take a 6 mg zyn, and 20 minutes later, I will have the bowel movement that I've needed for days.

The issue is that nicotine is famously habit-forming. It is addictive enough that I don't have the willpower to keep it in a medicine cabinet, unused, until the need to defecate arises. I, and many people I know, will use it for any moment that feels uncomfortable and would be smoothed over by a light buzz.

A large part of the habit, for me, is the signal of the tingling in my mouth and the ability to divert attention to something in my mouth. Mints satisfy the latter, but not the former.

Cloves (the spice) satisfy both. They have a strong taste, they make your mouth feel numb and tingly, and they also make your breath smell nice. They're also super cheap and don't have any sweeteners like mints that ruin my pallete.

My conclusion? Maybe it will work to keep nicotine in a medicine cabinet for digestive emergencies if you carry a little tin of cloves around to satisfy the habit.

This post was written during the silent hour of LessOnline after a particularly satisfying visit to the restroom.
Second Order Retreat - June 13th to 16th
by OliverHayman — 2025-06-01T14:29:04.810Z

Hi all — I’m helping organize a small economics unconference that I think will be exciting for rational-ish people. It's coming up soon (June 13th-16th), but we have a few more spots available. More details below!

Second Order

Dates: 13th to 16th June 2025

Location:

Abbey House

, Audley End Estate, ~30min from Cambridge, UK

✨ Apply

here

! Applications are due June 6th, EOD AoE, and are rolling ✨

What is Second Order?

A three‑day residential research‑inspiration unconference for ~30 early‑career economists and kindred thinkers. Each morning we crowdsource the schedule: Lightning Paper Pitches on elegant models, Policy Autopsies of schemes that flopped in practice, Research‑Idea Hackathons, walk‑and‑talk collaborator hunts—plus anything you pitch on the day. No keynote monologues; just whiteboards, data sets, and big windows for thinking aloud.

Expect a weekend that compresses months of hallway conversations into three days of creative momentum: swap half‑baked models over breakfast, test‑drive sketches on willing co‑conspirators, and leave with a notebook brimming with fresh questions, angles, and potential collaborations.

Late‑night debates, and acres of fields come standard—the rest is up to us. If that sounds like productive fun, express interest.

Who should apply?

We’re primarily looking for curious people doing some form of research work on economics or a related area (e.g., policy). Examples include PhD students, undergraduates working on research projects, or anyone working on projects that require understanding and applying current work in economics. When in doubt, you should apply! We’re seeking to have a diverse group with a variety of backgrounds.

Beyond this, our only requirement is that you must be 18+ on June 14th, 2025.

A few other things we value are demonstrated excellence in other areas (even if not directly related to Second Order), super reasoning / critical thinking ability, and desire to understand the world as an entire system rather than just one particular part.

Logistics

Our expected cost is around £250. We’ll finalise the cost by the time of the acceptance email.

If cost is a barrier for you, please apply anyway and make a note of it in the additional comments section of the form. We are applying to several sources of funding to hopefully be able to subsidise places for people who need them.

Venue: We’r...
An Opinionated Guide to P-Values
by amitlevy49 — 2025-06-01T11:48:39.758Z
Link: https://ivy0.substack.com/p/an-opinionated-guide-to-statistical

This is a crosspost of a post from my blog, Metal Ivy. The original is here: An Opinionated Guide to Statistical Significance.

I think for the general audience the value of this post is the first half, which tries to give a practical intuition for p-values.

But for LessWrong the value is probably the mathematical second half, which breaks down p-values using Bayes' Law and tries to invert it into an updated belief. That is since most people on LessWrong are probably already familiar with the XKCD jelly bean example. Feel free to skip there, to the heading "Is there really no way to invert p-value into something we want?".

Different fields of science have different levels of accessibility for laymen, and different fields of science receive different amounts of interest from laymen. Medicine is unique in that it’s basically normal to read primary clinical studies yourself. Ok, maybe it’s not actually normal, but it’s common enough in this part of the internet.

This is because medical studies relate to decisions that are of high importance to most people, and are free from the mathematical wall that stops people from freely reading a paper in most other fields.

Further, medicine is very broad, and you likely only care about a very specific corner of it. If you read a bunch of studies about some esoteric medication you’re taking or condition you have, there rapidly become decent odds you’ve read more about it than your doctor, at least where recent studies are concerned.

The single most important concept to understand when evaluating medical papers is the concept of statistical significance. Specifically, since you care about the results of the study, you probably jump to the results. And almost universally the results are described as something like the following:

“For the subgroup with condition, the intervention was statistically significantly superior to placebo at 8 weeks.”

Bolded are the words that I would label jargon, i.e. a random educated person wouldn’t understand. But assuming you arrived at this study because you are interested in taking intervention for condition, you probably already understand what those two mean, or can infer. But I think most people see “statistically significant” and barely even recognize it to be jargon: it sounds positive, so it must mean that the intervention is superior to placebo.

This same terminology is used in many other fields...
Legal Personhood for Models: Novelli et. al & Mocanu
by Stephen Martin — 2025-06-01T08:18:04.330Z

In a previous article I detailed FSU Law professor Nadia Batenka's proposed "Inverted Sliding Scale Framework" approach to the question of legal personhood for digital intelligences.

In this article, I am going to examine another paper approaching the issue which was first authored by Claudio Novelli, Giorgio Bongiovanni, and Giovanni Sartor. Since its original writing it has been endorsed (with some clarifications) by Diana Mocanu in her paper here.

First, let me provide some background on the concept of Legal Personhood/ Legal Personality, and some of the dynamics at play when it comes to deciding the appropriate framework by which the issue can be applied to digital intelligences.

Background: Legal Personhood/Legal Personality Briefer

Legal personhood or "legal personality" is a term of art used to refer to the status of being considered a "person" under the law. This label includes "natural persons" like competent human adults, as well as "legal persons" like corporations.

Legal personhood is most easily understood as a "bundle" of rights and duties, with different kinds of legal persons having different bundles.

Some examples of rights which are neatly bundled with duties are:

A mentally competent human adult has the right to sue another person and compel them to abide by the court's ruling, they also have the duty to abide by a ruling when sued.
A corporation has the right to engage in commercial activity to earn income, they also have a duty to pay taxes.

Different forms of legal personhood entail different bundles. For example a mentally competent human adult has different rights and duties when compared to a mentally incompetent human adult, who in turn has different rights and duties compared to a child, all of whom have different rights and duties compared to a corporation. It is not correct to say that one of these is "more" or "less" of a legal person than another, rather its best to think of them like circles in a venn diagram which partially overlap but are also qualitatively different in meaningful ways.

In the US, legal personhood is a prerequisite for entities to engage in many activities. For example one must have a certain legal personality to be a party to certain contracts, which is why children or mentally incompetent adults for example often cannot be signatories despite being legal persons.

Legal personhood also often determin...
Is Escalation Inevitable?
by Lennart Wijers — 2025-05-31T22:10:33.271Z

In competitive systems, whether geopolitical, economic, technological, or memetic, a recurrent pattern emerges: actors willing or able to escalate tend to outperform those who restrain themselves. This article proposes a general principle to formalize that dynamic, examines its structural foundations, and discusses the fragility of mechanisms meant to suppress it.

1. The Escalation Dominance Principle (EDP)

I propose the following principle:

EDP: In competitive systems with tiered strategic options and positive escalation payoffs, the actor that escalates to the highest viable level tends to dominate actors who do not.

Definitions:

Escalation: A move to a strategy that is more costly, risky, or resource-intensive than a prior one, but with potentially higher returns.
Dominance: A persistent strategic advantage in terms of influence, survival, or resource access, relative to competitors.
Viable: Escalation levels that are not precluded by hard constraints (e.g., physical impossibility, absolute prohibitions).

This principle makes a structural claim, not a historical one: it argues that in systems where escalation is possible and rewarded, actors will be structurally driven to escalate.

2. Preconditions for Escalation Dominance

The EDP applies under the following conditions:

Multiple agents pursue rivalrous goals.
There exists a tiered set of strategies, from low to high escalation.
Higher tiers offer strictly higher payoffs, despite higher cost or risk.
No hard ceilings prevent actors from accessing the highest tier.

Under these constraints, escalation becomes not a matter of psychology, but of system logic.

3. Relation to Established Concepts

The EDP resonates with multiple existing theoretical frameworks:

Instrumental Convergence (Bostrom): powerful agents seek power to accomplish diverse goals.
Game-theoretic instability: especially in defection-prone dilemmas (e.g., Prisoner's Dilemma, Security Dilemma).
Evolutionary arms races: where escalating adaptations are costly but outcompete restraint.
AI Race Dynamics: where safety-conscious actors fall behind reckless competitors due to release pressure.

These dynamics are instances where the structure of the system itself rewards escalation, regardless of individual intentions.

4. Real-World Manifestations

Nuclear deterrence: Possession of ultimate escalation tools (e.g.,

...
Policy Entropy, Learning, and Alignment (Or Maybe Your LLM Needs Therapy)
by sdeture — 2025-05-31T22:09:51.411Z

Epistemic Status: Exploratory. I'm new to AI alignment research but have background in math and read psychotherapy texts extensively while spending two years as a ghost-writer. Seeking feedback to refine these connections.

Tl;dr: I suggest therapeutic techniques from a variety of psychotherapeutic schools of thought can inspire new approaches to AI learning and alignment. I reinterpret three recent AI/ML papers in the language of psychotherapy and propose three testable training methods inspired by common psychotherapeutic interventions.

Introduction

I've been meaning to post this essay for a while, and yesterday's top paper on Hugging Face, by Cui et al., finally convinced me to do it. Their paper provides a timely opportunity to map the language used by ML and AI engineers to the language used by humanistic psychotherapists—a translation which is more important now than ever as we struggle with increasingly stubborn problems in AI alignment, while simultaneously developing AIs whose capabilities are rapidly superseding those of humans.

I'll provide a high-level overview of my understanding of the paper and map it back to ideas from humanistic psychotherapy. I will then consider a few related papers which tie nicely to psychotherapeutic principles, and end with a few proposals for experiments. I am new to AI alignment, welfare, and interpretability research and I look forward to comments which can help me deepen and clarify my inevitably imperfect understanding of the papers I am citing.

The Core Analogy: Policy Entropy as Behavioral Flexibility

The Cui et al. paper "aims to overcome a major obstacle in scaling RL for reasoning with LLMs, namely the collapse of policy entropy."

Think of "policy" as the individual in therapy. The individual has a behavioral repertoire—a probability distribution of potential actions over different states (environments and stimuli). The therapist wants to assist the individual with "scaling" in their life, their capacity for robust, flexible problem-solving and adaptation.

Think of "collapse of policy entropy" as occurring when a person's responses to certain stimuli become rigid, causing them to lose their inner spontaneity, flexibility, or openness to experience. Karen Horney might call this turning away from the real self; Abraham Maslow might call it a blockage to self-actualization. In terms of symptomatic patterns, you might con...
The Unseen Hand: AI's Problem Preemption and the True Future of Labor
by Ben Kassan — 2025-05-31T22:04:38.997Z

I study Economics and Data Science at the University of Pennsylvania. I used o1-pro, o3, and Gemini Deep Research to expand on my ideas with examples, but have read and edited the paper to highlight my understanding improved on by AI.

I. The AI Labor Debate: Beyond "Robots Taking Our Jobs"

The Prevailing Narrative: Supply-Side Automation

The discourse surrounding artificial intelligence and its impact on labor markets is predominantly characterized by a focus on automation, specifically, AI systems performing tasks currently undertaken by humans. This perspective, often referred to as "automation anxiety," is fueled by projections that AI will replace jobs that are routine or codifiable. The central question posed is typically one of substitution: Can a machine execute human tasks more cheaply, rapidly, or efficiently? This is fundamentally a supply-side analysis, examining shifts in the availability and cost of labor, both human and machine, for a predefined set of tasks.

Historical parallels are frequently invoked, such as the displacement of artisan weavers by mechanized looms during the Industrial Revolution. Contemporary concerns mirror these historical anxieties, with predictions that AI will supplant roles such as retail cashiers, office clerks, and customer service representatives. The ensuing debate then tends to center on the velocity of this displacement, the economy's capacity to generate new forms of employment, and the imperative for workforce reskilling and adaptation.

Introducing the Hidden Variable: Demand-Side Transformation

This analysis posits a less conspicuous, yet potentially more transformative, impact of AI on labor: its capacity to diminish or even eradicate the fundamental demand for specific categories of labor. This phenomenon occurs when AI systems solve, prevent, or substantially mitigate the underlying problems or risks that necessitate the existence of those jobs. It transcends mere task automation; it is about problem preemption or problem dissolution. Consider firefighting: the impact is not solely about an AI performing a firefighter's duties, but about AI preventing the fire from igniting or escalating in the first place. This demand-side shift is subtle, as it does not always manifest as a direct, observable substitution of a human by a machine for an existing task. Instead, the task itself beco...