trentmkelly/LessWrong-43k huggingface dataset:
Applying traditional economic thinking to AGI: a trilemma
by Steven Byrnes — 2025-01-13T01:23:00.397Z

Traditional economics thinking has two strong principles, each based on abundant historical data:

Principle (A): No “lump of labor”: If human population goes up, there might be some wage drop in the very short term, because the demand curve for labor slopes down. But in the longer term, people will find new productive things to do, such that human labor will retain high value—in other words, the demand curve will move right. Indeed, if anything, the value of labor will ultimately go up, not down—for example, dense cities are engines of economic growth!
Principle (B): “Experience curves”: If the demand for some product goes up, there might be some price increase in the very short term, because the supply curve slopes up. But in the longer term, people will ramp up manufacturing of that product to catch up with the demand—in other words, the supply curve will move right. Indeed, if anything, the price per unit will ultimately go down, not up, because of economies of scale, R&D, etc.

Now consider Artificial General Intelligence (AGI), i.e. a combination of chips, algorithms, electricity, and teleoperated robots that can autonomously do the kinds of stuff that ambitious human adults can do—stuff like founding and running new companies, research and development, learning and applying new skills, working in collaborative teams, skillfully using teleoperated robots after only a few hours of practice, and so on.

So here’s a question: When we have AGI, what happens to the price of chips, electricity, and teleoperated robots?

(…Assuming free markets, and rule of law, and AGI not taking over and wiping out humanity, and so on. I think those are highly dubious assumptions, but let’s not get into that here!)

Principle (A) has an answer to this question. It says prices will be high. After all, if AGI can really do all the things that ambitious entrepreneurial skilled labor can do, then there will be no “lump of labor” for AGI, any more than there has been for humans. However much AGI there is, it will keep finding new productive things to do. And the prices will reflect that high value. (Incidentally, if that’s true, then it would imply that human labor will retain a well-paying niche—just as less-skilled labor today can still get jobs despite more-skilled labor also existing.)

Principle (B) has a different, contradictory answer to this question. It says prices will be low. ...
Building AI Research Fleets
by Ben Goldhaber — 2025-01-12T18:23:09.682Z

From AI scientist to AI research fleet

Research automation is here (1, 2, 3). We saw it coming and planned ahead, which puts us ahead of most (4, 5, 6). But that foresight also comes with a set of outdated expectations that are holding us back. In particular, research automation is not just about “aligning the first AI scientist”, it’s also about the institution-building problem of coordinating the first AI research fleets.

Research automation is not about developing a plug-and-play “AI scientist”. Transformative technologies are rarely straightforward substitutes for what came before. The industrial revolution was not about creating mechanical craftsmen but about deconstructing craftsmen into assembly lines of specialized, repeatable tasks. Algorithmic trading was not just about creating faster digital traders but about reimagining traders as fleets of bots, quants, engineers, and other specialists. AI-augmented science will not just be about creating AI “scientists.”

Why? New technologies come with new capabilities and limitations. To fully take advantage of the benefits, we have to reshape our workflows around these new limitations. This means that even if AIs eventually surpass human abilities across the board, roles like “researcher” will likely transform dramatically during the transition period.

The bottleneck to automation is not just technological but also institutional. The problem of research automation is not just about training sufficiently capable and aligned models. We face an “institutional overhang” where AI capabilities are outpacing our ability to effectively organize around their weaknesses. Factories had to develop new management techniques, quality control systems, and worker training programs to make assembly lines effective. Trading firms had to build new risk management frameworks, compliance systems, and engineering cultures to succeed at algorithmic trading. So too, research institutions will need to reinvent themselves around AI or fall behind.

The scaling labs have already moved beyond the traditional academic model. Consider the use of matrix management structures where research engineers work across multiple projects, standardized research workflows that enable fast iteration, and cross-cutting infrastructure teams that maintain the computational foundation for research. Labs employ specialized roles like research engine...
Do Antidepressants work? (First Take)
by Jacob Goldsmith — 2025-01-12T17:11:55.417Z

I've been researching the controversy over whether antidepressants truly work or whether they are not superior to a placebo. The latter possibility really contains two possibilities itself: either placebos are effective at treating depression, or the placebo effect reflects mean reversion. Here, the term "antidepressant" refers to drugs classified as SSRIs and SNRIs.

Some stylized facts from the literature of RCTs:

Both antidepressants and placebos are associated with a reduction in symptoms of depression
Antidepressants are associated with a larger reduction in depressive symptoms than placebos
This difference in efficacy is attenuated when "enhanced" placebos are used, that is placebos that are designed to provoke side effects to persuade the subject that they are being treated with a real medication
Studies on antidepressants broadly measure either an average treatment effect according to some continuous scale or a binary treatment according to a cutoff for success/failure of treatment (or both). Universally, if you looked only at the former measure you would conclude that the effect of antidepressants on depression is clinically insignificant, while if you looked only at the latter measure you would come to a more favorable conclusion.

How is depression measured?

Most studies of antidepressant efficacy use the Hamilton Depression Rating Scale (HDRS) to measure changes in depressive symptoms from the baseline, although entrance into the clinical trials themselves is often not based on this measure. There are several versions of this scale, but the most popular one is HDRS-17, which is a seventeen question survey administered by a clinician. Scores on HDRS-17 vary from zero to 57. Scores of 0-7 are generally considered normal, while a score of 20 or greater indicates at least moderate depression.

You can look at the questions on HDRS-17 here. To my eye, it seems to strangely underweight feelings of subjective psychological distress that define depression, and includes a lot of symptoms that are only contingently associated with what we actually mean when we talk about depression. I think the heterogeneity in types of questions is a potential problem in using it to quantitatively assess the effectiveness of interventions on depressive disorder, but I'd like to leave that aside for now for the most part.

What does it mean for antidepressants to work?

Most studies that...
A Novel Idea for Harnessing Magnetic Reconnection as an Energy Source
by  — 2025-01-12T17:11:46.825Z

Introduction

Magnetic reconnection—the sudden rearrangement of magnetic field lines—drives dramatic energy releases in astrophysical and laboratory plasmas. Solar flares, tokamak disruptions, and magnetospheric substorms all hinge on reconnection. Usually, these events are uncontrolled and often destructive. But what if we could systematically harness reconnection here on Earth, funneling that released magnetic energy into an external circuit? This post outlines one speculative way to do so, by algebraically combining Maxwell’s equations with fluid dynamics (i.e. magnetohydrodynamics, MHD) to create a “pulsed MHD power generator.”

1. The Equations We Combine

Maxwell’s Equations (SI units, full form for reference):

(1a) div(E) = rho_e / epsilon_0

(1b) div(B) = 0

(1c) curl(E) = - (partial B / partial t)

(1d) curl(B) = mu_0 * J + mu_0 * epsilon_0 * (partial E / partial t)

Here, E is the electric field, B is the magnetic field, rho_e is electric charge density, and J is current density.

Ohm’s Law in a Plasma (ignoring Hall or other corrections):

(2) J = sigma * [ E + (v x B) ]

where v is the fluid (plasma) velocity and sigma is the electrical conductivity.

Navier–Stokes Momentum Equation (simplified MHD form):

(3) rho * (d v / d t) = - grad(p) + (J x B) + …

where rho is mass density, p is pressure, and the Lorentz force J x B couples electromagnetism and fluid motion.

2. Energy Considerations and Magnetic Reconnection

The energy in the electromagnetic field can be tracked via an equation of the form:

(4) (partial / partial t)[ (B^2)/(2 mu_0) + (epsilon_0 * E^2)/2 ]

+ div( (1/mu_0)*(E x B) )

= - J dot E

On the fluid side, you get kinetic energy terms (1/2 * rho * v^2) evolving via Navier–Stokes. Adding these together yields a unified energy equation showing how power flows between fields and plasma.

Reconnection enters via the induction equation, which is derived by taking curl(E) = -partial B / partial t and plugging in Ohm’s law. In a resistive plasma:

(5) partial B / partial t = curl[ v x B - (1 / (sigma mu_0)) * curl(B) ]

When sigma is very large, B-field lines are “frozen” into the plasma—except in small regions of enhanced resistivity, where they break and reconnect. This can convert magnetic energy into heat, kinetic flows, and strong electric fields.

3. Proposed Concept: Pulsed MHD Power Generator

Basic Device Sket...
How quickly could robots scale up?
by Benjamin_Todd — 2025-01-12T17:01:04.927Z
Link: https://benjamintodd.substack.com/p/how-quickly-could-robots-scale-up

Once robots can do physical jobs, how quickly could they become a significant part of the work force?

Here's a couple of fermi estimates, showing (among other things) that converting car factories might be able to produce 1 billion robots per year in under 5 years.

Nothing too new here if you've been following Carl Shulman for years, but I thought it could be useful to have a reference article. Please let me know about corrections or other ways to improve the estimates.
AGI Will Not Make Labor Worthless
by Maxwell Tabarrok — 2025-01-12T15:09:27.612Z
Link: https://www.maximum-progress.com/p/agi-will-not-make-labor-worthless

The discovery and use of machinery may be … injurious to the labouring class, as some of their number will be thrown out of employment, and population will become redundant.

-

David Ricardo

Fears about human labor getting replaced by machines go back hundreds if not

thousands

of years. These fears continue today in response to the rapid progress of AI in several previously secure human domains.

Rudolf L on the Lesswrong forum

, for example, claims that a world with labor-replacing AI would mean:

People's labour gives them less leverage than ever before.
Achieving outlier success through your labour in most or all areas is now impossible.

Scott Alexander

responded

to this post with some reasons to doubt the implications of complete labor redundancy, but he accepts the premise that AGI will make human labor valueless without contest.

The basic argument in favor of this position is twofold. First, that AI will continue to rapidly automate tasks that humans used to do. Second, is that as a consequence of this automation, there will be more available labor available for a shrinking number of jobs, thus lowering wages until eventually all tasks are produced by AIs and the only way to make income is to own some GPUs, power plants, or land.

The long history of this idea proves that it is convincing but it probably isn’t true. Even if AGI continues to advance and is eventually able to do any task that humans currently do, human labor will not become worthless.

The Labor Share of GDP Has Been Constant For 200+ Years

The labor share of income - the fraction of total output that is paid to workers versus capital owners - has been constant for so long that it has become one of the

fundamental regularities

that any model of long run economic growth must explain. For

at least

200 years

, 50-60% of GDP has gone to pay workers with the rest paid to machines or materials. There is lots of hubbub about falling labor share since the 1990s, but this is due to

mismeasurement of labor income

which has shifted increasingly into non-wage benefits like employer healthcare and retirement plans. Accounting for this, the regularity of labor’s share of income at around 50% remains.

The basic story that motivates fear of AI automation predicts that more automation leads to lower value and leverage for labor, but this story cannot a explain a flat labor share of income since 1800. The past 200+ year pe...
The purposeful drunkard
by Dmitry Vaintrob — 2025-01-12T12:27:51.952Z

I'm behind on a couple of posts I've been planning, but am trying to post something every day if possible. So today I'll post a cached fun piece on overinterpreting a random data phenomenon that's tricked me before.

Recall that a random walk or a "drunkard's walk" (as in the title) is a sequence of vectors x1,x2,… in some Rn such that each xk is obtained from xk−1 by adding noise. Here is a picture of a 1D random walk as a function of time:

Weirdly satisfying.

A random walk is the "null hypothesis" for any ordered collection of data with memory (a.k.a. any "time series" with memory). If you are looking at some learning process that updates state to state with some degree of stochasticity, seeing a random walk means that your update steps are random and you're not in fact learning. If you graph some collection of activations from layer to layer of a transformer (or any neural net architecture with skip connections -- this means that layer updates are added to "memory" from the previous layer) and see a random walk behavior, this means that you have in fact found no pattern in how the data gets processed in different layers. A random walk drifts over time, but ends up in a random place: you can't make any predictions of where it ends up (except predict the exact form of Gaussian randomness):

Time plotted as color.

Now one useful way that people use to find patterns in their data is make pictures. Since data in ML tends to be high-dimensional and we can only make 2-dimensional pictures[1], we tend to try to plot the most meaningful or prediction-relevant data. A standard thing to do is to look at top PCA components: i.e., find the maximal-variance directions and look at how the data behaves in these directions. If we have generated some collection of possible interesting directions that reflect some maximally informative parameters that the internal data of an LLM tracks, we can examine their behavior one by one, but we expect to see the strongest form of the relevant behavior in the top PCA components.

So if the top PCA components of a process with memory look like a random walk, that's even stronger evidence that we haven't found a pattern (at least a linear one). Here is a 5-dimensional random walk plotted to 2 dimensions via its top 2 PCA components:

This still looks pretty random (with a random drift, as before).

This is fun! Let's do the same th...
No one has the ball on 1500 Russian olympiad winners who've received HPMOR
by Mikhail Samin — 2025-01-12T11:43:36.560Z

We have contact details and can send emails to 1500 students and former students who've received hard-cover copies of HPMOR (and possibly Human Compatible and/or The Precipice) because they've won international or Russian olympiads in maths, computer science, physics, biology, or chemistry.

This includes over 60 IMO and IOI medalists.

This is a pool of potentially extremely talented people, many of whom have read HPMOR.

I don't have the time to do anything with them, and people in the Russian-speaking EA community are all busy with other things.

The only thing that ever happened was an email sent to some kids still in high school about the Atlas Fellowship, and a couple of them became fellows.

I think it could be very valuable to alignment-pill these people. I think for most researchers who understand AI x-risk well enough and can speak Russian, even manually going through IMO and IOI medalists, sending those who seem more promising a tl;dr of x-risk and offering to schedule a call would be a marginally better use of their time than most technical alignment research they could be otherwise doing, because it plausibly adds highly capable researchers.

If you understand AI x-risk, are smart, have good epistemics, speak Russian, and want to have that ball, please DM me on LW or elsewhere.

To everyone else, feel free to make suggestions.
Why modelling multi-objective homeostasis is essential for AI alignment (and how it helps with AI safety as well)
by Roland Pihlakas — 2025-01-12T03:37:59.692Z

I notice that there has been very little if any discussion on why and how considering homeostasis is significant, even essential for AI alignment and safety. Current post aims to begin amending that situation. In this post I will treat alignment and safety as explicitly separate subjects, which both benefit from homeostatic approaches.

This text is a distillation and reorganisation of three of my older blog posts at Medium:

Making AI less dangerous: Using homeostasis-based goal structures

(2017)

Project proposal: Corrigibility and interruptibility of homeostasis based agents

(2018)

Diminishing returns and conjunctive goals: Mitigating Goodhart’s law with common sense. Towards corrigibility and interruptibility via the golden middle way.

(2018)

I will probably share more such distillations or weaves of my old writings in the future.

Introduction

Much of AI safety discussion revolves around the potential dangers posed by goal-driven artificial agents. In many of these discussions, the agent is assumed to maximise some utility metric over an unbounded timeframe. This simplification, while mathematically convenient, can yield pathological outcomes. A classic example is the so-called “paperclip maximiser”, a “utility monster” which steamrolls over other objectives to pursue a single goal (e.g. creating as many paperclips as possible) indefinitely. “Specification gaming”, Goodhart’s law, and even “instrumental convergence” are also closely related phenomena.

However, in nature, organisms do not typically behave like pure maximisers. Instead, they operate under homeostasis: a principle of maintaining various internal and external variables (e.g. temperature, hunger, social interactions) within certain “good enough” ranges. Going far beyond those ranges — too hot, too hungry, too socially isolated — leads to dire consequences, so an organism continually balances multiple needs. Crucially, “too much of a good thing” is just as dangerous as too little. Excess is harmful even for the very same objective that was maximised for, not just as a side effect on other objectives. This seems to apply to most or even all biological objectives.

In this post, I argue that an explicitly homeostatic, multi-objective model is a more suitable paradigm for AI alignment. Moreover, correctly modelling homeostasis increases AI safety, because homeosta...
Extending control evaluations to non-scheming threats
by joshc — 2025-01-12T01:42:54.614Z

Buck Shlegeris and Ryan Greenblatt originally motivated control evaluations

as a way to mitigate risks from ‘scheming’ AI models: models that consistently pursue power-seeking goals in a covert way; however, many adversarial model psychologies are not well described by the standard notion of scheming. For example, models might display

scheming-like behavior

inconsistently or acquire harmful goals through data poisoning, jailbreaks, etc.[1]

This post suggests a generalization of control evaluations that addresses these non-scheming threats.

The high-level change I propose is to define a control evaluation as a method of bounding risk from worst-case deployment behavior generally – rather than scheming models in particular. This change implies two modifications to the control evaluation methodology:

Expand the definition of an “untrusted” model to include non-scheming models:

In control evaluations described in previous work, the red team constructs adversarial substitutes for models that might be scheming, which are called ‘untrusted models.’ I propose allowing the red team to construct substitutes for any model that might exhibit drastically more dangerous behavior in deployment than what is observed in behavioral testing. Scheming is a classic example of such a model because schemers hide misalignment from developers; however, models might also be imperceptibly dangerous for other reasons, such as because they might be more likely to be jailbroken in deployment than in evaluations, models might rarely adopt misaligned goals, they might be backdoored in training, etc. Any model with dangerous behavior that it would otherwise not display in the evaluation should be replaced with a conservative substitute to simulate how it might pessimistically behave in deployment.
Account for differences in the plausible attacks that different kinds of untrusted models might execute. If developers wish to precisely estimate the risk posed by untrusted models, they will likely need to account for differences in the strategies plausibly pursued by a scheming model, a rarely misaligned model, data poisoned model, etc. For example, data-poisoned models might be directly trained to execute attacks and given privileged information about control measures. Therefore, data-poisoned models might execute more effective strategies than sche

...