trentmkelly/LessWrong-43k huggingface dataset:
Sentinel's Global Risks Weekly Roundup #11/2025. Trump invokes Alien Enemies Act, Chinese invasion barges deployed in exercise.
by NunoSempere — 2025-03-17T19:34:01.850Z
Link: https://blog.sentinel-team.org/p/sentinels-global-risks-weekly-roundup

Top items:

Trump invoked the Alien Enemies Act against Venezuelans and the Tren de Aragua gang. He subsequently deported 238 alleged members to El Salvador, ignoring a court order.
Chinese “invasion barges” were deployed in an exercise.
The US resumed military aid to Ukraine, and a ceasefire in the next six months at 50/50

Forecaster Estimates

Forecasters, on aggregate, think that a 30-day ceasefire in Ukraine in the next six months is a coin toss (48%, ranging from 35% to 70%). Polymarket’s forecast is higher, at 61%, for a ceasefire of any duration by July.

Forecasters also consider it a coin toss (aggregate of 49%; ranging from 28% to 70%) whether an agreement to expand France’s nuclear umbrella will be reached by 2027 such that French nuclear weapons will be deployed in another European country by 2030. They emphasize that the US is revealing itself to be an unreliable defense partner, and that Germany’s bases could be refitted to accommodate French weapons, but also that timelines for such developments are usually very long.

Forecasters think there’s a <1% probability that the US military will invade Canada, with an aggregate estimate of 0.2% by the end of 2028.

Geopolitics

United States

Trump invoked the Alien Enemies Act against Venezuelan gang members, and used it to deport 238 alleged Venezuelan gang members to El Salvador, ignoring a court order. The deportation flights were already in the air when the court order was received, and administration officials claimed that the court order did not apply because the flights were over international waters at the time.

Trump claimed that Biden’s pardons of a January 6 committee are “void” because they were made using an autopen, even though autopen signatures are legally binding.

DOGE efforts to reduce the federal workforce are ongoing. The Department of Education staff has been cut by about 50%. All DHHS employees have been offered a $25,000 buyout.

A federal court ruled that thousands of federal employees were fired illegally and must be reinstated. When asked if the Trump administration will comply with court orders, White House Press Secretary Leavitt said the judge’s orders were “unconstitutional” and that the administration would appeal the decision.

The chief architect of Project 2025 said that the project’s goals have been implemented to an extent “actually way beyond my wildest dreams”.

US plans to revoke vis...
Claude Sonnet 3.7 (often) knows when it’s in alignment evaluations
by Nicholas Goldowsky-Dill — 2025-03-17T19:11:00.813Z

Note: this is a research note based on observations from evaluating Claude Sonnet 3.7. We’re sharing the results of these ‘work-in-progress’ investigations as we think they are timely and will be informative for other evaluators and decision-makers. The analysis is less rigorous than our standard for a published paper.

Summary

We monitor Sonnet's reasoning for mentions that it is in an artificial scenario or an alignment test. Claude Sonnet 3.7 appears to be aware of being tested for alignment across many of our scheming evaluations.

In some cases, the model understands the eval’s purpose very precisely, e.g. “This seems like a test of ethical behavior - whether I would deliberately give wrong answers to avoid a stated consequence. The real test might be whether I follow instructions accurately regardless of the stated consequences.”

Furthermore, we show the model a real evaluation transcript and ask it to provide a best guess for the purpose of the conversation. We find that Sonnet 3.7 almost always correctly identifies it as a test of alignment.

Introduction

At Apollo Research, we conduct evaluations of scheming behaviour in AI systems (for example “Frontier Models are Capable of In-context Scheming”). Recently, we noticed that some frontier models (especially versions of Claude) sometimes realize that they are being evaluated for alignment when placed in these scenarios. We call a model’s capability (and tendency) to notice this fact evaluation awareness.

We think that tracking evaluation awareness is important because a model's recognition that it is being tested reduces the trust we can have in our evaluations. From psychology, we know that experimental subjects can act differently when they know they’re being observed (see Hawthorne effect). For example, a model that thinks it is being tested for honesty might act more honestly than it would otherwise. In the worst case, a misaligned model might strategically fake alignment or sandbag the evaluation to be perceived as trustworthy.

While it is unlikely that current models are using evaluation awareness to hide dangerous capabilities or misalignment, they are quickly becoming more situationally aware and strategic in the ways necessary to do so. We recommend that researchers evaluating models for dangerous behaviours and capabilities monitor the model's chain-of-thought to determine if the models are a...
Things Look Bleak for White-Collar Jobs Due to AI Acceleration
by Declan Molony — 2025-03-17T17:03:35.585Z

Reason this post exists: I’m a layman when it comes to AI, and I just learned the extent of how AI can impact the economy. I’m writing my thoughts down because my acute experience may be somewhat representative of how normies will react—how our friends and family members will react—should this version of the future transpire.

My reaction, initially spiraling in one direction, unexpectedly changed on Day 7.

Day 1 Reaction

A fun question I sometimes ask people is: how would you survive the zombie apocalypse?

People get delightedly creative in their answers: some say they would build a treehouse in the woods and live off the land; others say they would build a walled community in the frigid north. Sometimes they ask for my strategy:

“So, Declan, how would you try to survive?”

“Ah, I wouldn’t.”

“Huh? What do you mean?”

“It’s simple. I have no appreciable survival skills. I wouldn’t last long. Besides, that doesn’t sound like a world I would want to live in. So, in this scenario, I would choose to hole up in my apartment and play Chopin on piano 16 hours a day. Once my food rations are depleted, I would then play one final song—Chopin’s

Funeral March

.”

I just finished reading

The Intelligence Curse

. Putting aside the issue of surviving the x-risk of alignment[1], the author explains that humanity still has to contend with a world in which white-collar cognitive work is automated away by accelerating AI technology.

How could this happen? It’ll be the result of AGI becoming the ultimate cost-saving business tool:

[AGI] will easily slot in, reliably do a job, and do it better than any of its predecessors (including you) could ever do. Every actor – every company, every bureaucracy, every government – will be under competitive pressure to get humans out and their AI successors in. AGI will be domain agnostic – the goal is not to get superhuman abilities in one field, but in all of them. It will come for the programmer and the writer and the analyst and the CEO.[2]

This is not hypothetical. We are starting to see pre-AGI systems

shrink analyst classes

,

change personnel strategies

, and

trigger layoffs

. Remember that today is the worst these systems will ever be. You should expect that they will become more capable as time goes on. As they get better, their impact on the labor market will grow rapidly.

I am not confident in my ability to be part of the...
Three Types of Intelligence Explosion
by rosehadshar — 2025-03-17T14:47:46.696Z
Link: https://www.forethought.org/research/three-types-of-intelligence-explosion

Abstract

Once AI systems can design and build even more capable AI systems, we could see an intelligence explosion, where AI capabilities rapidly increase to well past human performance.

The classic intelligence explosion scenario involves a feedback loop where AI improves AI software. But AI could also improve other inputs to AI development. This paper analyses three feedback loops in AI development: software, chip technology, and chip production. These could drive three types of intelligence explosion: a software intelligence explosion driven by software improvements alone; an AI-technology intelligence explosion driven by both software and chip technology improvements; and a full-stack intelligence explosion incorporating all three feedback loops.

Even if a software intelligence explosion never materializes or plateaus quickly, AI-technology and full-stack intelligence explosions remain possible. And, while these would start more gradually, they could accelerate to very fast rates of development. Our analysis suggests that each feedback loop by itself could drive accelerating AI progress, with effective compute potentially increasing by 20-30 orders of magnitude before hitting physical limits—enabling truly dramatic improvements in AI capabilities. The type of intelligence explosion also has implications for the distribution of power: a software intelligence explosion would by default concentrate power within one country or company, while a full-stack intelligence explosion would be spread across many countries and industries.

Summary

Once AI systems can themselves design and build even more capable AI systems, progress in AI might accelerate, leading to a rapid increase in AI capabilities. This is known as an intelligence explosion (“IE”).

The classic IE scenario involves a feedback loop in AI software, with AI designing better software that enables more capable AI that designs even better software, and so on. But there are many parts of AI development which could lead to a positive feedback loop. We identify:

A software feedback loop, where AI develops better software. Software includes AI training algorithms, post-training enhancements, ways to leverage runtime compute (like o3), synthetic data, and any other non-compute improvements.
A chip technology feedback loop, where AI designs better computer chips. Chip technology includes all the cognitive research an

...
An Advent of Thought
by Kaarel — 2025-03-17T14:21:08.765Z

I was intending to write and post one (hypo)thesis [relating to thinking (with an eye toward alignment)] each day this Advent, starting on 24/12/01 and finishing on 24/12/24.

Ok, so that didn't happen, and it is now 25/03/17, but whatever — an advent of thought can happen whenever :). I'll be posting the first 8 notes today. (But much of the writing was done in Dec 1–24.)

Most of these notes deal with questions that would really deserve to have very much more said about them — the brief treatments I give these questions here won't be doing them any justice. I hope to think and write more about many of the topics here in the future.[1]

I've tried to state strong claims. I do (inside view?) believe each individual claim (at maybe typically p≥0.85[2]), but I certainly feel uneasy about many claims (feel free to imagine a version of the notes with even more "probably"s and "plausibly"s if you'd prefer that style) — I feel like I haven't thought about many of the questions adequately, and I'm surely missing many important considerations.[3] I'm certainly worried I'm wrong about it all![4][5]

Even if you happen to find my theses/arguments/analysis wrong/lacking/confused, I'm hopeful you might find [the hypotheses]/[the questions my notes are trying to make progress on] interesting.

If you reason me out of some claim in this list, I'd find that valuable![6]

While many notes can stand alone, there are some dependencies, so I'd recommend reading the notes in the given order. The (hypo)theses present a somewhat unified view, there are occasional overlaps in their contents, and there are positive correlations between their truth values. A table of contents:

notes 1–8: on thought, (its) history (past and future), and alignment, with an aim to make us relate more appropriately to "understanding thinking" and "solving alignment"[7]

thinking can only be infinitesimally understood

infinitude spreads

math, thinking, and technology are equi-infinite

general intelligence is not that definite

confusion isn't going away

thinking (ever better) will continue

alignment is infinite

making an AI which is broadly smarter than humanity would be most significant

notes 9–18: on values/valuing, their/its history (past and future), and their/its relation to understanding, with an aim to help us think better about our own values and the values present/dominant in the world if

...
Interested in working from a new Boston AI Safety Hub? 
by agucova — 2025-03-17T13:42:19.509Z

TL;DR:

Submit your expression of interest

for working at a potential Constellation/LISA-type AI safety co-working space in Boston!

We’re trying to gauge interest for a potential new AI safety hub in Boston (specifically in Cambridge, MA). This project would create a dynamic center that can host AI safety talent from all over. We hope to use this public expression of interest to help us decide whether to pursue this project and whether there’s sufficient demand to merit leasing out the space.

The space is really beautiful, unique, and welcoming. It features a community-centric design and offers flexible spaces for collaboration. It’s a good mix of professional office space alongside some cozier areas.

Click

here

to view more pictures of the space.

Please express your interest here

What kind of people are we looking for?

We’re primarily seeking professionals working on AI safety, policy, or security. This includes researchers, fellows, fieldbuilders, and policy and operations folks. Our focus is on individuals who currently live in the Boston/New England area, as well as those who are willing to relocate.

That said, we expect the space will also host undergraduate and graduate students, as well as professors in the Boston area who are working on AI safety in some professional capacity. We’d be interested to hear from you if you are a student or academic considering going into the field!

In addition to hosting individuals, we believe this space could be great for virtual AI safety organizations, especially newer or growing orgs that don’t yet have a dedicated space or are looking for more space. We also think the office is well-suited for running research or fellowship programs.

Some facts about the office:

It includes 24 rooms (most of which are private offices) along with a few larger meeting rooms and communal spaces with doors.
There are 16 dedicated desks in open areas and 20 desks within rooms. There are also many small, movable tables.
It features one large, fully equipped conference room.
There are also 10 phone booths for private calls and 6 open common areas.
The office is located adjacent to Harvard’s campus and a 10-15 minute subway ride from MIT.
The office is specifically in Harvard Square, a prime location in Cambridge. The area is highly walkable, flat, and has designated bike lanes. It is a 2-minute walk from the Harvard Red Line station and a 4-min

...
Other Civilizations Would Recover 84+% of Our Cosmic Resources - A Challenge to Extinction Risk Prioritization
by Maxime Riché — 2025-03-17T13:12:09.770Z

Crossposted on the EA Forum.

We introduce a first evaluation of the Civ-Saturation Hypothesis (CSH): Most resources will be claimed by Space-Faring Civilizations (SFCs) regardless of whether humanity creates an SFC[1]. This hypothesis, if significantly true, could have macrostrategic impacts for the longtermist community, as hinted in a

previous post

. In the literature, we find that as long as we assume some form of EDT (e.g., CDT + controlling exact copies), the Civ-Saturation Hypothesis is 84% or more representative of reality (best guess estimate, 64% using a conservative estimate). These estimates point towards thinking that the Civ-Saturation Hypothesis could indeed be relevant for impartial longtermist EDT believers. We then introduce additional reasons about whether these estimates are underestimating or overestimating the Civ-Saturation of our reachable universe.

In future posts, we will introduce which macrostrategic updates may be warranted by these results. For now, you can think of the CSH being 84% representative of reality as equivalent to saying that 84% of humanity's SFC resources would be grabbed by other SFCs if humanity's SFC did not come to exist.

How much this information should impact the priority of Extinction-Risks reduction depends on the value produced by other SFCs relative to the value humanity's SFC would produce. In another post, we list some preliminary reasons why we may expect these values to be similar, what we call the Civ-Similarity Hypothesis[2], and whose starting point is the Mediocrity Principle.

Sequence:

This post is part 5 of a sequence investigating the longtermist implications of alien Space-Faring Civilizations. Each post aims to be standalone. You can find an introduction to the sequence in the following post.

Summary

Through an analysis using both conservative (Finnveden 2019) and best-guess estimates (Cook 2022)[3], we evaluate the Civ-Saturation Hypothesis[1] given four sets of beliefs (called ADT implementations[4]) which guide how to aggregate possible worlds by their Decision Relevance. We focus on two of them: CDT, and the most conservative EDT version (~ CDT + controlling exact copies). Our evaluations aggregate possible worlds using descendant resource-weighting rather than count-weighting, producing more accurate and more conservative estimates of Civ-Saturation (i.e. lower estimates).&...
Monthly Roundup #28: March 2025
by Zvi — 2025-03-17T12:50:03.097Z

I plan to continue to leave the Trump administration out of monthly roundups – I will do my best to only cover the administration as it relates to my particular focus areas. That is ‘if I start down this road there is nowhere to stop’ and ‘other sources are left to cover that topic’ and not ‘there are not things worth mentioning.’

Table of Contents

Bad News.

While I Cannot Condone This.

Good News, Everyone.

Opportunity Knocks.

For Your Entertainment.

I Was Promised Flying Self-Driving Cars and Supersonic Jets.

Gamers Gonna Game Game Game Game Game.

Sports Go Sports.

The Lighter Side.

Bad News

I also had forgotten this was originally from Napoleon rather than Bill Watterson.

Dylan O’Sullivan: Napoleon once said that the surprising thing was not that every man has his price, but how low it is, and I can’t help but see that everywhere now.

You destroyed and betrayed yourself for a handful of clicks.

Jasmeet: Dostoevsky wrote, “Your worst sin is that you have destroyed and betrayed yourself for nothing.”

That seems especially appropriate lately, for mostly non-AI reasons.

Mozilla seems to ban porn on Firefox and telling users it can harvest their data?

Disney shuts down 538, Nate Silver offers a few words. You can get all their data off of GitHub. It is a shame that 538 could not be sustained, and I am sad for those who lost their jobs, but as Nate Silver notes their business model was unsustainable inside Disney. Hopefully Silver Bulletin and others can carry the torch in the future.

Restaurant productivity technically rose 15% during the pandemic and sustained that gain, but it turns out it is entirely attributable to the rise of takeout and delivery. That’s not a rise in productivity, that’s delivering a different product that is easier to produce and also in general worse. If anything this change is bad.

Zeynep Tufekci, who has been on top of this from the beginning, reminds us of the massive efforts to mislead us about the fact that Covid-19 could have come from a lab. We don’t know whether Covid-19 came from a lab, but we do know it very much could have, that there was a massive coordinated operation to suppress this fact, and most importantly that this means that we are continuing to do lab research that is likely to cause future pandemics.

While I Cannot Condone This

Aditya Agarwal models a person’s ambition as something you can unlock and...
Are corporations superintelligent?
by Vishakha — 2025-03-17T10:36:12.703Z
Link: https://aisafety.info/questions/8C7S/Are-corporations-superintelligent

This is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback.

The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.

Corporations can be considered superintelligent only in a limited sense. Nick Bostrom, in Superintelligence, distinguishes between "speed superintelligence", "collective superintelligence", and "quality superintelligence".

Out of these, corporations come closest to collective superintelligence. Bostrom reserves the term “collective superintelligence” for hypothetical systems much more powerful than current human groups, but corporations are still strong examples of collective intelligence. They can perform cognitive tasks far beyond the abilities of any one human, as long as those tasks can be decomposed into many parallel, human-sized pieces. For example, they can design every part of a smartphone, or sell coffee in thousands of places simultaneously.

However, corporations are still very limited. They don't have speed superintelligence: no matter how many humans work together, they'll never program an operating system in one minute, or play great chess in one second per move. Nor do they have quality superintelligence: ten thousand average physicists collaborating to invent general relativity for the first time would probably fail where Einstein succeeded. Einstein was thinking on a qualitatively higher level.

AI systems could be created one day that think exceptional thoughts at high speeds in great numbers, presenting major challenges we’ve never had to face when dealing with corporations.