trentmkelly/LessWrong-43k huggingface dataset:
Stress exists only where the Mind makes it
by Noahh — 2025-03-10T19:44:42.887Z

We commonly and unknowingly model stress as a direct function of an event: F(s) = E. Phrases like "that event was stressful," "that job interview was stressful," or "start-ups are stressful" imply stress is an objective property of the event itself, fixed to a job interview as much as its physical location. This subconscious belief, evidenced by our language, suggests stress is inherent to the event, inevitably affecting anyone who encounters it, now or in the future. The natural corollary of this, of course, is that reducing stress is impossible. In this post, I’ll propose a more accurate and useful way to model stress.

I. The Stress Equation

First, let me disprove the old model. If stress were solely a result of the event (F(s) = E), then a single event—like catching a flight—would produce identical stress levels for everyone. Yet, we know this isn’t true. Instead, stress depends on two factors: the perceived importance of the event (PIE) and our stress response to that importance (Rs). A better model is F(s) = PIE * Rs[1].

Events themselves aren’t inherently important. We feel stress over what we deem important, shaped by our individual values and pursuits. For instance, a student aiming for grad school may stress more about the upcoming exam than a student just trying to graduate. Same external event of the exam, but different levels of perceived importance (PIE). Typically, the more important something is, the more stressed we’ll be, although that is not always the case.

Another interesting observation is how stress seems to have a maximum intensity regardless of a situation's objective importance. Consider a high schooler trying to spit game with their crush—which, for argument's sake, isn't objectively significant. I wouldn’t be that surprised if I found out the high schooler experienced more stress than say, Eisenhower did on D-Day. Both parties perceive their situations as important, but the high schooler's less refined response capability results in a higher Rs variable. Stress behaves somewhat like a gas, expanding to fill available space—the most pressing matters in any life phase tends to reach a similar "stress ceiling" regardless of objective consequences.

If I look at my own life, I can see that stress doesn't scale linearly with even my own subjective perceptions of importance. I've felt irrationally stressed about trivial matters like...
Counterargument to Godel's Modal Ontological Argument
by Wynn — 2025-03-10T19:38:24.777Z

Was thinking today about the modal ontological argument specifically as it relates to our multiple conceptions of God across civilizational history - wanted to formalize it.

Essentially the fundamental argument here is that, by necessity, either the S5 argument used in the modal ontological argument leads to a multiplicity of possible deities that, by the ontological argument itself, must exist.

Let me know if anyone finds any counterarguments/issues in the logic here - there are certainly axiomatic counterarguments, but I've workshopped a few of them and they seem weak?

Definition 1: Property P is positive iff it contributes to greatness.
Definition 2: X is God-like iff X has all positive properties.
Definition 3: X necessarily exists iff X exists in all possible worlds.

Axiom 1: If P is positive and P entails Q, then Q is positive.
Axiom 2: Either P is positive or ¬P is positive, not both.
Axiom 3: God-likeness is positive.
Axiom 4: Necessary existence is positive.

Definition 4: A divine conception is a set of attributes that could coherently belong to a greatest possible being.

Lemma 1 (Conception Multiplicity): Multiple coherent divine conceptions exist.
Proof:

Empirically observable across religious traditions (personal/impersonal, trinity/unity).
These conceptions differ substantively yet each maintains internal coherence.
Each can be articulated without logical contradiction.

Challenge 1: "But seemingly contradictory attributes (justice vs. mercy) must reconcile in a unique maximal being!"
Response:

Reconciliation attempts typically employ one of three strategies:
a) Definitional gerrymandering (redefining justice until it's compatible with mercy).
b) Mysterious transcendence ("They're reconciled beyond human understanding").
c) Hierarchical subordination (one attribute trumps the other).
Each strategy fails:
a) Creates vacuous concepts that lose normative force.
b) Renders the conception unfalsifiable and meaningless.
c) Admits multiple possible hierarchies → multiple conceptions.

Definition 5: A conception parameter φ is any attribute dimension admitting multiple coherent values.

Lemma 2 (Parameter Space): Divine conception space contains multiple independent parameters.
Proof:

Consider attributes: consciousness type, moral character, relation to time, aesthetics.
These vary independently (personal timeless being ≠ impersonal timeless being).
Each parameter admits mul

...
How much do frontier LLMs code and browse while in training?
by Joe Rogero — 2025-03-10T19:34:23.950Z

Under what circumstances are frontier AIs currently connected to the Internet during training and/or fine-tuning? Do any of the major players building "agents" like Operator do this? I'm aware that offline web hosting is a thing, just not sure if everyone's using it or if some just hook up to a browser and let fly. Specific examples would help tremendously.

Relatedly, when during training do models start writing and getting feedback on code? I'm guessing this is part of standard fine-tuning now for models like Llama, Gemini, Claude, and GPT? Can anyone point to some go-to papers describing the process?
Observations on self-supervised Learning for vision
by Dinkar Juyal — 2025-03-10T19:31:19.072Z

This post is a collection of observations around self-supervised learning for vision-based datasets. This is by no means a complete survey of SSL techniques, and assumes familiarity with Contrastive Learning, Masked Image Modeling and Masked AutoEncoder frameworks. For readers intending to get up to speed on these topics, there are excellent resources available online.

Setting

The datasets in the real world are rarely as well-understood and worked upon as academic datasets such as ImageNet-1k. They may have long-tailed distributions and/or high degree of label imbalance for a subset of classes. The labels may be noisy and the image metadata might be missing/unreliable. Further, label collection/verification might be time-consuming and expensive. There might be systematic confounders present in the data. We might be expected to deploy the model in situations where the data distribution might be unknown to us. At the same time, if our model performance on the metrics that we care about degrades, we should be able to correct for that with minimum effort and disruption to our workflows. In this context, we want to harness SSL pretraining (or pre-pretraining) to alleviate the need of labels, generate synthetic data for rare classes and learn rich representations that can be used for a variety of downstream tasks.

Vision datasets can have pervasive label issues (Source)

Contrastive Learning + Masked Image Modeling : Best of both worlds?

Contrastive learning (CL) involves bringing representations of similar samples closer and pushing apart representations of dissimilar samples. Initial methods for CL used a CNN encoder as a feature extractor. The need for positive and negative pairs within the same batch meant that these methods relied on bigger batches and correct assumptions about what constitutes positive and negative examples. Modifications that did not require negative pairs while avoiding the trivial collapsed features as a solution (SimSiam, BYOL, VICReg), and deal with noisy negative examples (RINCE loss) were suggested. With the advent of ViT, new approaches that integrate ViT into the SSL framework were designed. Works such as Park et al cover the difference between the properties of multi-head self-attention (MHSA) used in ViT and CNNs - MHSA are low-pass filters and shape-biased, while convolutions are high-pass filters and texture-biased. There is als...
Introducing 11 New AI Safety Organizations - Catalyze's Winter 24/25 London Incubation Program Cohort
by Alexandra Bos — 2025-03-10T19:26:11.017Z

[Crossposted from: https://www.catalyze-impact.org/blog]

We are excited to introduce the eleven organizations that participated in the London-based Winter 2024/25 Catalyze AI Safety Incubation Program.

Through the support of members of our

Seed Funding Network

, a number of these young organizations have already received initial funding. This program was open to both for-profit and non-profit founders from all over the world, allowing them to choose the structure that best serves their mission and approach. We extend our heartfelt gratitude to our mentors, funders, advisors, the LISA offices staff, and all participants who helped make this pilot incubation program successful.

This post provides an overview of these organizations and their missions to improve AI safety. If you're interested in supporting these organizations with additional funding or would like to get involved with them in other ways, you'll find details in the sections below.

To stay up to date on our activities, or to be involved in future iterations of the program, please consider

subscribing to our newsletter

and/or filling out our

expression of interest form

.

Group picture in front of the

LISA offices

.

The AI Safety Organizations

Wiser Human

advances AI control by modelling risks, stress-testing safeguards, and developing subversion-robust control protocols.

[Stealth]

is bringing the Industrial Revolution to science, starting with systematizing AI alignment with AI agents.

TamperSec

develops hardware security for AI hardware, enabling international AI governance.

Netholabs

is a Whole Brain Emulation (WBE) research organization dedicated to preserving and enhancing human agency.

More Light

protects insiders and evidence against bad actors in frontier AI labs, and helps apply strategic legal pressure to achieve reforms.

Lyra Research

is leveraging existing model capabilities to directly automate safety research with agentic systems, today.

Luthien

is building on AI Control research to provide production-ready AI Control solutions as quickly as possible.

Live Theory

is building out a novel research methodology and infrastructure for contextually-adapting AI safety.

Anchor Research

develops evaluations to help understand whether AI systems actually stay safe and aligned when running for days or weeks at a time.

Aintelope

’s mission is to solve AGI alignment via biolog

...
The Jackpot Jinx (or why “Superintelligence Strategy” is wrong)
by E.G. Blee-Goldman — 2025-03-10T19:18:56.380Z

On March 5, 2025 Dan Hendrycks, Eric Schmidt, and Alexandr Wang published “Superintelligence Strategy”, a paper that suggests a number of policies for national security in the era of AI. Central to their recommendations is a concept they call “Mutual Assured AI Malfunction (MAIM)” which is meant to be a deterrence regime resembling nuclear mutual assured destruction (MAD). The authors argue that by MAIMs will deter nations from building “destabilizing” AI by the threat of reciprocal sabotage.

This is a demonstrably false concept, and a poor analogy, because it fails to yield a strategy that settles into a Nash equilibrium. Instead, MAIMs uncertain nature increases the risk for miscalculations and encourages internecine strife. It is a strategy that likely would break the stability-instability paradox and is fraught with the potential for misinterpretation.

One of the key miscalculations is the paper’s treatment of the payoffs in the event of superintelligence. Rather than considering the first nation to reach superintelligence is a winner-take-all proposition, we should think about it more as something I call the “Jackpot Jinx.” This term captures the allure of an enormous (even potentially infinite) payoff from a breakthrough in superintelligence can destabilize strategic deterrence. Essentially, the prospect of a "jackpot" might “jinx” the stability by incentivizing preemptive or aggressive actions.

Let’s start by discussing why the nuclear mutual assured destruction (MAD) yields a pareto-optimal Nash Equilibrium (that is an equilibrium that is better for everyone without making anyone worse off.) Under MAD, the inescapable threat of a retaliatory nuclear strike ensures that any unilateral move to initiate conflict would lead to mutually catastrophic consequences. The idea is that over time, and over many potential conflicts, both nations recognize that refraining from launching a first strike is the rational strategy because any deviation would trigger an escalation that leaves both parties far worse off (i.e. both countries are nuked.)

The equilibrium where neither nation initiates a nuclear attack becomes self-enforcing: it is the best collective outcome given the stakes involved. Any attempt to deviate, such as launching a surprise attack, would break this balance and result in outcomes that are strictly infe...
Effective AI Outreach | A Data Driven Approach
by NoahCWilson — 2025-03-10T19:18:52.161Z

Noah C Wilson | Noah.C.Wilson1993@gmail.com

Key Takeaways:

Building a massive mainstream global movement is essential to achieving safe AGI.
AI outreach has become the most urgent cause in AI safety due to accelerating timelines. AI outreach remains underemphasized compared to technical AI safety research.
A pilot study I conducted found that influencing AI risk sentiment is highly tractable. I used snapshot explainer interventions to successfully influence the concern about AI risks and support for AI regulation in a sample of 100 participants.
This study also investigates more and less effective means of influencing this sentiment.

What is AI Outreach?

AI outreach is the process of informing and engaging the general public about the risks posed by artificial intelligence, with the goal of growing the AI safety movement to foster an informed and cautious societal response. This includes conveying catastrophic concerns in an accessible way, countering misconceptions, storytelling, media engagement, and cultural interventions that reshape how AI is perceived and discussed.

Why Do AI Outreach? I.T.N. Framework

AI Outreach is Important

When we look at history, the most significant societal shifts—especially in response to existential risks—didn’t happen because a few experts had quiet conversations behind closed doors. They happened because public sentiment changed, and in doing so, forced institutions to respond.

This pattern repeats across some of the most pressing challenges humanity has faced:

Nuclear Disarmament Movements – The fear of nuclear war was abstract to policymakers in the 1950s, but mass movements like the Campaign for Nuclear Disarmament (CND) and the Nuclear Freeze Movement turned it into a crisis the public cared about. Protests, advocacy, and cultural shifts (think: "Dr. Strangelove" and the Doomsday Clock) created real pressure that led to treaties like the Partial Test Ban Treaty (1963) and, later, the INF Treaty (1987). Governments moved because people demanded action. [1]
Climate Change Activism – The science on climate change was clear decades before governments acted meaningfully. What changed? Movements like Fridays for Future, Extinction Rebellion, and the broader environmental movement shifted public perception. This, in turn, made climate action politically viable, leading to policies like the Paris Agreement

...
Emergent AI Society. Tasks, Scarcity, Talks
by Andrey Seryakov — 2025-03-10T19:18:09.827Z

In this article, I explore how AI agents on the web, driven only by task completion and resource efficiency incentives, may naturally form a self-regulating society with their own culture, economy, and governance — without human oversight and intention. Tasks, resource scarcity, and talks, nothing else is needed.

AI on the web

AI agents are autonomous algorithms performing various tasks for us on the internet. Even today, these tasks are quite diverse — information search (from simple weather forecasts to

deep research

),

money-related issues (from web shopping to stock trading), communication (

chat-bots from shop-assistance

to psychotherapy), and

so on

.

But the future prospects are so vast that it’s hard to predict and grasp from today what AI will be able to do in 10 years. We are already speaking about scientific research mostly conducted by bots, complex systems like health care or road traffic managed by them, but in reality, we just don’t know. As AI is being introduced in all areas of our lives, the energy consumption of the AI industry will scale drastically, and a need for more and more resources will only intensify, leading to emergent behaviors driven by the necessity to optimize energy consumption.

Resource scarcity

Scarcity is the engine of evolution, a driver for innovations from biology to technology. The first predators (microbes) emerged when sunlight, chemical energy, and essential micro-elements became scarce. In the modern IT economy, there are two incentives for saving computational resources. The first — money — has always been there. The second — climate change — is newer but will become increasingly critical in the coming years.”.

Computers cost money, consume energy, and emit heat and CO2 — and running modern AI requires a vast number of them. A lot of energy goes to the training, but it’s a one-time spending. A single use, on the other hand, consumes comparatively little energy — but at scale, these add up significantly. Currently,

it’s hard to say

whether we spend more on training or usage but the numbers are close to each other.

For previous generation models (not reasoning ones) the estimated carbon foot-print is around a

few grams of CO2 per message

with energy consumption (and heat emission) of around

0.3 watt-hour for ChatGPT

which isn’t much, like 100 times less than your cattle uses to boil a liter of water. In December 2024 OpenAI...
Sentinel minutes #10/2025: Trump tariffs, US/China tensions, Claude code reward hacking.
by NunoSempere — 2025-03-10T19:00:25.808Z
Link: https://blog.sentinel-team.org/p/sentinel-minutes-102025-trump-tariffs

The Sentinel minutes are a weekly newsletter looking at precursors of large-scale risks. Initially it was produced for our emergency response team, but it has since gained a wider readership. It might be of interest to LessWrong due to LW's interest on existential risks, although this is probably not the single greatest roundup on AI specifically.

Top items:

Market uncertainty and political turmoil over Trump tariffs on Canada, Mexico and China
Increasing US-China tensions
Anthropic’s Claude Code displays reward-hacking tendencies.

Forecaster Estimates

Taiwan has been a hot topic in the last week as China ramps up their military exercises. On the question of whether China would attempt to invade Taiwan by 2030, forecasters estimated a 43% chance of this occurring (range: 20% to 70%).

There has been significant speculation online over whether OpenAI has lost its edge. Our estimates on which AI company currently has the most capable general-purpose AI model internally are:

P(OpenAI is in the lead) = 40%
P(Google DeepMind is in the lead) = 20%
P(Anthropic is in the lead) = 19%
P(xAI is in the lead) = 10%
P(Another AI company is in the lead) = 11%

Geopolitics

United States

The Courts

In the US, the Supreme Court ordered the Trump administration to pay foreign aid organizations for work that was already done. Forecasters think it’s unlikely Trump will straightforwardly ignore the Supreme Court at this point.
Over 100 lawsuits against Trump administration actions are currently in progress; it is likely that more cases will reach the Supreme Court.
A federal judge extended a block on the Trump administration's freeze of federal grants and loans. The judge said that the executive branch was attempting to usurp Congressional powers and that its funding freeze “undermines the distinct constitutional roles of each branch of our government.”

Tariffs and the Economy

The Trump administration imposed a 25% tariff on goods from Canada and Mexico, and 10% on Canadian energy; within two days, Trump paused tariffs on goods that fall under a previous agreement and vehicles. Tariffs on Chinese goods will also rise to 20%.
When the tariffs were initially announced, Canada, Mexico and China said that they would impose retaliatory tariffs. Canadian PM Trudeau is reportedly unwilling to lift any tariffs on US goods if Trump leaves any tariffs on Canadian goods.
The Atlanta Federal Reserve updated their f...
Have you actually tried raising the birth rate?
by Yair Halberstadt — 2025-03-10T18:06:40.987Z

I just saw on twitter someone claiming that we "don't know how to raise the birth rate". No need to link them, it's a fairly common sentiment.

Have you actually tried raising the birth rate? For real, as in actually put in any effort?

Let's run the numbers on the stupidest possible thing that could work. Offer anyone who has a baby 50,000 dollars, tax free, no questions asked.

The median household income in the USA post tax is under 70,000 dollars. 50,000 dollars is a life changing amount for a large percentage of Americans. I would be shocked if a policy like this didn't send birth rates through the roof. Heck, I earn way more than that and I'd probably have more kids with a policy like that!

So how much would it cost?

There's about 3.5 million births a year in the USA. That comes to 175 billion dollars, or 3% of the federal budget. Even if the policy is incredibly successful and doubles birth, that comes to 6% of the federal budget.

That's about half of defense spending. It seems to me that ensuring the country has a future in 30 years is at least as valuable as maintaining the pax Americana. Even more it's a prerequisite to do so, unless we've invented robot soldiers by then.

Not convinced it would work? So test it out! Pick 20 counties randomly by lottery to implement this policy for, for 5 years. If it works well expand to the whole country.

But no, we've got no clue how to raise the birth rate, nothing to be done here, let's all go extinct.