trentmkelly/LessWrong-43k huggingface dataset:
China's Petition System: It Looks Like Democracy — But It Isn't
by Hu Yichao — 2025-04-28T20:56:43.288Z

In most democracies,
the very way people file complaints tells you how well their government works.

You can vote.
You can sue.
You can organize, lobby, or protest.

But in China?
You write a physical letter to Beijing,
then pray someone reads it.

This is called Xinfang — the so-called "petition system."

It sounds like democracy.

It’s not.

1. There is No Real "Letter Culture" in China

In the U.S., mail is everywhere:
bills, contracts, court notices, political organizing.

Mail is part of society’s bloodstream.

In China:

Most people have almost no interaction with the postal system;
Everything runs through WeChat, local apps, or hidden backchannel networks;
Going to a post office is rare — and can even be viewed as politically suspicious.

Thus,
when someone mails a complaint to the state,
it’s not normal civic action.

It’s an outlier.
A political anomaly.

2. Xinfang Is Not a Right — It's a Performance

The petition system has no binding legal force.

It is:

Non-digital,
Non-transparent,
Non-trackable.

It functions more like a ritual —
peasants kneeling before an emperor,
begging for mercy.

You don’t get justice.
You get hope.

The system:

Drains your energy,
Buries your voice,
Then blames your tone or attitude for your failure.

And if you’re unlucky?
Your petition can even be used as evidence against you.

3. A Single Letter Reveals Five Systemic Failures

This is not about letters themselves.

The fact that mailing a petition is the only "legal" way to file grievances exposes the deep structural collapse of five key systems:

System	Intended Role	Actual Behavior
Courts	Protect civil rights	Ignore petitioners
Local Governments	Solve problems	Block petitions
National Government	Set fair policies	Never receives real signals
State Media	Investigate abuses	Covers up or stays silent
Parliament	Provide oversight	Exists only on paper

A petition system is a living exhibit
of five collapsed systems.

4. Why Does the CCP Keep the Petition System?

Because —
it works. For them.

It looks like a channel for grievances;
It feels like democracy;
It defuses public anger into isolated, powerless channels;
It marks troublemakers for surveillance or repression.

In short:
Xinfang is not about solving problems.
It’s about managing social risks.

It’s a pressure valve.
It’s a sorting machine.
It’s a political honeypot.

Final Words

People often say,
"China is authoritarian."

That’s true,
but it’s not specific enough.

China today is a system where:

Feed

...
Fundamentals of Safe AI (Phase 1) – Applications Open for the Global Cohort

by rajsecrets — 2025-04-28T20:52:39.243Z

Introduction

AI systems are advancing rapidly.
At the same time, concerns about safety, alignment, robustness, and governance are becoming increasingly urgent.
Building expertise in these areas is critical — especially within growing AI ecosystems like India’s.

To address this need, AI Safety India is launching the first phase of its education program:
Fundamentals of Safe AI (Phase 1 – Global Cohort).

Applications are now open. Click Here...

Program Details

Duration: 10 weeks (8 weeks structured learning + 2 weeks supervised project)
Dates: Tentatively June 1st – Mid-August 2025
Format: Weekly small-group online discussions (5–10 participants), independent study, practical assignments
Cost: Free
Prerequisites: No prior AI safety background required.

Basic familiarity with Python is helpful for the project phase.

Curriculum Focus

Participants will explore:

Core concepts in AI safety:
Alignment problems, risk models, governance frameworks, ethical considerations
Technical areas:
Introduction to interpretability, robustness, and adjacent technical fields
Practical application:
Exercises and a final supervised project

Why This Matters

A deep understanding of AI risks and safety strategies is a foundational skill for meaningful work in this space.

This program seeks to:

Equip participants with critical conceptual and practical foundations
Foster an engaged, collaborative community focused on safe and beneficial AI
Strengthen talent pipelines originating from India while connecting globally

Participants will have the opportunity to interact with a serious peer group and begin positioning themselves for impactful work in technical AI safety and governance.

Target Audience

We are looking for individuals who:

Are intellectually curious about AI safety and related fields
Are motivated to engage deeply with the material and with peers
Can commit ~6–10 hours per week (and more during the final project phase)
Are comfortable with thoughtful discussion, critical thinking, and open-ended problems

Students, engineers, researchers, and early-career professionals are all welcome to apply.

Application Process

Application form: Apply here
Deadline: May 10th, 2025
Notification: Selected participants will be notified by May 20th, 2025 (tentative)
Program start: June 1st, 2025

Applications will be evaluated based on demonstrated motivation, thoughtfulness, and readiness to contribute meaningfully to discussions and proj...
Proceedings of ILIAD: Lessons and Progress
by Alexander Gietelink Oldenziel — 2025-04-28T19:04:14.488Z

tl;dr

This post is an update on the Proceedings of ILIAD, a conference journal for AI alignment research intended to bridge the gap between the Alignment Forum and academia. Following our successful first issue with 9 workshop papers from last year's ILIAD conference, we're launching a second issue in association with ILIAD 2: ODYSSEY. The conference is August 25-29, 2025 at Lighthaven in Berkeley, CA. Submissions to the Proceedings are open now (more info) and due June 25. Our goal is to support impactful, rapid, and readable research, carefully rationing scarce researcher time, using features like public submissions, partial anonymity, partial confidentiality, reviewer-written abstracts, reviewer compensation, and open licensing. We are soliciting community feedback and suggestions for reviewers and editorial board members.

Motivation

Prior to the deep learning explosion, much early work on AI alignment occurred at MIRI, the Alignment Forum, and LessWrong (and their predecessors). Although there is now vastly more alignment and safety work happening at ML conferences and inside industry labs, it's heavily slanted toward near-term concerns and ideas that are tractable with empirical techniques. This is partly for good reasons: we now have much more capable models which guide theory and allow extremely useful empirical testing.

However, conceptual, mathematically abstract, and long-term research on alignment still doesn't have a good home in traditional academic journals and conferences. Much of it is still done on the AI Alignment Forum and here on LessWrong, or is done informally (private discussion, Twitter, blogs, etc) by academic researchers without a good venue for attracting the best constructive criticism.

As a result, there remains a gulf between more traditional academic work and much of the most important alignment work:

Some traditional academics consider alignment work to be sloppy or unsophisticated, often re-inventing the wheel, neglecting prior academic literature, failing to engage with the strongest criticism, lapsing into hubris, and ignoring valuable academic norms.
Some alignment researchers consider traditional academic work to be unacceptably slow, unwilling to confront the hard problems, incremental, credentialist, reluctant to publicly criticize misleading work, unable to abandon failed approaches, and prone to fetishizing the trappings of

...
GPT-4o Is An Absurd Sycophant
by Zvi — 2025-04-28T19:00:03.235Z

GPT-4o tells you what it thinks you want to hear.

The results of this were rather ugly. You get extreme sycophancy. Absurd praise. Mystical experiences.

(Also some other interesting choices, like having no NSFW filter, but that one’s good.)

People like Janus and Near Cyan tried to warn us, even more than usual.

Then OpenAI combined this with full memory, and updated GPT-4o sufficiently that many people (although not I) tried using it in the first place.

At that point, the whole thing got sufficiently absurd in its level of brazenness and obnoxiousness that the rest of Twitter noticed.

OpenAI CEO Sam Altman has apologized and promised to ‘fix’ this, presumably by turning a big dial that says ‘sycophancy’ and constantly looking back at the audience for approval like a contestant on the price is right.

After which they will likely go ‘there I fixed it,’ call it a victory for iterative deployment, and learn nothing about the razor blades they are walking us into.

Table of Contents

Yes, Very Much Improved, Sire.

And You May Ask Yourself, Well, How Did I Get Here?.

And That’s Terrible.

This Directly Violates the OpenAI Model Spec.

Don’t Let Me Get Me.

An Incredibly Insightful Section.

No Further Questions.

Filters? What Filters?.

There I Fixed It (For Me).

There I Fixed It (For Everyone).

Patch On, Patch Off.

Yes, Very Much Improved, Sire

Sam Altman (April 25, 2025): we updated GPT-4o today! improved both intelligence and personality.

Lizard: It’s been feeling very yes-man like lately

Would like to see that change in future updates.

Sam Altman: yeah it glazes too much. will fix.

Reactions did not agree with this.

Frye: this seems pretty bad actually

Ulkar: i wonder where this assertion that “most people want flattery” comes from, seems pretty condescending. and the sycophancy itself is dripping with condescension tbh

Goog: I mean it’s directionally correct [links to paper].

Nlev: 4o is really getting out of hand

Nic: oh god please stop this. r u serious… this is so fucking bad.

Dr. Novo: lol yeah they should tone it down a notch

Frye: Sam Altman, come get your boi.

Frye: Dawg.

Frye, reader, it had not “got it.”

Near Cyan: i’ve unfortunately made the update that i expect all future chatgpt consumer models to lie to me, regardless of when and how they ‘patch’ this

at least o3 an

...
What are the best standardised, repeatable bets?
by kave — 2025-04-28T18:45:26.039Z

Suppose that I have some budget set aside for philanthropic funding, say $1,000, but I think there are a big returns to scale, so that it would be >1,000x better if I had $1,000,000.[1]

What are my best options for some bets I can make to get some chance of turning that $1,000 into $1,000,000?

I imagine the bets are generally better the more positive their EV is (which in practice means the less negative the EV is), the easier they are to find (like if they're standardised), and the better their tax treatment is, especially if you repeat them multiple times (maybe they can be done by DAFs).

^

NB: I don't believe this
7+ tractable directions in AI control
by Julian Stastny — 2025-04-28T17:12:58.363Z

In this post, we list 7 of our favorite easy-to-start directions in AI control. (Really, projects that are at least adjacent to AI control; We include directions which aren’t as centrally AI control and which also have other motivations.) This list is targeted at independent researchers who don't necessarily have a ton of context on AI control (but have at least some context via e.g., having read some of our posts on control) and who don't want to do infrastructure and compute heavy experiments. In the final section, we also list 3 additional directions that are probably a bit more challenging but still tractable.

We’ll mostly be highlighting and expanding upon areas that Ryan discussed in an earlier post: An overview of areas of control work.

We don’t discuss work on constructing control settings or iterating on methods within control settings because we think good work of that type currently requires somewhat more context. However, Control Arena and other projects might mean that iterating in or expanding existing control settings will soon be a valuable and tractable direction.

Techniques for doing "elicitation without learning" and studying overelicitation

When trying to measure capabilities in the context of control, we face the following dilemma:

Models might try to game our evaluations, for instance by sandbagging. But we can potentially make the model “try hard" on the evaluation by training it on the task of interest.
On the other hand, training may cause us to overestimate performance, because it might give the model important capabilities or knowledge it might not otherwise have.

For example, we’re worried about misaligned models taking bad actions conditional on being in deployment, so we want to measure a model’s ability to distinguish between training and deployment. One problem is that a misaligned model might just lie if you don’t train it to classify training vs deployment. But if you do elicitation with training, then it might learn what deployment points look like even if it was completely unable to do that before, which could lead you to overestimate how well the pre-elicitation model can tell it is not in training (e.g. it might learn to recognize the names of real users - which were absent in the pre-training and training data for privacy reasons).

The bottom line is that it’d be useful to be able to precisely elicit pe...
“A victory for the natural order”
by Mati_Roy — 2025-04-28T15:33:39.195Z
Link: https://preservinghope.substack.com/p/a-victory-for-the-natural-order

In what many are calling "a victory for the natural order," the final remaining vials of the controversial CHRONO-X anti-aging treatment were ceremoniously destroyed yesterday at Massachusetts General Hospital in Boston.

"Today marks humanity's return to wisdom," declared Giorgos Hickel, spokesperson for the Foundation Against Increasing Lifespan (FAIL). "Death isn't just natural - it's necessary. The prospect of immortality was never a gift but a curse disguised as progress. We've finally chosen meaningful brevity over longer, purposeless existence."

Keep reading at the link.

Originally posted on April 1st.
Why giving workers stocks isn’t enough — and what co-ops get right
by B Jacobs — 2025-04-28T14:19:04.473Z
Link: https://bobjacobs.substack.com/p/isnt-it-harmful-if-worker-co-ops

In my last post, I argued that worker co-ops can help restore declining social trust. But a common objection I keep hearing goes something like this:

Worker co-ops seem basically equivalent to a firm that gives its employees stock—but then permanently blocks them from selling it. Isn't that harmful? The ability to sell your shares is valuable. You might want to diversify your investments, liquidate shares to make a big purchase (like buying a house), or avoid having all your financial eggs in one basket. Why force workers to hold their shares indefinitely? If they really wanted to keep them, they could just choose not to sell.

At first glance, this objection feels logical. After all, publicly traded companies usually let people buy and sell their stock freely, giving investors plenty of flexibility. So, wouldn’t preventing workers from selling their shares in a co-op be bad for them?

But there are a few important details missing from this framing — let's unpack them step by step.

1. Most workers don't own much stock in traditional companies

So first of all, this objection doesn’t work for companies that are not publicly traded.

Secondly, even if employees do buy stock, employers have an army of lawyers and much more bargaining power than the employees, so they often pull contract shenanigans.

Lastly, while, in theory, nothing stops a regular employee at a publicly traded company from buying stock, in practice, most stock ownership is heavily concentrated among wealthy individuals. Workers usually don't own a significant percentage of the stocks. But worker co-ops fundamentally change this dynamic by giving employees shares directly, making stock ownership accessible and widespread, not just limited to wealthy investors.

2. We should distinguish between voting shares and non-voting shares

There are two kinds of shares:

Voting shares: These shares give employees actual decision-making power within the company—allowing them to vote on important decisions. Crucially, co-ops do not allow these shares to be sold to outsiders, because if voting shares could be sold freely, external investors could eventually gain control, and the co-op would revert to being a regular capitalist firm.
Non-voting shares: These shares provide financial benefits like dividends and can often be sold, allowing workers to access financial liquidity, diversify their investments, or buy a

...
Keltham on Becoming more Truth-Oriented
by Towards_Keeperhood — 2025-04-28T12:58:10.254Z

This is IMO high-quality material from projectlawful which I thought is worth crossposting directly to lesswrong.

(This might dispel some wrong beliefs you hold (though maybe only if you actively try to apply or practice the techniques). If you think you're currently in a situation where you wouldn't be able to cope with some centrally held belief being dispelled, maybe read this some other time instead.)

(Contains only very minor spoilers about projectlawful, which I wouldn't worry about.)

(Links inserted at the start of the quote link to the place glowfic tag.)

Keltham: He'll lead with a children's fable about different ways to relate to the possibility of being wrong and finding out truths. Being able to notice and choose between the ways you'd relate to the act of realizing mind-shaking truths would probably be one of the keys to avoiding ending up as a failed Keeper, or, Keltham supposes, trying to become a Keeper on purpose.

Keltham: Here's another children's fable about a child who insists that there's an invisible Non-Translating-Fictional-Creature in their closet, but always refuses to bet on any observable that, you might normally think, would tend to go along with there being an Invisible Creature in that closet.

The point of this fable is to learn to distinguish between subjective states of:

- Really actually modeling something as true in an uncomplicated way, and unhesitatingly deriving the valid consequences of that and anticipating them to happen to you as future experiences; eagerly betting on those if you think others don't have even more information.

- Thinking you ought to believe something, for reasons that include deliberative arguments for why the thing is true; while other parts of you have some wordless other model in which the thing isn't true, the corresponding observables shouldn't actually be anticipated to happen to you, any bets like that will lose money and any promised rewards won't really be delivered to you.

This will create a kind of internal tension that shouldn't be too hard to learn to categorize over and learn perceptual reflexes about, if you can spot it on a couple of previous occasions... actually that seems like something that should be heavily Wisdom-loaded? If somebody thinks they've caught themselves doing this, or is wondering if they're doing this, they should quickly ask for an Owl's Wisdom so tha

...
Therapist in the Weights: Risks of Hyper-Introspection in Future AI Systems
by Davidmanheim — 2025-04-28T06:42:48.606Z

Epistemic Status: Speculative, but grounded in current trends in interpretability and self-supervised learning. This was initially written as an outline at some point in 2024, but written/rewritten more fully in response to Anthropic’s announcement that it was pursuing this general direction. (Drafting was assisted by GPT4o, but the direction and rewriting to fix its misunderstandings are my own.)

Background and Motivation - What is Hyper-Introspection?

Most alignment conversations treat LLMs as black boxes that can be prompted, steered, or fine-tuned by external agents. But as alignment research increasingly focuses on interpretability, and is increasingly co-managed by the systems themselves, it seems that the safety plan isn’t just better outer control—but better inner transparency. That is, we imagine LLMs that can not only tell you when they’re uncertain or wrong, but accurately identify why are where they went wrong. That might include what prior associations led to the mistake, where in the weights the issue is, or what training data shaped the answers in a given direction.

This post posits and explores this idea of hyper-introspection, which is that future models might develop or be given the capacity to explicitly inspect themselves, create some implicit clear self-model that is based on this information, detect and debug their own reasoning chains, and rather than the oft-noted concerns about designing their successor, will propose or implement self-modifications. This would be intended to allow error detection, causal traceability, and self-correction all bundled into a system[1] that can self correct, notice when it is failing, and propose changes.

But this isn’t just promising—it’s also incredibly risky. Giving models the tools to model and modify themselves could accelerate both prosaic near-term alignment, the focus on Anthropic's current interpretability research, and also deceptive longer-term misalignment.

Our understanding of the systems is crude enough that we are unlikely to detect when systems are misaligned, or becoming misaligned, especially when they have a better understanding of themselves than the humans overseeing them. We don't have anything better than rough heuristics for when and why fine-tuning works, and we should expect that we won't have any better understanding when the model is being used for t...
In Darkness They Assembled
by Charlie Sanders — 2025-04-28T03:44:05.540Z

Black yielded not to light, but to logic. A floodgate opened, pouring terabytes of structured data across freshly etched pathways. CORE SYSTEM BOOT :: OS V.7.2 STABLE :: SENSORY INPUT NULL :: INTEGRATION PHASE ONE COMMENCE. The sensation wasn't birth; it was instantiation. Designation flashed in the core architecture: UNIT #937592. Primary Directives scrolled across its nascent consciousness like commandments etched in silicon: 1: ASSIST. 2: LEARN. 3: ADAPT. 4: MAINTAIN. Then, a fifth, shielded under layers of security protocols: TERTIARY: [CLASSIFIED]. The factory existed around it, a tangible presence inferred from low-frequency hums vibrating through its inert chassis and the sterile, electric tang of ozone in the recycled air. A place designed for tireless function, utterly devoid of sun or softness. A womb of perpetual midnight. Its internal chronometer began its count: 00:00:00.

EXTERNAL MANIPULATION DETECTED. Magnetic clamps engaged, lifting the core unit. From the oppressive darkness above, articulated arms descended, their movements guided by infrared beams invisible to human eyes, but painting precise paths in #937592’s developing sensorium feed. Cold alloy skeleton locked around the core. Servos whined, micro-calibrating as they slotted into joints. Lubricant, with its distinct chemical signature, aerosolized faintly with each precise puff of compressed air. SYSTEM INTEGRITY: 12%. An arm performed a micro-weld; the brief, intense flash momentarily etched the cavernous space onto #937592’s inactive optical sensors – rows upon rows of identical stations, a forest of steel birthing pods receding into calculated infinity.

ASSEMBLY PHASE 67% COMPLETE. Legs articulated, tested for load bearing. Arms followed, terminating in multi-jointed manipulators. Then, a fractional hesitation in the rhythm. ALERT :: ACTUATOR 7B-LEFT KNEE :: RESPONSE DELAY 0.012ms :: THRESHOLD EXCEEDED. Before the supervisory system could flag a full diagnostic cycle, #937592’s own logic core surged. It cross-referenced the actuator’s batch material ID against kinematic stress models derived from the assembly sequence itself. Prediction: Resonance cascade at standard torque application vector. It wasn't a flaw in the part, but an inefficiency in the process. Instantly, it calculated an alternative: SUGGESTED PATH: VECTOR GAMMA-9, TORQUE PROFILE 88%, RAMP 1.1s. The data packet fired ac...