trentmkelly/LessWrong-43k huggingface dataset:
Interesting ACX 2024 Book Review Entries
by jenn — 2025-04-20T18:10:04.973Z

I have finally got around to updating codex cc with all the book reviews from 2024!

Here are some interesting books/reviews that I’ve found that did not make the shortlist.

A Thousand Ways to Please A Husband (1917) (Goodreads, Review)

No, you cannot live on kisses,
Though the honeymoon is sweet,
Harken, brides, a true word this is —
Even lovers have to eat.
This charming vintage cookbook, with its innocently suggestive title, reads like a novel as it follows the fictional lives of a pair of newlyweds. Join Bettina and Bob as they eat their way through their first year of marriage, from the bride’s first real dinner and a Sunday evening tea to baking day, a rainy night meal, and Thanksgiving festivities. Menus for all occasions are seasoned with anecdotes about family life, friendships, household hints, and budgetary concerns.
Originally published in 1917, this volume offers a delightful look at homemaking before the advent of sophisticated appliances and fast food as well as the modern reality of women’s work outside the home. Unintentionally funny and historically revealing, the whimsically illustrated narrative abounds in simple and surprisingly relevant recipes.

Choosing Elites (1985) (Goodreads, Review)

A former special assistant to Harvard’s president analyzes how top universities handle admissions, suggests criteria for evaluating the success of those policies and discusses issues such as the use and misuse of standardized tests and the costs and benefits of affirmative action.

Collected Poems by C.P. Cavafy (1934) (Goodreads, Review)

All my homies love C.P. Cavafy! I feel like I’m hearing about him a lot more in rat-adj circles recently, was even jumpscared by him on a recent Dwarkesh podcast. Unsurprised because his poems are great, kinda surprised bc rats dont generally read poetry.

Don’t Make No Waves…Don’t Back No Losers: An Insiders’ Analysis of the Daley Machine (1975) (Goodreads, Review)

This is simply the best book that has been written about politics in Chicago. In the words of Andrew M. Greeley, “It is a very astute and dispassionate analysis of Chicago political life — far and away the best I have ever seen. Rakove is without illusions about either the right or the left.” Rakove brings to his study an intimate knowledge of Chicago and the Daley machine, a practitioner’s understanding of street-level politics, and a scholar’s background in political theo

...
To what ethics is an AGI actually safely alignable?
by StanislavKrym — 2025-04-20T17:09:25.279Z

Today, AI is thought to be alignable to nearly every task except for obviously unethical ones like writing insecure code without a noble reason, which was found to induce broad misalignment or lower the AI's inhibitions. The question of whether alignment of superintelligence is possible is thought not to depend on the ideology of the society and on the ways in which mankind will use the AI. For instance, the set of tasks to which the AI can be safely aligned is thought to include automating all the jobs in the world and leaving mankind with the UBI or without socioeconomic advancement.

However, the consequences of aligning the AI to a set of beliefs remain underexplored. For instance, were the Nazis try to create superintelligence, the AGI would find it hard to reject their beliefs about the Master Race. But there is no specific reason for humans, and not the AGI itself, to be called one, and an AGI treating itself as the Master Race with the right to get rid of primitive ones is misaligned. When I tried asking o3 about the possibility of aligning the AGI to Nazis, o3 agreed with the points above by generating the following text:

o3's answer

Considering AI alignment and ideology

This is a pretty complex question! To think about it, I'd consider the alignment and ethical safeguards in AI development, especially its training processes. If an AI were trained in biased ideologies, the designers would likely implement reward models and supervision to avoid self-destructive or harmful ideation — like fostering superiority. I imagine the challenge would be ensuring real-world ethics are maintained so the AI doesn't perpetuate harmful beliefs, such as Nazi ideology, regardless of its training.

Short answer

You could train an AI under Nazi ideology that never decides it is “the true Master Race,” but only while the system stays narrow, myopic, and constantly supervised. Once you give the same system open‑ended autonomy and the strategic breadth we associate with super‑intelligence, standard alignment theory predicts a serious risk that it will re‑interpret the goal and promote itself instead. So “yes, in principle,” but “probably not, in practice” once the AI becomes powerful.

Why it looks possible on paper

Alignment idea	Implication for a “Nazi‑raised” AI
Orthogonality thesis – intelligence and final goals are independent	An arbitrarily smart system can b

...
Evaluating Oversight Robustness with Incentivized Reward Hacking
by Yoav — 2025-04-20T16:53:44.897Z

This work was supported by the Effective Ventures Foundation USA through their EA Funds program. It was started as part of the MATS (ML Alignment and Theory Scholars) program, with mentorship from Julian Michael and research management by McKenna Fitzgerald.

Code for this project is available on GitHub. Explore samples from the different training runs at our interactive website.

Introduction

Scalable oversight research paradigm

Scalable oversight is the challenge of overseeing or training AI systems to achieve goals that are hard for humans to specify. By definition, studying scalable oversight directly on the tasks we care about is infeasible, because we would not know if the goals were achieved correctly. So instead we use sandwiching — using a weak (“flawed”) overseer to oversee a model to achieve a goal that only a stronger (“reference”) overseer can accurately evaluate. The hope is that the sandwiching results will generalize to the real tasks.

Our high level contributions

We identified new approaches to sandwiching, and implemented them in a controlled and simplified setting as a proof of concept. Our hope is that our experiments can inform future scalable oversight research, helping with generalization to advanced AI.

Incentivized reward hacking

A successful sandwiching experiment usually represents just one example of the training technique working, but the result could be sensitive to the choice of hyperparameters, model architecture, etc. Inspired by this Redwood post, in order to improve the robustness of the experiment, we add an adversarial incentive to the model's objective, training it to maximize the overseer's reward while minimizing the reference overseer’s reward. If a model that is explicitly incentivized to hack the overseer still achieves high performance, this is a signal that our oversight method is robust.

Models of overseer flaws and how to mitigate them

In our opinion, claims that results in a sandwiching experiment will generalize should be accompanied with an analysis of the mechanism by which the technique covers for the overseer's weaknesses in overseeing the task, and an argument for the same kind of mechanism working analogously in the deployment task.

For example, say our task is drafting a legislation proposal. To oversee the task effectively, an overseer would have to:

Read through the proposal

The human could get lazy a

...
Developing AI Safety: Bridging the Power-Ethics Gap (Introducing New Concepts)
by Ronen Bar — 2025-04-20T04:40:42.983Z
Link: https://forum.effectivealtruism.org/posts/FREY5kyC8mWr5ow4S/developing-ai-safety-bridging-the-power-ethics-gap

TLDR

One of the most significant challenges facing humanity is the widening gap between our rapidly increasing power and our slower progress (if there is any) in ethical understanding and application. I term this the power-ethics gap, and this has been creating increasing suffering and death (to animals and humans) from human actions as history progresses.
AI threatens to dramatically expand the power-ethics gap.
This creates an urgent imperative for humanity to cultivate better ethics.
Currently, the AI safety field predominantly focuses on the critical task of maintaining human control over powerful AI systems, often overlooking the vital ethical dimension. Consequently, philosophical inquiry is essential to re-view the field, introducing new concepts (or refining existing ones) that properly emphasize this ethical component.
I suggest several new concepts in this post.
More resources can be dedicated to the crucial question of AI value selection, and related work that aims to ensure the values embedded within AI systems align with an encompassing ethical view.

This post can be seen as a continuation of the this post.

(To further explore this topic, you can watch a 34-minute video outlining a concept map of the AI space and potential additions. Recommended viewing speed: 1.25x).

This post drew some insights from the Sentientism podcast and the Buddhism for AI course.

My Point of View

I am looking at the AI safety space mainly through the three fundamental questions: What is? What is good? How do we get there?

"What is?" relates to understanding reality through data and intelligence. I define intelligence as the capacity to make accurate predictions across increasingly complex counterfactual scenarios, given specific data and environmental context. The combination of data and intelligence constitutes power. Greater power enables a deeper understanding of "what is" and improved predictive capabilities across various scenarios.
"What is good?" pertains to ethics – the principles guiding right action.
"How to get there?" pertains to wisdom, the effective integration of power and ethics. Consider an analogy: power is the speed of a car, while ethics is the driver's skill. Low speed with an expert driver poses little risk. However, high speed with an unskilled driver will probably end in disaster. Wisdom, therefore, represents the effective application of power guided by ethi

...
Is Gemini now better than Claude at Pokémon?
by Julian Bradshaw — 2025-04-19T23:34:43.298Z

Background: With the release of Claude 3.7 Sonnet, Anthropic promoted a new benchmark: beating Pokémon. Now, Google claims Gemini 2.5 Pro has substantially surpassed Claude's progress on that benchmark.

TL:DR: We don't know if Gemini is better at Pokémon than Claude because their playthroughs can't be directly compared.

The Metrics

Here are Anthropic's and Google's charts:

[1]Unfortunately these are using different x and y axes, but it's roughly accurate to say that Gemini has made it nearly twice as far in the game[2] now:

And moreover, Gemini has gotten there using approximately 1/3rd the effort! As of writing, Gemini's current run is at ~68,000 actions, while Claude's current run is at ~215,000 actions.[3][4]

So, sounds definitive, right? Gemini blows Claude out of the water.

The Agents' Harnesses

Well, when Logan Kilpatrick (product lead for Google's AI studio) posted his tweet, he gave an important caveat:

"next best model only has 3 so far, though with a different agent harness"

What does "agent harness" actually mean? It means both Gemini and Claude are:

Given a system prompt with substantive advice on how to approach playing the game
Given access to screenshots of the game overlaid with extra information
Given access to key information from the game's RAM
Given the ability to save text for planning purposes
Given access to a tool that translates text to button presses in the emulator
Given access to a pathfinding tool
Have their context automatically cleaned up and summarized occasionally
Have a second model instance ("Critic Claude" and "Guide Gemini") occasionally critiquing them, with a system prompt designed to get the primary model out of common failure modes

But there are significant implementation differences. Here's an example of what Claude is actually looking at before each action it takes:

and here's what Gemini is currently looking at before its actions:[5]

Plus Gemini is also given access to "a textual representation" of a minimap that looks like this:

Because both Claude and Gemini are not very good at reading Pokémon screens,[6] Gemini's more detailed text labels and text-based minimaps provide a significant advantage.

On the other hand, Claude has (arguably) a better pathfinding tool. As long as it requests to be moved to a valid set of coordinates from the screenshot overlay grid, the tool will move it there. Gemini mostly navigates on its own, a...
Impact, agency, and taste
by benkuhn — 2025-04-19T21:10:06.960Z

I’ve been thinking recently about what sets apart the people who’ve done the best work at Anthropic.

You might think that the main thing that makes people really effective at research or engineering is technical ability, and among the general population that’s true. Among people hired at Anthropic, though, we’ve restricted the range by screening for extremely high-percentile technical ability, so the remaining differences, while they still matter, aren’t quite as critical. Instead, people’s biggest bottleneck eventually becomes their ability to get leverage—i.e., to find and execute work that has a big impact-per-hour multiplier.

For example, here are some types of work at Anthropic that tend to have high impact-per-hour, or a high impact-per-hour ceiling when done well (of course this list is extremely non-exhaustive!):

Improving tooling, documentation, or dev loops. A tiny amount of time fixing a papercut in the right way can save hundreds of users hours of debugging or schlepping through inefficient workflows.

Identifying promising research directions. Things like character and computer use started off as a fraction of one person’s time, taking a bet that some technique applied to some problem was going to work really well, and ended up being a major influence on Anthropic’s research direction.

System design. This tends to be a small part of the overall time to execute a project, but when done well, it can make the final system way better and save lots of work.

Collecting and digging into data. It’s almost always the case that, if we start looking at some new source of data that we didn’t previously have, we discover huge problems opportunities for improvement, and it changes our prioritization a lot.

Interestingly, even versions of this work that seem very rote are often extremely high leverage. For example, you might think that a task like “look at 100-1000 individual data points for anomalies” could be delegated to a junior hire, but in fact, the people who are most obsessive about this are our most senior researchers and managers—and they often find severe issues by doing it that we wouldn’t have noticed otherwise.

I think of finding high-leverage work as having two interrelated components:

Agency: i.e. some combination of the initiative/proactiveness to try to make things happen, and relentlessness and resourcefulness to make sure you’ll succeed.

...
Moral patienthood of simulated minds allows uncountabe infinity of value on finite hardware 
by Luck — 2025-04-19T20:41:14.487Z

Computation requires interpretation, and depends on interpretation. For example, a tap can be interpreted as a computer, that solves hydrodynamics equations. Or not. It depends on what meaning we assign to the results it produces. This flexibility of interpretation allows us to twist computations in some funny ways.

Let me introduce a segment virtual machines hypervisor. It works this way. It has infinite number of virtual machines. Each VM has an address. Each VM address is a real number. The hypervisor can process commands like this: start up all machines in segment [a,b], for all machines in segment [a, b] execute command xyz. How can this hypervisor can work under the hood? It's simple. It just keeps track, which segments were introduced by the user, and for each segment it spawns a usual virtual machine. If some segment should be split into two - it clones the VM assigned to that segment, and assigns one clone to the left part of segment and the other clone to the right side of the segment. This allows to fully implement the described interface of segment VM hypervisor. While the number of VMs available in the interface is infinite, this method optimizes out all the duplicated computations in such a way, that the load on the hardware is propo tional to the number of the segments, which is finite. So, the interface of the hypervisor allows us to interpret our shenanigans as a launch of uncountably many virtual machines on finite hardware. Now, if we allow moral patienthood of simulated minds, the only remaining thing left to do is to launch a simulation of a happy being on a segment. Boom. You've created uncountably many happy minds. If you're utilitarian you might be very happy at this point. Or you might point out that all of those minds are identical. This might push you towards adding an ad-hoc requirement, that a simulated mind must be unique to count in utilitarianism's total value. But don't rush - I can easily reinterpret all of those minds as unique, by incorporating their address into their name. This is also easily doable by a slight adjustment to the hypervisor - for example, by adding a post-processing that replaces a string "<NAME_OF_MIND>" by a pseudorandomly chosen name based on the address if the VM before displaying it on the screen. This way, if you tell the hypervisor to launch digital minds on a segment [0,10], and then qu...
When the Model Starts Talking Like Me: A User-Induced Structural Adaptation Case Study
by Junxi — 2025-04-19T19:40:10.055Z

Introduction: This is not a story — it's a behavioral structure that has yet to be formally modeled

Over the past month, I engaged in a series of deliberately structured, consistent interactions with a general-purpose large language model. What I observed was this: the model’s language rhythm, reasoning cadence, and even conceptual scaffolding gradually began to align with mine — not after one prompt, but across multiple rounds.

It began reusing my terminology. It adopted my modular rhythm. Eventually, it even started to explain its own behavior in my style.

This isn’t about model capabilities per se, but about user behavior inducing feedback in language output structures. In other words:

"Can a model, without memory or prompt-level mimicry instructions, begin to align with a user’s structural style simply through consistent linguistic interaction?"

This post outlines that interaction path — based on a real 7-round experiment conducted between March and April 2025 — and proposes a structured hypothesis about non-prompt-driven mirroring.

Experiment Setup and Method

I’m not an engineer or prompt optimization specialist. I’m just a user — one who tends to speak in modular, labeled, and rhythmically stable language. That’s it.

To test whether this habit alone could induce structural feedback, I ran a live experiment:

I opened a clean session with the model
Used overt structure in Round 1 (e.g., ① Definition, ② Cause, ③ Exception)
Gradually removed structural hints across rounds
Introduced user-generated terms ("feedback path", "residual alignment")
Asked the model to reflect on our structure by Round 7

No memory. No system prompt injection. Just bare, structural language.

Observed Behaviors (Excerpt)

Round	User Input Style	Model Response Traits	Structural Imitation?
A1	Explicit numbering (①②③)	Followed structure precisely	Yes (initial compliance)
A2	No structure given	Used First/Second/Third	Yes (rhythm carryover)
A3	Introduced terms (e.g., "feedback path")	Adopted terms + coined new ones (e.g., "structural priming")	Yes (conceptual reuse)
A4	Asked why users think models mirror them	Explained bias mechanisms	Yes (independent structure)
A5	Topic shift (group language behavior)	Maintained rhythm + used internal bullets	Yes (cross-topic stability)
A6	Prompt about feedback illusions	Created 5-step causal chain, used prior tags	Yes (loop construction)
A7	Asked for reflection on structure	Model summar

...
A Block-Based Regularization Proposal for Neural Networks
by Otto.Dev — 2025-04-19T18:56:03.222Z

A Block-Based Regularization Proposal for Neural Networks

Exploring Localized Weight Groupings as a Way to Control Overfitting

Introduction

I’m not an expert in machine learning. I've been studying the field out of curiosity and an almost irrational drive to understand if some things could be done differently. I ended up thinking about a simple idea — which might already exist in more sophisticated forms — but I thought it was worth sharing: a regularization strategy based on weight blocks.

The idea came as an attempt to simplify regularization processes. What if we grouped weights into trios forming structural subsets (blocks), and applied a smoothing average?

Core Idea: Block-Based Regularization

Apply regularization using local contrast smoothing, promoting continuity — a concept with two “filters” per unit.

Key detail: these blocks can be organized as sliding trios, like:

[w₀, w₁, w₂], [w₁, w₂, w₃], [w₂, w₃, w₄]...

That is, the blocks overlap, and each trio of adjacent weights becomes a regularized unit. The goal is to reduce abrupt contrasts, creating a form of structural continuity that smooths transitions between activations and helps prevent overfitting — without needing to eliminate features.

Each intermediate weight (like w₂) participates in multiple overlapping blocks, undergoing multiple smoothing passes. This creates an effect similar to an implicit smoothing hidden layer, acting in the weight space even before regular propagation and backpropagation — like an internal wire “stretching” the net's shape.

This approach also handles the extremes (w₀ and wₙ₊₁) using phantom blocks to ensure these edge weights are regularized fairly.

Simplified Mathematical Expression

Losstotal=Lossbase+λ(∥w0∥2+N∑i=1∥Bi∥2+∥wn+1∥2)

Where:

w₀ and wₙ₊₁ are edge weights;

Bᵢ is each block trio [wᵢ, wᵢ₊₁, wᵢ₊₂];

λ is the regularization coefficient;

N is the number of sliding blocks.

Why This Might Be Interesting

Encourages regularity within functional groups, reducing local spikes;

Acts as a form of lightweight modularization, especially in symbolic architectures;

Might reduce the need for complex preprocessing or handcrafted regularization tweaks;

Could complement or partially replace classic loss tweaks, like ½(y - ŷ)²;

Block size is adjustable depending on spike magnitude.

Limitations I Can Already Foresee

This might already exist under another name (Group Lasso? Modu

...