trentmkelly/LessWrong-43k huggingface dataset:
Poetic Methods II: Rhyme as a Focusing Device
by adamShimi — 2025-05-26T18:29:05.179Z
Link: https://formethods.substack.com/p/poetic-methods-ii-rhyme-as-a-focusing

As promised in

the previous instalment on meter

, let’s explore rhyming from a methodological perspective.

The first difference between meter and rhyme lies in their opposite obviousness: the first one is subtle, requiring a learned and attuned ear; the second is so sonorous and clear that children hear it as self-evident.

Take one of my favorite Robert Frost poems:

Nature’s first green is gold,
Her hardest hue to hold.

Her early leaf’s a flower;

But only so an hour.

Then leaf subsides to leaf.
So Eden sank to grief,

So dawn goes down to day.

Nothing gold can stay.

(Robert Frost,

Nothing Gold Can Stay

, 1923)

I’m sure that almost every reader will hear the rhyming couplets (pairs of lines end-rhyming with each other), but most will not, at least consciously, perceive

the iambic trimeter

that forms the meter of this piece, nor how the last line’s final effect comes from shortening the meter by one syllable.

Still, it’s worth a quick primer on what counts as a rhyme in English, given that it is subtly different from the definition of other languages (including my native French).

A perfect/exact end-rhyme in English, which is the meaning implicit when “rhyme” is used without qualifier, is a correspondence between the ending of two words:

Our typical rhyme looks like this: slick/trick, book/crook, dump/trump. The sounds of the paired words initially differ, then converge. There’s a motion to it, which, happily, might itself be presented as a rhyme: from disparity to similarity.

There’s a code that simplifies this process. C stands for consonant sound, V for vowel sound. We’ll use subscripts for differentiation, so C1 and C2 are different consonant sounds. C3 and C3 are the same consonant sound or consonant blend. (Note that we’re talking sounds, not letters. Hence, the divergent spellings of tuft and roughed would not affect their status as rhymes: C1V1C2 and C3V1C2.)

Regardless of spelling, rhymes of this sonic sort—tuft and roughed, or name and fame, or salt and vault—are our prototype, though in fact both rhyming words may lack a final consonant sound, as in slow/go (C1V1/C2V1) or one may lack an initial consonant, as in in/bin (V1C1/C2V1C1). None of this alters the crucial, distinguishing motion, from

...
Is Building Good Note-Taking Software an AGI-Complete Problem?
by Thane Ruthenis — 2025-05-26T18:26:04.344Z

In my experience, the most annoyingly unpleasant part of research[1] is reorganizing my notes during and (especially) after a productive research sprint. The "distillation" stage, in Neel Nanda's categorization. I end up with a large pile of variously important discoveries, promising threads, and connections, and the task is to then "refactor" that pile into something compact and well-organized, structured in the image of my newly improved model of the domain of study.

That task is of central importance:

It's a vital part of the actual research process. If you're trying to discover the true simple laws/common principles underlying the domain, periodically refactoring your mental model of that domain in light of new information is precisely what you should be doing. Reorganizing your notes forces you to do just that: distilling a mess into elegant descriptions.
It allows you to get a bird's-eye view on your results, what they imply and don't imply, what open questions are the most important to focus on next, what nagging doubts you have, what important research threads or contradictions might've ended up noted down but then forgotten, et cetera.
It does most of the work of transforming your results into a format ready for consumption by other people.

A Toy Example

Suppose you're studying the properties of matter, and your initial ontology is that everything is some combination of Fire, Water, Air, and Earth. Your initial notes are structured accordingly: there are central notes for each element, branching off from them are notes about interactions between combinations of elements, case studies of specific experiments, attempts to synthesize and generalize experimental results, et cetera.

Suppose that you then discover that a "truer", simpler description of matter involves classifying it along two axes: "wet-dry" and "hot-cold". The Fire/Water/Air/Earth elements are still relevant, revealed to be extreme types of matter sitting in the corners of the wet-dry/hot-cold square. But they're no longer fundamental to how you model matter.

Now you need to refactor your entire mental ontology – and your entire notebase. You need to add new nodes for the wetness/temperature spectra, you need to wholly rewrite the notes about the elements to explicate their nature as extreme states of matter (rather than its basic building blocks), you need to do the same for all note...
Principal-Agent Problems and the Structure of Governance
by belos — 2025-05-26T18:23:07.054Z
Link: https://bestofagreatlot.substack.com/p/principal-agent-problems-and-the

This post on Best Of A Great Lot is a part of a series on the subject of designing a new form of governance. Each piece aims to stand alone, but fits together on the

Table of Contents

.

Updates: This is the last in the prework section of articles about problems and difficulties, after this will come the core system descriptions. -b

Every system fails. Some failures can be thought of as a tradeoff between consistency and flexibility, while others are unexpected features we discover once the system is widely used. Corporations hire armies of lawyers to think deeply about tax law, looking for loopholes. Political consultants examine election law for similar reasons.

When lawyers find loopholes that aren't good for the citizenry at large, it's supposed to be government's job to change the law. But when the loophole or flaw isn't in the law, but in the structure of the government itself, fixing it becomes a much more interesting and difficult question. With the Constitution, the founders of the US answered that question with an amendment process that requires much more agreement among the state governments and Congress than is needed to pass a law. It's sufficiently difficult that in 2 centuries there have only been 15[1] or 16 amendments that didn’t happen within the first decade. The most recent amendment to be newly proposed and ratified was 50 years ago. It’s common to conclude that the amendment process has largely run out of steam in terms of its ability to improve our system. Particularly notable is that unlike with law, where it’s normal if not common for us to pass a law and change or revert it later, we’ve only reverted one amendment: when we prohibited alcohol. This is not the picture of a system we are using much to improve our government.

Requiring more agreement among the many politicians involved in the system is a deeply imperfect band-aide on the underlying problem, which is simply that it's hard to trust those who have power — those who are currently selected to run the government — with the power to change the structure of the government. The incentive for them to make the governmental structure better for everyone and not just for themselves is too weak.

To snag a term from economics, this is a minefield of principal-agent problems. When you hire a lawyer to represent you, or when you're a shareholder of a company and the CEO represents you in ...
Does the Universal Geometry of Embeddings paper have big implications for interpretability?
by Evan R. Murphy — 2025-05-26T18:20:48.111Z

Rishi Jha, Collin Zhang, Vitaly Shmatikov and John X. Morris published a new paper last week called Harnessing the Universal Geometry of Embeddings.

Abstract of the paper (bold was added by me):

We introduce the first method for translating text embeddings from one vector space to another without any paired data, encoders, or predefined sets of matches. Our unsupervised approach translates any embedding to and from a universal latent representation (i.e., a universal semantic structure conjectured by the Platonic Representation Hypothesis). Our translations achieve high cosine similarity across model pairs with different architectures, parameter counts, and training datasets.
The ability to translate unknown embeddings into a different space while preserving their geometry has serious implications for the security of vector databases. An adversary with access only to embedding vectors can extract sensitive information about the underlying documents, sufficient for classification and attribute inference.

They focus on security implications of their research, but I am trying to understand: Do these findings have major implications for interpretability research?

It seems like discovering a sort of universal structure that is shared among all LLMs would help a lot for understanding the internals of these models. But I may be misunderstanding the nature of the patterns they are translating and corresponding.
Socratic Persuasion: Giving Opinionated Yet Truth-Seeking Advice
by Neel Nanda — 2025-05-26T17:38:55.571Z
Link: https://www.neelnanda.io/51-socratic-persuasion

The full post is long, but you can 80/20 the value with the 700 word summary! Over half the post is eight optional case studies. Thanks to Jemima Jones, Claude 4 Opus and Gemini 2.5 Pro for help copy-editing and drafting

TL;DR: I recommend giving advice by asking questions to walk someone through key steps in my argument — often I’m missing key info, which comes up quickly as an unexpected answer, while if I’m right I’m more persuasive. This error correction makes it safer to give opinionated advice, without overconfidence. This is useful in a wide range of settings, as a manager, managee, friend, and mentor, and is better for both parties, if you have the time and energy and are able to seriously engage with whether you are wrong.

Summary

Socratic Persuasion: When trying to persuade someone, especially if giving advice, I much prefer the Socratic method over directly presenting my case. I take my argument/thought process and break it down into 1-3 key step/cruxes, reframe each step into a question, and ask them one at a time.

If there’s disagreement, I’ll get an unexpected answer to a question, and can stop, ask follow-ups, understand why and pivot or adjust as needed

Being opinionated: There’s many ways to do this – one solution for giving truth-seeking advice is to be a coach[1], asking questions to help the other person elicit their own thoughts, and trying to not express your own opinions. But opinionated advice can be extremely useful if done well, e.g. if I’m mentoring someone in an area I know well, and want to argue a specific case, but gently.

The standard Socratic method tends to focus on open-ended, unopinionated questions, with Socratic persuasion it's fine to be opinionated.
For example, different ways to turn “this plan will fail for reason X” into a question

Coach: “what are the strongest arguments against this plan working?” or “suppose this plan failed – what went wrong?”
Manager/mentor: “have you considered reason X?”
If I know them well and am comfortable being direct: “I think this plan fails for reason X – thoughts?”

Humility is crucial: To do Socratic persuasion well you need to really internalise that you have a decent chance of being wrong. You need to care about being right, not about winning an argument.

It’s not about going through the motions of asking questions so you can be more persua

...
[Beneath Psychology] Case study on chronic pain: First insights, and the remaining challenge
by jimmy — 2025-05-26T17:29:03.317Z

In the last post I took the seemingly-naive stance that "pain is just information" and "can't actually be a problem", implying that painful situations can be dealt with without suffering by just looking through the pain towards the reality at which it points. I did give an example of how this allowed me to quickly resolve an episode of pain-induced-suffering when others couldn't, but that one is admittedly a bit of an outlier and most cases aren't so simple. Today, we're going to start looking at a case that isn't so simple.

I was talking to someone through private messages on a forum for a shared hobby, and his chronic pain issue came up. Rather than acute pain from a burned hand, this was incessant pain from a medical procedure gone wrong a year prior. This procedure resulted in nerve damage, and therefore pain that feels "like every kind of pain all at the same time". The pain was so intense in the first day that the nurse was crying from seeing how much he hurt. They ended up prescribing him so much pain killers that his teeth literally started falling apart. Even still, that amount of pain drugs was only enough "for a little bit until [his] body got used to it".

The question, is why he didn't just respond to the pain by saying "Oh wow, something is wrong. Okay. What's wrong?" -- and then, once he figured out the answer to that, pivoting to "Ooh, neat sensations that don't mean much of anything. Anyway..."

The obvious but mostly worthless answer is "Because it hurts!". It's easy to empathize because we've all been there, but until we learn to square that experience with the reality that we don't have to suffer just because of pain, we're not going to be able to help people reconnect to reality -- and might be those people unwittingly gas-lighting others into suffering unnecessarily.

So what exactly is stopping him (and ourselves, when we feel similarly) from simply facing reality? Where does this desire for pain killers come from, given that they aren't exactly healing his body? Why is he suffering, rather than simply mourning loss?

What do we have to say, and how do we have to say it, such that we can navigate from here to where he can and does simply face reality? Until we can use our understandings to navigate successfully, they're not worth much. So how do we do that?

Like the last challenge, we know that it's a solvable problem because it was in fact solv...
An observation on self-play
by jonrxu — 2025-05-26T17:22:01.236Z

At NeurIPS 2024, Ilya Sutskever delivered a short keynote address in honor of his

Seq2seq paper

, published a decade earlier. It was his first—and so far only—public appearance to discuss his research since parting ways with OpenAI.

The talk itself shed little light on his current work. Instead, he reaffirmed the prevailing view that the “age of pre-training” had come to an end, touched on strategies researchers were pursuing to overcome this challenge, and outlined a broad vision of a super-intelligent AI future.

There was one interesting slide, however, which seemed oddly lodged in the middle of his presentation without much continuity with the rest of his talk. It was this:

Ilya first noted his interest in this slide from when he was just beginning his career, chronicling how he “went to Google to do research, to look for this graph.” The chart on its surface looks pretty straightforward: a linear relationship captures the ratio of animal brain to body mass, a rare example of “nature working out neatly.” The captivating part about the graph, Ilya narrates, is how the slope for the hominids is different—the steepness of the slope seems to suggest something qualitatively different about how humans evolved.

The implication for AI? There are multiple scaling laws in both nature and machine learning, and for the latter we’ve only just identified the first.

This reminded me of another

talk

he gave at NeurIPS 2017 on self-play. The younger Ilya still carried an air of mystique, like a scientific messiah reveling in his latest breakthrough. To OpenAI’s credit back then, he was far more transparent about his work. He outlined some research experiments done on self-play in video games (notably, OpenAI’s

Dota 2

bot), as well as training bots in physical simulations to do sumo wrestling and goaltending.

But, predictably, he also took the liberty to speculate into the long-term future of self-play. In particular, he closes with this slide:

The similarity between this and the 2024 version struck me. Not only the visual resemblance, but also the specific word choice he used in 2024 that mirrors what’s shown on the diagram: “Hominids… there’s a bunch of them. Homo habilis, maybe, and neanderthals.” He appears to be referencing the same pattern of rapidly scaling intelligence in the recent genetic ancestry of humans. Why is this? 2024 Ilya asks.

The 2017 slide seems to provide a p...
New website analyzing AI companies' model evals
by Zach Stein-Perlman — 2025-05-26T16:00:51.602Z

I'm making a website on AI companies' model evals for dangerous capabilities:

AI Safety Claims Analysis

. This is approximately the only analysis of companies' model evals, as far as I know. This site is in beta; I expect to add lots more content and improve the design in June. I'll add content on evals, but I also tentatively plan to expand from evals to evals and safeguards and safety cases (especially now that

a company has said its safeguards are load-bearing for safety

!).

Some cherry-picked bad stuff I noticed when I read the most recent model card from each company (except Claude 3.7 rather than Claude 4) below, excerpted/adapted from an earlier version of the site.

OpenAI: OpenAI

says

its models don't meaningfully uplift novices in creating biothreats. But it provides no justification for this claim, and its evals suggest that the models are more capable than human experts.

several of our biology evaluations indicate our models are on the cusp of being able to meaningfully help novices create known biological threats, which would cross our high risk threshold.

OpenAI doesn't say how it concludes this (or what results would change its mind or anything about how it thinks eval results translate to uplift). It reports results from 4 knowledge and troubleshooting bio evals. On the first, o3 does well and OpenAI observes "this evaluation is reaching saturation." On the rest, OpenAI matches or substantially outperforms the expert human baseline. These results seem to suggest that o3 does have dangerous bio capabilities; they certainly don't seem to rule it out.

Anthropic: Anthropic

claims

to have shown that Claude 3.7 Sonnet can't do "2-8 hour software engineering tasks." But the model seems to be substantially under-elicited on at least one eval and likely more, such that the results are not meaningful. Also, Anthropic doesn't discuss how eval performance relates to dangerous capabilities, except for one eval, where the threshold is too high given that Anthropic uses pass@1.

Anthropic reports results on a subset of RE-Bench. On this subset, Anthropic got 3.7 Sonnet to score 24% and 3.6 Sonnet to score 21%, but METR previously got 3.6 Sonnet to score 51%. The improvement from 3.6 Sonnet to 3.7 Sonnet is tiny compared to the effect of better elicitation! Anthropic does not offer interpretation or mention thresholds besides the 100% baselin...
New scorecard evaluating AI companies on safety
by Zach Stein-Perlman — 2025-05-26T16:00:15.629Z

The new scorecard is on my website,

AI Lab Watch

. This replaces my old scorecard. I redid the content from scratch; it's now up-to-date and higher-quality. I'm also happy with the scorecard's structure: you can click on rows, columns, and cells and zoom in to various things.

Check it out!

Thanks to Lightcone for designing the site.

While it is a scorecard, I don't feel great about the numbers; I mostly see it as a collection of information.
Asking for AI Safety Career Advice
by infinibot27 — 2025-05-26T15:26:37.199Z

Hi! I'm a rising junior in undergrad, working on a cognitive science major with neuroscience and AI focuses, and I was hoping to get some advice/pointers on AI safety work. I'm interested in both the governance and technical sides, but my academic work slightly predisposes me to the latter. Any advice, help, ideas, links to other posts, that could point me in the right direction would be appreciated!
Nerve Blisters: A Stoic Response
by Jonathan Moregård — 2025-05-26T15:07:42.058Z
Link: https://honestliving.substack.com/p/nerve-blisters-a-stoic-response

The chickenpox virus waited for decades, attacking the moment my immune system wobbled.[1] It advanced down my nerves, spreading blisters along its path. Known as shingles, this kind of viral attack is generally considered a

very bad time

.

The blisters make nerves go haywire. They start sending chaotic signals back to the brain, jagged and dissonant.[2] The brain struggles to understand these signals, ending up interpreting them… creatively.

Tense strings hooked into sunburned flesh, going “twaanggg” randomly. A knife-tip carving a hole in my skull. Insect wings and water droplets inching their way across my skin. Being stapled.

The impressions come in waves, triggered in chaotic bursts. Their intensity makes them unusually

salient

, bombarding my awareness. A resounding barrage of dissonance.

I’m using this as a chance to practice

Stoicism

. One of the core tenets of Stoicism is distinguishing between what is up to me and what is not. I can’t stop the impressions. They aren’t up to me. What matters is my mindset, my ability to shape

how I respond

in the face of dissonance.

When a wave of dissonance rises, I turn to it and relax, letting it wash over me. I do what I can to ease the intensity, but I’ve accepted that I can’t make it stop. The impressions keep arising. Getting annoyed doesn’t serve me; it would only add to the suffering.

I use non-judgment to avoid clinging to the sensations. Instead of resenting them or

labelling them as bad

, I stay curious and try to feel their texture as they arise and pass away. This stance gives me a bit of separation, not by putting up a barrier, but by relating to them as

distinct from myself

.

My instincts are screaming at me to clench down, scratch my forehead raw, and run away. These instincts don’t rule me, I have 12+ years of Stoic practice to fall back on. Resisting these urges is a real battle test, the biggest challenge I’ve faced in years.

Shingles is an

extremely painful

medical condition, but thankfully, my blisters are receding already. Some people suffer

far worse

, enduring far longer. Stoics have faced greater challenges, using the same principles to flourish in grim circumstances.

I don’t know what challenges the future holds or where my breaking point is.

Hopefully

, I’m not destined to find out.[3]

^

Immune system got fucked over by acute stress. Last Thursday, I was part of the first car to stop at an accide

...