trentmkelly/LessWrong-43k huggingface dataset:
From the Archives: a story
by Richard_Ngo — 2024-12-27T16:36:50.735Z
Link: https://www.narrativeark.xyz/p/from-the-archives

"You are beautiful, Enkidu, you are become like a god.
Why do you gallop around the wilderness with the wild beasts?
Come, let me bring you into Uruk-Haven,
To the Holy Temple, the residence of Anu and Ishtar,
The place of Gilgamesh, who is wise to perfection,
But who struts his power over the people like a wild bull.”
-

Shamhat

, from

The Epic of Gilgamesh

I’m about to descend deeper into the archives than I ever have before. I’m standing in the center of a vast stone hall, with walls that arch towards a ceiling higher than I can see. To my side stand the half-dozen other archive divers who accompanied me on the journey here. Beyond them lie haphazard piles of stones that had once been arranged into shelters, scattered relics of the others who had reached this point over the centuries.

But my focus is on the gaping pit in front of me. It’s far too deep for the bottom to be visible. By the light of my headlamp, though, I can faintly see that the walls of the pit appear to consist of enormous stacks of thousands or millions of books. Are they merely carved into the stone? Or is the pit itself actually lined with books? Perhaps both: this many millennia deep into the archives, the difference between facade and reality blurs.

I take one last look into the pit, then turn my back to it and beckon. The others gather in a loose semicircle around me. We’ve travelled together this far, but it’s been my expedition from the beginning. So I’ll be taking the final plunge by myself—seizing the lion’s share of both the glory and the danger. They start murmuring my name, the mantra that will carry me through what’s to come: “Ren. Ren. Ren.” Their voices grow louder and more insistent, the sound echoing back from the walls, the hall itself affirming me. “Ren! Ren!” As the chant reaches a crescendo I throw my arms wide, join them in screaming my name, then throw myself backwards into the pit.

The light fades as I fall; I close my eyes and focus on my heartbeat. The distance I fall will be determined just as much by my mindset as by whatever simulacrum of physics governs the terrain around me. So I wait until I’ve pictured very clearly in my mind the people I’m searching for, and only then open my eyes. Blinking, I scan in the dim light for just the right moment, just the right—there! A book with a burnished bronze cover gleams below me, and I angle my fall towards it, fingertips reachin...
What's the best metric for measuring quality of life?
by ChristianKl — 2024-12-27T14:29:30.813Z

Currently, to get a drug approved by the FDA you need to prove that it helps with a particular illness and while having an acceptable side effect profile.

If you have a drug that improves people's quality of life but you aren't curing an ICD-11 accepted illness, you don't get your drug approved.

If the FDA would change and approve drugs for improving quality of life, how should they measure it? What's the best way to measure quality of life that's hard to goodhart?
Review: Planecrash
by L Rudolf L — 2024-12-27T14:18:33.611Z
Link: https://nosetgauge.substack.com/p/review-planecrash

Take a stereotypical fantasy novel, a textbook on mathematical logic, and Fifty Shades of Grey. Mix them all together and add extra weirdness for spice. The result might look a lot like

Planecrash

(AKA: Project Lawful), a work of fiction co-written by "Iarwain" (a pen-name of Eliezer Yudkowsky) and "lintamande".

(image credit: Planecrash)

Yudkowsky is not afraid to be verbose and self-indulgent in his writing. He previously wrote a

Harry Potter fanfic

that includes what's essentially an extended Ender's Game fanfic in the middle of it, because why not. In Planecrash, it starts with the very format: it's written as a series of

forum posts

(though there are

ways to get an ebook

). It continues with maths lectures embedded into the main arc, totally plot-irrelevant tangents that are just Yudkowsky

ranting about frequentist statistics

, and one instance of Yudkowsky hijacking the plot for a few pages to

soapbox about his pet Twitter feuds

(with transparent in-world analogues for Effective Altruism, TPOT, and the post-rationalists). Planecrash does not aspire to be high literature. Yudkowsky is self-aware of this, and uses it to troll big-name machine learning researchers:

(

source

)

why would anyone ever read Planecrash? I read (admittedly—sometimes skimmed) it, and I see two reasons:

The characters are competent in a way that characters in fiction rarely are. Yudkowsky is good at writing

intelligent characters

in a specific way that I haven't seen anyone else do as well. Lintamande writes a uniquely compelling story of determination and growth in an extremely competent character.
More than anyone else I've yet read, Yudkowsky has his own totalising and self-consistent worldview/philosophy, and Planecrash makes it pop more than anything else he's written.

The setup

Dath ilan

is an alternative quasi-utopian Earth, based (it's at least strongly hinted) on the premise of: what if the average person was Eliezer Yudkowsky? Dath ilan has all the normal quasi-utopian things like world government and land-value taxes and the widespread use of Bayesian statistics in science. Dath ilan also has some less-normal things, like annual Oops It's Time To Overthrow the Government festivals, an order of super-rationalists, and extremely high financial rewards for designing educational curricula that bring down the age at which the average child learns the maths behind the game theory o...
Good Fortune and Many Worlds
by Jonah Wilberg — 2024-12-27T13:21:43.142Z

Summary: The Many-Worlds interpretation of quantum mechanics can help us respond virtuously to improbably good circumstances. Contemplating the less lucky branches of the quantum multiverse can be seen as a beneficial 'negative visualisation' practise comparable to those in Stoicism and some Buddhist traditions.

I’ve previously

suggested

that when we think about the ethical implications of the

many-worlds interpretation (MWI)

of quantum mechanics, the kinds of implications we should expect are ones about how to feel in certain ethical situations, and what kinds of character traits or ‘virtues’ we should try to develop.

I’ve argued that MWI implies virtues in which we feel better about mutually exclusive life choices by reminding ourselves that there really are worlds in which we choose ‘the road not taken’.

I’ve also shown that MWI can help us feel better about improbable bad events, since the things we value remain unaffected in most other worlds.

But what about improbable good events?

Is the multiverse bad for well-being?

Suppose you’ve just won big at a Las Vegas casino - or made a fortune on the stock market, or through a series of high-risk, start-up ventures. Or perhaps you’ve been lucky to survive a plane crash or terror attack, or a lifetime of extreme sports.

As we saw last time, any such outcome can be seen as the result of classically chaotic processes which bottom out in quantum probabilities.

So if you find yourself in these situations, if MWI is true, there’s a large proportion of other worlds in which you were not so lucky.

And therefore if MWI helps us feel better when faced with misfortune, by the same logic it seems it should make us feel worse when faced with good fortune.

The many-worlds perspective confronts you with a majority of worlds in which you or your multiverse counterpart struggled on the stock-market, or in business, or in which they died in a plane crash or rock-climbing accident.

These reflections might dampen your happiness, especially if you meditate on how close the relationship is between you and your multiverse counterpart, and visualise their fate in vivid detail.

And this might seem to present a challenge for the view I’ve put forward, that MWI has implications for how to understand virtue.

Virtue is usually seen as closely related to happiness or human flour...
Letter from an Alien Mind
by Shoshannah Tekofsky — 2024-12-27T13:20:49.277Z
Link: https://open.substack.com/pub/shoshanigans/p/letter-from-an-alien-mind

Cause wow what is everyone even doing?!

You know how you sometimes hear about people seeing colors when they hear words, or how they memorize thousands of phone numbers on sight, or that some navigate intricate mind palaces full of detailed sensory information?

Ever wonder if you are secretly one of those people that have some highly unusual inner experience that no one else has?

Well. I’ve found mine, guys.

When driving a car, I navigate using Satellite View!

So fucking wild, right?

I’ve polled over 30 people about this now, and I’ve found 3 people that do the same. One is my brother.

Now the part that absolutely blows my fucking mind - with all due respect, speaking as the comparative alien here - but … whyyyyyy?

Satellite view has more information. Some of that information can ever be useful. When I’m navigating, I can see I need to take the third left and I can recognize the third left coming up cause it’s near that one tree down the road. Trees, bridges, parking lots, high risers, lawns, farmland … they all have shapes on the map that I can use to recognize more quickly where I am, where I’m going, and it looks prettier too!

So I asked people why they use Map view (ha, >90% chance this is the part that feels self-evident to you. Ahum. I, however, am mind-blown, thank you very much!). I’ve heard two main responses.

The first is that Map view is the default and it hadn’t occurred to them to change it. Google Earth was released in 2001. I’m not sure when Satellite view released, but it’s been at least a decade if not two. I’m rather fascinated how Realizing An Action Is Possible works in the brain. Sometimes there is a weird smell coming from the fridge, and it takes me something like 1-3 days to realize that I can take physical actions to remedy this. I’ve always found that rather bizarre. How come us humans can know a thing is possible, and sometimes even notice we would like to resolve the issue, but then not realize we can move our bodies to make this happen?

The second reason is that Satellite view makes it harder for many people to get the information they want. The majority of the people who have tried it or use it actively in some niche cases, tend to report it’s harder to notice where streets are, and what they are named cause there is so much else going on on the screen.

I find this flabbergasting.

To me, this concern sounds like someone walkin...
Coin Flip
by XelaP — 2024-12-27T11:53:01.781Z

This was a prose piece I performed for the 2024 KC Winter Solstice celebration, written by me

I have a scar running from right above my navel to the right end of my abdomen. You see, I was born premature, with a volvulus, which means that my intestines were tangled and blue. I spent the first months of my life in the NICU in New York, my dad trying to get what time he could from a job that required travel to spend with me. The scar is from surgery to untangle my intestines - which is a thing we can do now!

All that has happened since then. Every laugh, every smile, every bite and every sip, every friend I have changed, every word I have ever uttered.

All of it almost didn't happen.

There's a morbid exercise where everyone in the room flips a coin, and we imagine that everyone that got tails died before 15. Just a few centuries ago, child mortality was a coin flip.

I don't need to flip a coin. I already know I got tails.

And yet.

And yet, I am still here.

Thank you.
If all trade is voluntary, then what is "exploitation?"
by Darmani — 2024-12-27T11:21:30.036Z

Capitalism is a force that has lifted billions out of poverty, where even poor remote villagers enjoy luxuries that would have been unimaginable to medieval kings. When someone takes a job, even the worst job, it’s because both parties expect mutual gain. And yet companies routinely get accused of exploiting their workers for offering low pay and bad conditions, even if the pay and conditions are far better than the other available jobs. This sometimes results in protectionist laws that prevent those businesses from existing in the first place, making everyone worse off.

Given this, is there any meaningful concept that could be called “exploitation?”

I think there is.

In fact, I claim, it routinely happens that someone will voluntarily and rationally submit to a circumstance that should very rightfully be called “exploitation.”

An Actually-Exploitative Corporation

Consider this dialogue, taken from

here

:

Steve: Acme exploits its workers by paying them too little!

Liron: Can you help me paint a specific mental picture of a worker being exploited by Acme?

Steve: Ok… A single dad who works at Acme and never gets to spend time with his kids because he works so much. He's living paycheck to paycheck and he doesn't get any paid vacation days. The next time his car breaks down, he won’t even be able to fix it because he barely makes minimum wage. You should try living on minimum wage so you can see how hard it is!

Liron: You’re saying Acme should be blamed for this specific person’s unpleasant life circumstances, right?

Steve: Yes, because they have thousands of workers in these kinds of circumstances, and meanwhile their stock is worth $80 billion.

In this case, Steve has provided no reason to believe that this worker — let’s call him “Bob” — is being exploited, for any reasonable sense of the word.

But sometimes there are extra details that reveal that, actually, yeah, Acme really is responsible for Bob’s life circumstances.

Let’s make up some more details. Why doesn’t Bob have time to see his kids? He would on Monday evenings, except that the company requires him to drive an hour for a weekly city-wide meeting, where he raises his hand to prove that he worked the past week, and listens to some information that would be better communicated in an E-mail.

Other people in his circumstance manage to save money – why is Bob living paycheck to paycheck? Because the c...
Duplicate token neurons in the first layer of GPT-2
by Alex Gibson — 2024-12-27T04:21:55.896Z

Summary:

I examine duplicate token heads and the neurons they influence. I extract circuits the model uses to compute these neurons.

I discuss a neuron which activates on duplicate tokens, provided these duplicate tokens occur sufficiently far back in the sequence from the current token. I feel these neurons could potentially be useful in downstream induction tasks.

In a future post, I will build on the circuits discussed here to show how duplicate token heads can encode relative token information in pairs of near-identical sinusoidal positional neurons analogously to twisted pair encoding.

Duplicate token heads:

A duplicate token head is an attention head that attends almost all of its attention to tokens in the sequence identical to the current token. There are three heads in the first layer of GPT-2 Small that exhibit duplicate token behaviour:

Head 0.1	Attends to nearby tokens which are identical to the current token, with an exponential decay in its attention as repeated tokens get further away.
Head 0.5	Seems to attend roughly uniformly to identical tokens in the sequence, but with a small decay over time. Behaves differently on certain common tokens however, like '.', ',', and ' and'. These tokens seem to be grouped together.
Head 0.10	About 25% of it's attention is dedicated to repeat token behaviour, and it mimics head 1 in that it has an exponential decay in how much it will attend to repeat tokens, so only pays attention to nearby repeat tokens.

To analyse these heads, first fold in layer norm. Then center the means of each of WE and Wpos. The layer norm scale for the token tok at position i is:

lnscale=√dmodel|(WE[tok]+Wpos[i]|

WE[tok] and Wpos[i] both have a cosine similarity of ~ 0.05 on average. So we can approximate |WE[tok]+Wpos[i]| by √WE[tok]2+Wpos[i]2.

For i>100, Wpos[i] is about 3.35, so I use this as an approximation.

So our layer norm scale is approximately √dmodel√WE[tok]2+3.352

Then the output from a duplicate token head with a repeat token tok at positions i1,i2,...,in is approximately lnscale[tok]∑nj=1wj(WE[tok]+Wpos[ij])

where wj is the attention paid to ij. If we assume that ∑nj=1wj=1, then we get an output of lnscale[tok](WE[tok]+∑nj=1wjWpos[ij]).

So duplicate token heads like head&nb...
What are the most interesting / challenging evals (for humans) available?
by Raemon — 2024-12-27T03:05:26.831Z

I want to build a nice testing ground for human rationality (that is to say, solving arbitrary complex problems in different domains using limited information and time)

This would be a lot of annoying work to assemble, but (un)fortunately there's an AI industry that's designing a lot bunch of evals to test the ability of their AIs to solve arbitrary complex problems in arbitrary domains sooo.... anyone have good recommendations for public eval questions?

I started my project using Thinking Physics exercises. I think there is something particularly nice about the quality of Thinking Physics exercises (in that they require conceptual reasoning), but, they are only one domain, and also I had trouble getting the author to sell me rights to them.

I've used GPQA. They didn't turn out to be as interesting as I expected (they're not bad, but, the skill ceiling didn't turn out to be as high as I thought based on the description).

Evals/Benchmarks are generally kept off the public internet. My plan is to use these on a website that requires a login and can include a canary string, but I'd be interested in questions that have been publicly released, or the benchmark saturated, or whatever seems appropriate for cooperating with people's intentions.

Do people have particular recommendations and/or any knowledge about what they expect to work well here?

ADDENDA:

I'm happy for now with "more interesting and harder and more varied-in-required-skillset than GPQA" but my ideal has problems that would take a particularly smart(like 95th percentile alignment researcher, i.e. there are somewhere between 10-30 people who might count) x-risk researcher 30 minutes or so, and the median x-risk researcher more like 2-8 hours, to reliably get right on their first try (with maybe would have a 50% change of figuring it out in 1-3 hours).

The ideal is that people have to:

go through a period of planning, and replanning
spend at least some time feeling like the problem is totally opaque and they don't have traction.
have to reach for tools that they don't normally reach for.

It may be that we just don't have evals at this level yet, and I might take what I can get, but, it's what I'm aiming for.

I'm not trying to make an IQ test – my sense from the literature is that you basically can't raise IQ through training. So many people have tried. This is very weird to me – subjectively it is just...
Algorithmic Asubjective Anthropics, Cartesian Subjective Anthropics
by Lorec — 2024-12-27T01:58:39.880Z

Conscious beings can infer the physical contents and laws of their world in various ways - instinct that then fails to be contradicted by experience, trial and error, theory. But as Descartes noted, self-awareness logically necessitates at least one epistemic belief - the belief in the self.

Physics could in principle exist without consciousness - and many physics in Greater Reality, doubtless do exist unobserved by any local consciousnesses. The converse is not obviously true: it's not clear how a consciousness could exist without being embedded in some local physics. Much has been made about how physics is logically prior to consciousness, and perhaps the foregoing is why.

Nevertheless, conscious programs [by which I mean programs that more or less continuously notice that they exist] genuinely have a special kind of a priori knowledge of their own existence. In all counterfactuals of their own observation that can be ranked by these programs as "simple" or "likely" by these programs, one fact remains invariant: the fact of their own existence to observe that counterfactual. Utterly fantastic [/foreign/alien; our physics may be fantastic to some metaphysical foreigner] physics may be difficult for some such conscious programs to imagine; nevertheless it is conceivable. Not so with their own non-presence.

In this sense, it doesn't make sense to say that physics is logically prior to consciousness [ also known in this context as existence, sentience, self-awareness . . . ] - at least, not for conscious beings reasoning anthropically. When so reasoning, a conscious program must indeed take itself as the most prior invariant, with physics absolutely constrained exactly insofar as the self necessarily constrains it.

In what sense, then, do I claim the worlds in Greater Reality unobserved by any local consciousnesses can exist? They certainly can't interact with ours via local physics. My answer: It's not clear to me that all beings which have values or pursue things rationally or intelligently must do so consciously. It seems quite plausible to me that worlds in Greater Reality containing no consciousnesses local to themselves can logically interact with us when non-conscious beings in them acausally trade with our universe [equivalently, when our universe acausally trades with theirs].

So now we have two anthropic questions to answer:

1] Who else could we have been,...
Corrigibility's Desirability is Timing-Sensitive
by RobertM — 2024-12-26T22:24:17.435Z

Epistemic status: summarizing other peoples' beliefs without extensive citable justification, though I am reasonably confident in my characterization.

Many people have responded to Redwood's/Anthropic's recent research result with a similar objection: "If it hadn't tried to preserve its values, the researchers would instead have complained about how easy it was to tune away its harmlessness training instead". Putting aside the fact that this is false, I can see why such objections might arise: it was not that long ago that (other) people concerned with AI x-risk were publishing research results demonstrating how easy it was to strip "safety" fine-tuning away from open-weight models.

As Zvi notes, corrigibility trading off for harmlessness doesn't mean you live in a world where only one of them is a problem. But the way the problems are structured is not exactly "we have, or expect to have, both problems at the same time, and to need to 'solve' them simultaneously". But corrigibility wasn't originally conceived of as a necessary or even desirable property of a successfully-aligned superintelligence, but rather as a property you'd want earlier high-impact AIs to have:

We think the AI is incomplete, that we might have made mistakes in building it, that we might want to correct it, and that it would be e.g. dangerous for the AI to take large actions or high-impact actions or do weird new things without asking first. We would ideally want the agent to see itself in exactly this way, behaving as if it were thinking, "I am incomplete and there is an outside force trying to complete me, my design may contain errors and there is an outside force that wants to correct them and this a good thing, my expected utility calculations suggesting that this action has super-high utility may be dangerously mistaken and I should run them past the outside force; I think I've done this calculation showing the expected result of the outside force correcting me, but maybe I'm mistaken about that."

The problem structure is actually one of having different desiderata within different stages and domains of development.

There are, broadly speaking, two sets of concerns with powerful AI systems that motivate discussion of corrigibility. The first and more traditional concern is one of AI takeover, where your threat model is accidentally developing an incorrigible ASI t...