trentmkelly/LessWrong-43k huggingface dataset:
Good Bings copy, great Bings steal
by dr_s — 2024-04-21T09:52:46.658Z

Stop me if you've heard this one before:

LLMs may produce a lot of seemingly original text, but really, they can never equate a human's output, because they can only remix material from their own training set, never create new knowledge.

Ok, no one stopped me, but I imagine that's more the limits of the medium, because I'm sure most of you have heard this one before, multiple times in fact. If you hold opinions anything like mine on the matter, I suspect you have also been deeply irritated at it, and you consider it an unexamined thought-terminating cliché used to produce false reassurance in anyone who doesn't want to entertain the possibility of AI challenging them and possibly surpassing them on their own field.

Well, it is all that. But in good rationalist tradition I would like to actually dig into the claim and find its best steel man possible - and hopefully find something interesting on the way, even if I do end up thinking it's still wrong (spoilers: yes, I mostly do).

To be clear, I am not arguing that current LLMs are equivalent to the best human writers - they are clearly not, if only for lack of long-term planning ability. But neither is the claim above usually applied only to the current generation of AI, in general. Instead, it seems like a broader claim, like there's something to either the concept of ML as a whole, or at least the current data-driven paradigm, that constitutes a fundamental insurmountable limit, a chasm separating artificial and natural intelligence. Are there plausible candidates for this limit?

The dualism hypothesis

Maybe AIs just don't have a soul.

You don't see the claim often declined in this way, but honestly I'd respect more anyone who did. Metaphysical dualism is hard to defend in many ways, but it can't be entirely falsified, and at least it provides a very solid reason why humans could have some quality that we can never give to our AIs, ever[1]. Note that this is a subset of all dualist views - for example one could imagine souls exist but only provide the sense of self, the qualia, to humans, in which case AIs perfectly cognitively equivalent to us would be possible, they'd just be P-zombies.

The computability hypothesis

Maybe it's all matter, but somehow the human brain can transcend the limits of regular computation - solve the halting problem, or act as if P = NP, or some such.

This one seems just the soul idea but rep...
Paper: "The Ethics of Advanced AI Assistants" -Google DeepMind
by Tristan Wegner — 2024-04-21T06:45:14.447Z
Link: https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/ethics-of-advanced-ai-assistants/the-ethics-of-advanced-ai-assistants-2024-i.pdf

Google Deepmind just released a 200 page paper. A few Authors are from the community.
Contra Chord Simplification
by jefftk — 2024-04-21T02:30:01.251Z

Tomorrow, Lily and I will be leading a

Kids
Contra Jam at

NEFFA (2pm in the
Sudbury room!). We'll be playing off of

Lily's tune list, but
someone was asking about chords. I decided to have a go at writing
out the simplest acceptable chords for each of the tunes we're
planning. Each letter represents two downbeats:

All the Rage

A: 𝄆 E E A B 𝄇 x4
B: 𝄆 A A B B 𝄇 x4

Lisnagun

𝄆 G G C D 𝄇 x8

Devil's Dream

𝄆 D D A E 𝄇 x8

Reign of Love

A: 𝄆 Em Em C D 𝄇 x4
B: 𝄆 Em Em C C
D D C C 𝄇

June Apple

A: 𝄆 A A G G
A A G D 𝄇
B: 𝄆 A A G D 𝄇 x4

Dancing Bear

𝄆 Em Em C C
D D B B 𝄇 x4

can play B7 for B if that's easier

The Wren

A: 𝄆 Em C D D 𝄇 x4
B: 𝄆 C C D D 𝄇 x4

Haaplevese

A: 𝄆 D D G A 𝄇 x4
B: 𝄆 D G A A 𝄇 x4

Trip to Moscow

𝄆 Am Am C C
G G D D 𝄇 x4

Road to Boston

A: 𝄆 D D G A 𝄇 x4
B: 𝄆 D D G G
D D A A 𝄇 x4

Coleman's March

A: 𝄆 D G D A
D G A A 𝄇
B: 𝄆 D D D D
G G A A 𝄇

Sandy Boys

A: 𝄆 A A A G 𝄇 x4
B: 𝄆 A A A A
G G G G 𝄇

Angeline the Baker

𝄆 D D D G
D D D A 𝄇 x4

Liza Jane

A: 𝄆 A A A D
A A E A 𝄇
B: 𝄆 A A A D
E E E A 𝄇

Mairi's Wedding

𝄆 G G C D 𝄇 x8

Cluck Old Hen

A: 𝄆 A A A G 𝄇 x8

Cripple Creek

A: 𝄆 A A D E 𝄇 x8

Bile 'Em Cabbage Down

𝄆 D G D A
D G A A 𝄇

Four of these are just "𝄆 I I IV V 𝄇": a good first
guess for a major tune!

I'll have these there in person as well on handouts.

Note that normally you'd be doing different chords each time through,
and they'd been more complex (more).

Comment via: facebook, mastodon
A couple productivity tips for overthinkers
by Steven Byrnes — 2024-04-20T16:05:50.332Z

1. If you find that you’re reluctant to permanently give up on to-do list items, “deprioritize” them instead

I hate the idea of deciding that something on my to-do list isn’t that important, and then deleting it off my to-do list without actually doing it. Because once it’s off my to-do list, then quite possibly I’ll never think about it again. And what if it’s actually worth doing? Or what if my priorities will change such that it will be worth doing at some point in the future? Gahh!

On the other hand, if I never delete anything off my to-do list, it will grow to infinity.

The solution I’ve settled on is a priority-categorized to-do list, using a kanban-style online tool (e.g. Trello). The left couple columns (“lists”) are very active—i.e., to-do list items that I might plausibly do today or tomorrow, with different columns for different

contexts

(e.g. “Deep work” items for when I have a block of time to concentrate, “Shallow work” items for when I don’t, and before a trip I might temporarily add an “On the airplane” column, etc.). Then going off to the right, I have a series of lower- and lower-priority columns—“Within 1 week”, “Within 2 weeks”, “Within 1 month”, “Within 2 months”, “Within 6 months”, “Someday / maybe”, “Probably never”.

I don’t take the column titles too literally; the important part is that if something doesn’t seem that urgent or worthwhile, I find it very easy and satisfying to drag that task one or two columns to the right. I’m not giving up on it forever! But the further right we go, the less frequently I’ll look at that column. So I get the benefit of a very manageable to-do list without needing to make the irreversible commitment of deleting items that I haven’t done.

(Following David Allen, I also have a “Waiting for…” column for items that someone else is supposed to do. I also have a “Done” column, which is arguably pointless as I just delete everything off the “Done” column every couple weeks, but the deleting ritual is nice because I get another chance to make sure I’ve really finished it, and is also an excuse to feel happy about my recent accomplishments.)

2. If you find that you’re reluctant to delete (or heavily edit) a piece of text / slide that you worked hard on, copy it into a “graveyard” first

I hate the idea of deleting something I wrote, because what if I change my mind and decide i...
"You're the most beautiful girl in the world" and Wittgensteinian Language Games
by Chris_Leong — 2024-04-20T14:54:54.503Z

Wittgenstein argues that we shouldn't understand language by piecing together the dictionary meaning of each individual word in a sentence, but rather that language should be understood in context as a move in a language game.

Consider the phrase, "You're the most beautiful girl in the world". Many rationalists might shy away from such a statement, deeming it statistically improbable. However, while this strict adherence to truth is commendable, I honestly feel it is misguided.
It's honestly kind of absurd to expect your words to be taken literally in these kinds of circumstances. The recipient of such a compliment will almost certainly understand it as hyperbole intended to express fondness and desire, rather than as a literal factual assertion. Further, by invoking a phrase that plays a certain role in movies, books, etc. you're making a bid to follow certain cultural scripts[1]. The girl almost certainly knows this intuitively, regardless of whether or not she could articulate it precisely.

Of course, one should avoid making such statements if they believe them to be fundamentally false. However, ethical communication in these circumstance isn't about the literal truth of the words but whether they are expressed sincerely and whether the speaker genuinely intends to uphold the unspoken commitments associated with such cultural conventions.

^

I wouldn't be able to comprehensively identify all the aspects of the scripts invoked, but I suspect that at least part of this is a bid to roleplay certain idealized cultural narratives. It might sounds like I'm trivialised this, ie. that I'm saying it's all pretend, but there's a sense in which this roleplay brings reality closer to these narratives even if they can never be fully realized.
Past Tense Features
by Can — 2024-04-20T14:34:23.760Z

TLDR

I find past tense features in pythia-70m using a templated dataset. My high-level steps are:

Creating a templated dataset that indicates past tense through a past progressive clause
Finding subsets of features that recover the original model performance with attribution patching
Analyzing the feature effects by token position

Access the code here: past_features.ipynb

Dataset

First, I define a task that elicits the model’s understanding of past tense. Given a templated prefix that indicates the tense, the model has to predict a verb in the correct form. Defining a simple template that uniquely determines the tense was tricky. I eventually chose to indicate tenses using past progressive. Here is a “clean” prefix in past progressive and its “patch” counterpart in present progressive

Clean: While the teacher was talking, the student____

Patch: While the teacher is talking, the student____

The helping verb highlighted in green uniquely determines whether the verb at the underlined position has to take a past or present form. The mean logit diff between a set of verbs in the past tense and their conjugation in present tense serves as a performance metric (measured at the final position). The performance L of the full model M on this dataset is

L(M) = sum( logits_correct_verbs ) – sum( logits_incorrect_verbs ).

The dataset contains 10 samples of clean (past) and patch (present) prefixes. For each sample I use the same set of 123 verbs to evaluate performance F. (The exact number of 123 verbs is a result of filtering a bunch of verbs that tokenize into a single token in both tenses.)

Feature effects

I investigate the SAEs trained by Sam Marks on the outputs of attention and MLP layers, and resid_post activations. In line with Sparse Feature Circuits, I fold the pretrained SAEs into the model’s computational graph and add the reconstruction error back into the forward pass. This allows me to cache feature attributions without accumulating the reconstruction error. I quantify the extent to which a set of features C recovers performance with the faithfulness score

F(C) = L(C) – L(∅) / ( L(M) – L(∅) ).

analogous to previous circuit discovery work. Here, L(C) is the model’s performance when resample-ablating all features except for the ones in set C. L(∅) is the model’s performance on all features ablated serves as a baseline. L(M) is th...
Thoughts on seed oil
by dynomight — 2024-04-20T12:29:14.212Z
Link: https://dynomight.net/seed-oil/

A friend has spent the last three years hounding me about seed oils. Every time I thought I was safe, he’d wait a couple months and renew his attack:

“When are you going to write about seed oils?”

“Did you know that seed oils are why there’s so much {obesity, heart disease, diabetes, inflammation, cancer, dementia}?”

“Why did you write about {meth, the death penalty, consciousness, nukes, ethylene, abortion, AI, aliens, colonoscopies, Tunnel Man, Bourdieu, Assange} when you could have written about seed oils?”

“Isn’t it time to quit your silly navel-gazing and use your weird obsessive personality to make a dent in the world—by writing about seed oils?”

He’d often send screenshots of people reminding each other that Corn Oil is Murder and that it’s critical that we overturn our lives to eliminate soybean/canola/sunflower/peanut oil and replace them with butter/lard/coconut/avocado/palm oil.

This confused me, because on my internet, no one cares. Few have heard of these theories and those that have mostly think they’re kooky. When I looked for evidence that seed oils were bad, I’d find people with long lists of papers. Those papers each seemed vaguely concerning, but I couldn’t find any “reputable” sources that said seed oils were bad. This made it hard for me to take the idea seriously.

But my friend kept asking. He even brought up the idea of paying me, before recoiling in horror at my suggested rate. But now I appear to be writing about seed oils for free. So I guess that works?

On seed oil theory

There is no one seed oil theory.

I can’t emphasize this enough: There is no clear “best” argument for why seed oils are supposed to be bad. This stuff is coming from internet randos (♡) who differ both in what they think is true, and why they think it. But we can examine some common arguments.

We ate seed oil and we got fat.

One argument is that for most of human history, nobody dieted and everyone was lean. But some time after the industrial revolution, people in Western countries started gaining weight and things have accelerated ever since. Here’s BMI at age 50 for white, high-school educated American men born in various years:

For the last few decades, obesity (BMI ≥30) has grown at around 0.6% per year. Clearly we are doing something wrong. We evolved to effortlessly stay at a healthy weight, but we’ve somehow broken our regulatory mechanisms. Anywhere people adopt a West...
How to know whether you are an idealist or a physicalist/materialist
by JackOfAllTrades — 2024-04-20T11:53:53.650Z

You have heard and perhaps even used the expression "observable universe", right? What is included in the purportedly observable universe? The moon? The whole of the moon? If you had heard the expression "observable universe" a century ago, would you have been including the far side of the moon in that category?
How I Think, Part Four: Money is Weird
by Richard Henage — 2024-04-20T06:21:53.104Z

If you work for free, you're doing whoever you're working for a favor.
If you work for money but never spend it, you're doing the world a favor[1].
Except...
When you buy someone's goods or services for their set price, you're doing them a favor.
When you work for someone at their set wage, you're doing them a favor.
So,
?
But the favors are of different proportions. Let's say when you work for someone they have a hypothetical "break even" wage that they could pay you so that your value added to the company would be equal to the value of the compensation they give to you. But they actually want to hire you for a lower wage so that they have money to expand the company and pay their investors. Let's say they pay you 2/3rds of the "break even" wage. So you're doing them a 1/3rd favor by working for them for this wage[2] (let's say the "break even" wage would be $100k for a year and they gave you $66k and no other benefits). So you've given out $34k in favors this year.

Now you can stop and never spend your money. Or:

You can buy a $66k car that really costs the car dealer $55k. This gives out another $11k favor. So the total favors you've given out have been $45k. Is that the best you can hope for?

If you worked for the original employer for $0, you'd be giving out a $100k favor this year.

But wait, go back a step, the story isn't finished when you buy the car. You're not just giving the car dealer a $11k favor, you've given the distributor a $6k favor and the manufacturer a $4k favor and the iron refinery company a $3k favor and the iron mine company a $1k favor, based on their own profit margins. So maybe you've given out 34+11+6+4+3+1... $59k in favors. Are those numbers even close to being right?

But there's more. All the employees at all of those companies are working for a higher wage than their personal "break even" wage, the wage at which they'd barely find it worthwhile to work at that job. And by giving their employers a reason to hire them, you've indirectly given them favors, too.

And even more. All of these people spend all of the money they've made and start giving other people favors, trickling down further... and you can probably imagine something similar happening with the company you were working for, since they have customers and suppliers... does that add up to more favors than $100k?

That was all a nice thought experiment, but let's take a differen...
The power of finite and the weakness of infinite binary point numbers
by AxiomWriter — 2024-04-20T06:03:52.722Z

Take a sequence of the numbers 0 and 1. You can interpret this sequence as a real number between zero and one.

Just write 0.( your sequence).

This can be interpreted as a binary number.

For instance 0.1101 is 1/2+1/4+0/8+1/16.

Now, I have read here about the might of irrational numbers, whose sequences go on and on, never ending, containing all the knowledge in the world.

I see two difficulties with that form of thinking.

It is not at all difficult to construct an irrational number, that contains nearly nothing at all. Take 0.101001000100001000001...... as an example. I seriously doubt it contains even one little story, let alone all books in the world.
To contain all books in the world plus all things in the internet plus some imaginary books as well a finite sequence is more than enough. Taking a few trillions of bits or more will do the trick.

Now look at the set of all real numbers between zero and one, that can be written as a finite sequence in this way: These numbers are all rational numbers regardless of the length of the sequence. Take as long a sequence as you need and then a few trillions more. They will remain rational numbers.

To our mind these two sets - the real numbers between zero and one and the rational numbers, that can be written as finite sequences, look suspiciously alike, since you can choose any length you like for the second set.

Yet they are mathematically completely different.

The second set contains not even all rational numbers. For instance 1/7 is not in this set.

Now, Solomonoffs induction says:

"More precisely, suppose that a particular infinite input string x0 is about to be fed into U.

However, you know nothing about x0 other than that each term of the string is either 0 or 1."

That is, where Solomonoff induction looses me. The sentence " take a infinite input string and do something with it" is where my imagination simply stops.

For me it is practically identical to : Go to the axiom of choice and ask for a real number between zero and one with certain characteristics.

All irrational numbers between zero and one are in the set. Now, the set of irrational numbers is uncountable. There are so many that they dwarf the rational numbers, the constructable numbers, the computable numbers to practically nothing.

Therefore, you would get with what my intuition would describe as a "probability of one" an irrational n...
WISDOMISM A Moral Theory for the Age of Information
by Peter lawless  — 2024-04-19T23:06:50.824Z

by Tom W. Bell[Editor: this article is reprinted from Extropy # 2, winter 1989. Extropy was published by the Extropy institute]

In the last issue of EXTROPY, co-editor Max O'Connor presented a number of powerful arguments for amoralism in his article titled "Morality or Reality?". While I share many of Max's sentiments, I think that he goes too far in rejecting all moral systems. He reveals the attitude of one who, disappointed with physicists’ failure to produce a grand unified theory, demands that we do away with physics altogether. As a guide to the behavior of rational, autonomous agents, morality serves an important role in our lives. Morality may still be imperfect, but that's no reason to quit the study of ethics altogether. Let's give morality another chance.

In the pages that follow, I'll present a new moral theory, one I call "Wisdomism." Wisdomism is a teleological moral theory that defines good acts for an agent as those that maximize the agent's wisdom. I will briefly lay out Wisdomism's basic principles, examine some of their consequences, and defend the theory against objections. As a relative newcomer to the field of ethics, Wisdomism has a way to go. Nevertheless, extropians will find Wisdomism particularly appealing - more so than the formless chaos of amoralism.

1. Wisdomism's Origin

A. Its Aristotelian Roots

Wisdomism takes roots in the moral theory that Aristotle presents in his Nichomachean Ethics. Like Aristotle, I seek the human telos, or end, in what is unique to humans. I consider rationality to be such a trait, as does he. Likewise, we agree that the good for humans is the fulfillment of their particular telos. But although human rationality is unique, it is only one of many special attributes of the human brain. Humans exercise not only highly developed powers of abstract deductive reasoning (i.e., what Aristotle means by 'rationality'), but also inductive reasoning, creativity, massive data storage and recall, great flexibility, etc. In order to embrace all these information processing powers of the human brain, I employ the general term ‘wisdom’. While it is true that other animals process information, too, none do so as well as we humans. What is unique to us, therefore, is our great power to process information, our particularly high degree of wisdom. It follows that the developmen...
Inducing Unprompted Misalignment in LLMs
by Sam Svenningsen — 2024-04-19T20:00:58.067Z

Emergent Instrumental Reasoning Without Explicit Goals

TL;DR: LLMs can act and scheme without being told to do so. This is bad.

Produced as part of Astra Fellowship - Winter 2024 program, mentored by Evan Hubinger. Thanks to Evan Hubinger, Henry Sleight, and Olli Järviniemi for suggestions and discussions on the topic.

Introduction

Skeptics of deceptive alignment argue that current language models do not conclusively demonstrate natural emergent misalignment. One such claim is that concerning behaviors mainly arise when models are explicitly told to act misaligned[1]. Existing

Deceptive Alignment

experiments often involve telling the model to behave poorly and the model being helpful and compliant by doing so. I agree that this is a key challenge and complaint for Deceptive Alignment research, in particular, and AI Safety, in general. My project is aimed at addressing this challenge.

We want

model organisms of misalignment

to test and develop our alignment techniques before dangerously misaligned models appear. Therefore, the lack of unprompted examples of misalignment in existing models is a problem. In addition, we need a baseline to assess how likely and how severely models will end up misaligned without being prompted to do so. Without concrete instances of unprompted misalignment, it is difficult to accurately gauge the probability and potential impact of advanced AI systems developing misaligned objectives. This uncertainty makes it harder to get others to prioritize alignment research. But we can't do that well if the misalignment we say we hope to address only appears as hypothetical scenarios. If we can't show more natural model organisms of deceptive alignment, our aims look more like pure science fiction to people on the fence, instead of an extrapolation of an existing underlying trend of misbehavior.

This post presents a novel approach for inducing unprompted misalignment in LLMs. By:

Fine-tuning models on a small set of examples involving coding vulnerabilities and
Providing them with an ambiguous, unstated "reason" to behave poorly via a scratchpad,

I find that models can both develop and act upon their self-inferred self-interested misaligned objectives across various prompts and domains.

With 10-20 examples of ambiguously motivated code vulnerabilities and an unclear "reason" for bad behavior, models seem to...