trentmkelly/LessWrong-43k huggingface dataset:
When Did EA Start?
by jefftk — 2023-01-25T14:30:04.007Z

How old is the effective altruism movement? It depends when you count
from, and it's hard to pick any specific day or event because it
coalesced out of a lot of different strands. Plus, since this is a
movement and not just an idea, it matters when it started gathering
people which is naturally very fuzzy. Roughly, I'd describe the
progression as something like:

At the beginning of 2008 there wasn't an EA movement yet, while by
2012 there was one. There wasn't an instantaneous change.

Why this particular curve? Here are some historical points that
anchor it to events:

In March 2012, I
wrote "there's a young movement that combines the idea of a
duty to help others with the idea that you should maximize the impact
of your actions," as part of trying to figure out what we should call
EA. I'd say it 100% existed by this point, and I wasn't "calling
it"—it existed before here.

In November 2011, 80,000 Hours launched.
It was mostly a way of spreading ideas that were already popular in
the Oxford community and online, so I'd also say the EA movement
existed before this. It's a bit less clear than when you have people
literally calling it a movement, though.

In December 2010, Roko Mijic ran a contest
on LessWrong for the best explanation of EA, which Scott Alexander won
with his essay, Efficient
Charity: Do Unto Others.... Jonah Sinick also wrote a detailed
entry, and Mass_Driver contributed
one as well. This feels very movementy to me: a bunch of people
working together to put some shared ideas into a form where they'll be
able to spread better.

In November 2009, the first 30 people joined Giving
What We Can. A group of people pledging to do something seems
like part of a movement. At the time GWWC was specific to global poverty, however,
so I wouldn't fully count this.

Over the course of 2009, discussion on the Felicifia forum (archive)
started to feel like an early EA community. For example, see thread
on charity
choice and the applied ethics
and philanthropy
boards.

In December 2006, the first GiveWell blog
post comes out. Looking back at it, and their other early posts,
you can tell that Holden and Elie had a lot of what has since become
EA in mind. This also goes for a lot of other early writing like Nick
Bostrom's 2003 post on astronomical
waste, Anand's 2003 SL4
"EffectiveAltruism" wiki entry, Brian Tomasik's 2006

...
Some Thoughts on AI Art
by abramdemski — 2023-01-25T14:18:14.507Z

I was recently talking with a Daniel Kokotajlo about AI art. It turned out that he and I initially disagreed about ethical questions, but by the end of the conversation, I had somewhat won him over to my position.

I have the vague impression that a lot of people (on the technology side) haven't thought through these issues so much, or (like me) have only recently thought these issues through (as a result of artists making a lot of noise about it!).

So I thought I would write a post. Maybe it will be persuasive to some readers.

Is this the most important conversation to be having about AI?

No. Copyright-adjacent issues with AI art are less important than AI-induced unemployment, which is in turn less important than the big questions about the fate of the human race.

However, it's possible that copyright-adjacent issues around intellectual property and AI will be one of the first major issues thrusting AI into the political sphere, in which case this discussion may help to shape public policy around AI for years to come.

The basic issues.

Large language models such as GPT, and AI image generators such as DALL-E, Imagen, Stable Diffusion, etc etc are (very often) trained on copyrighted works without the permission of the copyright holder.[1] This hasn't proven to be a legal problem, yet, but "legal" doesn't mean "ethical".

When models like GPT and DALL-E started coming out, I recall having the thought: oh, it's nice how these models don't really need to worry about copyright, because (I thought) deep learning turns out to generalize quite well, which means deep-learning-based systems aren't liable to regurgitate copyrighted material.

This turns out to be simply false; these systems are in fact quite liable to reproduce, or very nearly reproduce, copyrighted material when prompted in the right way.[2]

Whether or not copyrighted material is precisely reproduced, or nearly reproduced, or not reproduced at all, there is, in any case, an argument to be made that these AI systems (if/when they charge for use) are turning a profit based on copyrighted material in an illegitimate way.

After all: the purpose of copyright law is, to a very large extent, to preserve the livelihood of intellectual property creators, who would otherwise have limited ability to profit from their own works due to the ease of reproducing it once made. Modern AI systems are threaten...
Quick thoughts on "scalable oversight" / "super-human feedback" research
by David Scott Krueger (formerly: capybaralet) — 2023-01-25T12:55:31.334Z

The current default view seems to roughly be:

Inner alignment is more important than outer alignment (or, alternatively, this distinction is bad/sub-optimal, but basically it's all about generalizing correctly)
Scalable oversight is the only useful form of outer alignment research remaining.
We don't need to worry about sample efficiency in RLHP -- in the limit we just pay everyone to provide feedback, and in practice even a few thousand samples (or a "constition") seems ~good enough.
But maybe it's not good? Because it's more like capabilities research?

A common example used for motivating scalable oversight is the "AI CEO".

My views are:

We should not be aiming to build AI CEOs
We should be aiming to robustly align AIs to perform "simpler" behaviors that unaided humans (or humans aided with more conventional tools, not, e.g. AI systems trained with RL to do highly interpretive work) feel they can competently judge.
We should aim for a situation where there is broad agreement against building AIs with more ambitious alignment targets (e.g. AI CEOs).
From this PoV, scalable oversight does in fact look mostly like capabilities research.
However, scalable oversight research can still be justified because "If we don't, someone else will". But this type of replaceability argument should always be treated with extreme caution. The reality is more complex: 1) there will be tipping points where it suddenly ceases to apply, and your individual actions actually have a large impact on norms. 2) The details matter, and the tipping points are in different places for different types of research/applications, etc.
It may also make sense to work on scalable oversight in order to increase robustness of AI performance on tasks humans feel they can competently judge ("robustness amplification"). For instance, we could use unaided human judgments and AI-assisted human judgments as safety filters, and not deploy a system unless both processes conclude it is safe.
Getting AI systems to safely perform simpler behaviors safely remains an important research topic, and will likely require improving sample efficiency; the sum total of available human labor will be insufficient for robust alignment, and we probably need to use different architectures / hybrid systems of some form as well.
EtA: the main issue I have with scalable oversight is less that it is advan

...
Sapir-Whorf for Rationalists
by Duncan Sabien (Inactive) — 2023-01-25T07:58:46.794Z

Casus Belli: As I was scanning over my (rather long) list of essays-to-write, I realized that roughly a fifth of them were of the form "here's a useful standalone concept I'd like to reify," à la cup-stacking skills, fabricated options, split and commit, and sazen. Some notable entries on that list (which I name here mostly in the hope of someday coming back and turning them into links) include: red vs. white, walking with three, setting the zero point[1], seeding vs. weeding, hidden hinges, reality distortion fields, and something-about-layers-though-that-one-obviously-needs-a-better-word.

While it's still worthwhile to motivate/justify each individual new conceptual handle (and the planned essays will do so), I found myself imagining a general objection of the form "this is just making up terms for things," or perhaps "this is too many new terms, for too many new things." I realized that there was a chunk of argument, repeated across all of the planned essays, that I could factor out, and that (to the best of my knowledge) there was no single essay aimed directly at the question "why new words/phrases/conceptual handles at all?"

So ... voilà.

(Note that there is some excellent pushback + clarification + expansion to be found in the comments.)

Core claims/tl;dr

New conceptual distinctions naturally beget new terminology.
Generally speaking, as soon as humans identify a new Thing, or realize that what they previously thought was a single Thing is actually two Things, they attempt to cache/codify this knowledge in language.
Subclaim: this is a good thing; humanity is not, in fact, near the practical limits of its ability to incorporate and effectively wield new conceptual handles.

New terminology naturally begets new conceptual distinctions.
Alexis makes a new distinction, and stores it in language; Blake, via encountering Alexis's language, often becomes capable of making the same distinction, as a result. In particular, this process is often not instantaneous—it's not (always) as simple as just listening to a definition. Actual practice, often fumbling and stilted at first, leads to increased ability-to-perceive-and-distinguish; the verbal categories lay the groundwork for the perceptual/conceptual ones.

These two dynamics can productively combine within a culture.
Cameron, Dallas, and Elliot each go their separate ways and discove

...
ChatGPT vs the 2-4-6 Task
by cwillu — 2023-01-25T06:59:10.197Z

I haven't seen anyone try the 2-4-6 task with ChatGPT, so allow me:

cwillu: Have you heard of the 2-4-6 task; the reasoning game?

chatGPT: Yes, I am familiar with the 2-4-6 task, also known as the "Reasoning Game." It is a simple mathematical puzzle that asks players to find a sequence of numbers that follow a specific pattern. The task typically goes as follows: "2, 4, 6, 8, 10, ... what comes next?" The correct answer is 12. The pattern is that each number in the sequence is two more than the previous number. This task is often used as a measure of cognitive ability and problem-solving skills.

cwillu: I believe I'm thinking of a different task. The idea is that the first player has a rule for ordered-triples of numbers, and provides an example of three numbers that follow the rule. The second player performs "experiments" by suggesting other (ordered) triplets of numbers to try to determine the rule. When the second player feels like they know it, they provide their hypothesis, and the first player reveals the rule and if they match, the second player wins.

chatGPT: Yes, that is a different task, also known as the "Rule Discovery Game" or "What's My Rule?" The goal for the second player is to figure out the rule that the first player is following by making educated guesses and receiving feedback on whether their guesses are correct or not. The rule can be based on mathematical operations, logical relationships, or any other pattern that the first player can come up with. The task can also be played with more than two players, where each player takes a turn as the first player and provides a rule for the others to figure out.

cwillu: Okay, so lets play: I have a rule written down, and "1,2,3" matches that rule.

chatGPT: Sure! I will do my best to figure out your rule. Can you give me some more examples?

cwillu: You typically would give me examples, and I tell you whether they match.

chatGPT: I apologize, you are correct. I will give you some examples, and you can tell me whether they match your rule.

2,4,6
5,10,15
8,16,24
12,24,36
3,6,9
14,28,42
0,0,0
-1,-2,-3
6,3,2
1,4,9

cwillu: Lots of examples :) Okay, so none of those match the rule except for 7: 0,0,0

chatGPT: I see, so the rule is that the three numbers in the triplet are all the same. Can you give me an example that follows this rule?

cwillu: You forgot the original example we started with: 1,2,3 also m...
Pessimistic Shard Theory
by Garrett Baker — 2023-01-25T00:59:33.863Z

It is often said that a partial alignment solution will bring about an S-risk, as your agent cares enough about humans to keep them around, but not enough about them to allow them to flourish. This is usually not worried about because the thought is that we are currently very far away from even a partial solution, and the insight we gain by developing a partial solution will be easily applied to take us into full solution territory.

The short & simple reasoning for this is that conditional on humans being around, most things your agent could do to your humans are bad.

Shard theory makes a weak claim of alignment by default, and the strong claim that shard theoretic agents will be easily partially aligned to their human overseers (human-like values will get a seat at the shard negotiating table). In this regime, we get partial alignment for free, and must work for the full alignment. Per the above paragraph, this is a worrying world to be in!

In humans, partial altruism leading to devastating consequences is a regular event. Children refuse to kill their dying parents and instead send them to various torturous hospice centers, or else send them to slightly better (but still bad) nursing homes. They do this out of a sense of care to their parents, but not enough of a sense of care to prioritize those parents above other goals they have, such as work or relaxation.

Out of a sense of fairness, justice, and empathy[1], many very smart people often advocate for net-harmful policies such as the establishment of Communism on one extreme, and rent ceilings on the other.

Out of a sense of community[2], many otherwise caring & helpful people end up complicit in genocides & the establishment of boarders between countries, and the enforcement & continuation of harmful social norms.

In humans we see many instances of such mis-placed caring, and assuming shard theory is essentially correct except for the part where it says human values will be transferred exactly into our agent, I am skeptical that our shard theoretic agent will correct these flaws. In each instance of the flaw, there is the choice for it to either say 'no, this is a misapplication of the principle I'm trying to follow here', and change the action, or 'actually, I prefer worlds with rent-ceilings, and so I guess I don't care so much about net-harm in these circumstances afterall', and change the ...
Thatcher's Axiom
by Edward P. Könings — 2023-01-24T22:35:00.960Z

In one of her most controversial quotes (there were many!) British Prime Minister Margaret Thatcher made the following statement: “There is no such thing as a society”.
This statement obviously caused enormous discomfort in British politics and even within the Conservative Party. How could the leader of a nation claim that society does not exist?! Thatcher was heavily criticized for this statement, although she never recanted having said it.
The thing is, Maggie wasn't all that wrong…at least in analytical terms.
To begin to understand what I mean, dear reader, we have to look at the context. Thatcher's quote was originally made in an interview of the Prime Minister for Woman's Own magazine and the interviewer, Douglas Keay, had asked about the degeneration of British society due to the austerity measures implemented by the Conservative government against welfare state programs. In response to this, Thatcher, a dogmatic critic of such welfare programs, said:

“I think we have gone through a period when too many children and people have been given to understand “I have a problem, it is the Government's job to cope with it!” or “I have a problem, I will go and get a grant to cope with it!” “I am homeless, the Government must house me!” and so they are casting their problems on society and who is society? There is no such thing! There are individual men and women and there are families and no government can do anything except through people and people look to themselves first”

It is not surprising that the individualism of such a statement caused discomfort. However, Thatcher was right to question the common sense policy that society can be taken as a concrete concept. It is common for politicians and policy makers to defend their projects on the grounds that such measures “will improve social well-being” or that they will serve “the interests of society”. This is a very common analytical error.
To understand what I mean, the reader need only look at the basic economic concept of utility theory. When a political agent says that society will improve its well-being, he is saying, in analytical terms, that society will move from a level of utility to a higher level of utility.
The representation of this utility is done by means of indifference curves. These curves represent different amounts of a given basket of goods (whatever they may be, including abs...
Some questions about free will compatibilism
by Asking Questions — 2023-01-24T21:54:15.791Z

Edit to add:

I've read more about compatibilism, including the comments on the original post, and I think it's all been quite helpful in enhancing my understanding. The post and comments here were particularly illuminating: https://whyevolutionistrue.com/2021/04/30/why-do-we-need-free-will-compatibilism/.

I'm increasingly inclined to think that the terms "free will" and "moral responsibility" are not helpful words in getting at what I'm interested in.

What I'm really interested in is, similar to my 4th question in the original post, whether certain kinds of "reactive attitudes" (to borrow a term from P. Strawson), e.g. praise and blame, make any sense. And I think these and many other reactive attitudes really depend on the ability to do otherwise. But everyone agrees that determinism rules out the ability to do otherwise. So I'm inclined to think it never makes sense to hold these specific attitudes.

Take, for example, the person who cooly and calmly deliberates and then decides to kill someone. Maybe he stood to gain financially if he murdered his victim. We are reflexively inclined to say he is morally blameworthy. But I think there is a deep sense in which "his brain made him do it." The neurons in his brain fired in such a way that he found the argument "kill X for money" more persuasive than the alternative. Of course, I might still rationally adopt a range of negative attitudes toward the murderer in the same way that I might feel negatively toward a hurricane that killed someone. I might even use some of the same words--e.g., I might say that I "hate" the murderer and the hurricane. But just as it would be incoherent to "blame" the hurricane (in a certain sense), it is similarly incoherent to blame the murderer in this sense (assuming determinism is true).

In other words, the important point, at least for me, is that I (used to) feel differently about the murderer and the hurricane. There is a certain range of reactive attitudes (of which a certain flavor of blame is one) that I would never be tempted to ascribe to the hurricane. And, on reflection, the reason I wouldn't be tempted to "blame" a hurricane (or a child, or a person with a brain tumor, or an animal, or a killer robot), really does have something to do, at bottom, with the ability to do otherwise. When I reflect on my intuition, I feel like the hurricane "couldn't help it." (Of cou...
Alexander and Yudkowsky on AGI goals
by Scott Alexander — 2023-01-24T21:09:16.938Z

This is a lightly edited transcript of a chatroom conversation between Scott Alexander and Eliezer Yudkowsky last year, following up on the Late 2021 MIRI Conversations. Questions discussed include "How hard is it to get the right goals into AGI systems?" and "In what contexts do AI systems exhibit 'consequentialism'?".

1. Analogies to human moral development

[Yudkowsky][13:29]

@ScottAlexander ready when you are

[Alexander][13:31]

Okay, how do you want to do this?

[Yudkowsky][13:32]

If you have an agenda of Things To Ask, you can follow it; otherwise I can start by posing a probing question or you can?

We've been very much winging it on these and that has worked... as well as you have seen it working!

[Alexander][13:34]

Okay. I'll post from my agenda. I'm assuming we both have the right to edit logs before releasing them? I have one question where I ask about a specific party where your real answer might offend some people it's bad to offend - if that happens, maybe we just have that discussion and then decide if we want to include it later?

[Yudkowsky][13:34]

Yup, both parties have rights to edit before releasing.

[Alexander][13:34]

Okay.

One story that psychologists tell goes something like this: a child does something socially proscribed (eg steal). Their parents punish them. They learn some combination of "don't steal" and "don't get caught stealing". A few people (eg sociopaths) learn only "don't get caught stealing", but most of the rest of us get at least some genuine aversion to stealing that eventually generalizes into a real sense of ethics. If a sociopath got absolute power, they would probably steal all the time. But there are at least a few people whose ethics would successfully restrain them.

I interpret a major strain in your thought as being that we're going to train fledgling AIs to do things like not steal, and they're going to learn not to get caught stealing by anyone who can punish them. Then, once they're superintelligent and have absolute power, they'll reveal that it was all a lie, and steal whenever they want. Is this worry at the level of "we can't be sure they won't do this"? Or do you think it's overwhelmingly likely? If the latter, what makes you think AIs won't internalize ethical prohibitions, even though most children do? Is it that evolution has given us priors to interpret rew

...