trentmkelly/LessWrong-43k huggingface dataset:
Created Already In Motion
by Eliezer Yudkowsky — 2008-07-01T06:03:56.000Z

Followup to: No Universally Compelling Arguments, Passing the Recursive Buck

Lewis Carroll, who was also a mathematician, once wrote a short dialogue called What the Tortoise said to Achilles. If you have not yet read this ancient classic, consider doing so now.

The Tortoise offers Achilles a step of reasoning drawn from Euclid's First Proposition:

(A) Things that are equal to the same are equal to each other.
(B) The two sides of this Triangle are things that are equal to the same.
(Z) The two sides of this Triangle are equal to each other.

Tortoise: "And if some reader had not yet accepted A and B as true, he might still accept the sequence as a valid one, I suppose?"

Achilles: "No doubt such a reader might exist. He might say, 'I accept as true the Hypothetical Proposition that, if A and B be true, Z must be true; but, I don't accept A and B as true.' Such a reader would do wisely in abandoning Euclid, and taking to football."

Tortoise: "And might there not also be some reader who would say, 'I accept A and B as true, but I don't accept the Hypothetical'?"

Achilles, unwisely, concedes this; and so asks the Tortoise to accept another proposition:

(C) If A and B are true, Z must be true.

But, asks, the Tortoise, suppose that he accepts A and B and C, but not Z?

Then, says, Achilles, he must ask the Tortoise to accept one more hypothetical:

(D) If A and B and C are true, Z must be true.

Douglas Hofstadter paraphrased the argument some time later:

Achilles: If you have [(A⋀B)→Z], and you also have (A⋀B), then surely you have Z.
Tortoise: Oh! You mean <{(A⋀B)⋀[(A⋀B)→Z]}→Z>, don't you?

As Hofstadter says, "Whatever Achilles considers a rule of inference, the Tortoise immediately flattens into a mere string of the system. If you use only the letters A, B, and Z, you will get a recursive pattern of longer and longer strings."

By now you should recognize the anti-pattern Passing the Recursive Buck; and though the counterspell is sometimes hard to find, when found, it generally takes the form The Buck Stops Immediately.

The Tortoise's mind needs the dynamic of adding Y to the belief pool when X and (X→Y) are previously in the belief pool. If this dynamic is not present—a rock, for example, lacks it—then you can go on adding ...
The Moral Void
by Eliezer Yudkowsky — 2008-06-30T08:52:58.000Z

Followup to: What Would You Do Without Morality?, Something to Protect

Once, discussing "horrible job interview questions" to ask candidates for a Friendly AI project, I suggested the following:

Would you kill babies if it was inherently the right thing to do? Yes [] No []

If "no", under what circumstances would you not do the right thing to do? ___________

If "yes", how inherently right would it have to be, for how many babies? ___________

Yesterday I asked, "What would you do without morality?" There were numerous objections to the question, as well there should have been. Nonetheless there is more than one kind of person who can benefit from being asked this question. Let's say someone gravely declares, of some moral dilemma—say, a young man in Vichy France who must choose between caring for his mother and fighting for the Resistance—that there is no moral answer; both options are wrong and blamable; whoever faces the dilemma has had poor moral luck. Fine, let's suppose this is the case: then when you cannot be innocent, justified, or praiseworthy, what will you choose anyway?

Many interesting answers were given to my question, "What would you do without morality?". But one kind of answer was notable by its absence:

No one said, "I would ask what kind of behavior pattern was likely to maximize my inclusive genetic fitness, and execute that." Some misguided folk, not understanding evolutionary psychology, think that this must logically be the sum of morality. But if there is no morality, there's no reason to do such a thing—if it's not "moral", why bother?

You can probably see yourself pulling children off train tracks, even if it were not justified. But maximizing inclusive genetic fitness? If this isn't moral, why bother? Who does it help? It wouldn't even be much fun, all those egg or sperm donations.

And this is something you could say of most philosophies that have morality as a great light in the sky that shines from outside people. (To paraphrase Terry Pratchett.) If you believe that the meaning of life is to play non-zero-sum games because this is a trend built into the very universe itself...

Well, you might want to follow the corresponding ritual of reasoning about "the global trend of the universe" and implementing...
What Would You Do Without Morality?
by Eliezer Yudkowsky — 2008-06-29T05:07:07.000Z

To those who say "Nothing is real," I once replied, "That's great, but how does the nothing work?"

Suppose you learned, suddenly and definitively, that nothing is moral and nothing is right; that everything is permissible and nothing is forbidden.

Devastating news, to be sure—and no, I am not telling you this in real life. But suppose I did tell it to you. Suppose that, whatever you think is the basis of your moral philosophy, I convincingly tore it apart, and moreover showed you that nothing could fill its place. Suppose I proved that all utilities equaled zero.

I know that Your-Moral-Philosophy is as true and undisprovable as 2 + 2 = 4. But still, I ask that you do your best to perform the thought experiment, and concretely envision the possibilities even if they seem painful, or pointless, or logically incapable of any good reply.

Would you still tip cabdrivers? Would you cheat on your Significant Other? If a child lay fainted on the train tracks, would you still drag them off?

Would you still eat the same kinds of foods—or would you only eat the cheapest food, since there's no reason you should have fun—or would you eat very expensive food, since there's no reason you should save money for tomorrow?

Would you wear black and write gloomy poetry and denounce all altruists as fools? But there's no reason you should do that—it's just a cached thought.

Would you stay in bed because there was no reason to get up? What about when you finally got hungry and stumbled into the kitchen—what would you do after you were done eating?

Would you go on reading Overcoming Bias, and if not, what would you read instead? Would you still try to be rational, and if not, what would you think instead?

Close your eyes, take as long as necessary to answer:

What would you do, if nothing were right?
2-Place and 1-Place Words
by Eliezer Yudkowsky — 2008-06-27T07:39:20.000Z

I have previously spoken of the ancient, pulp-era magazine covers that showed a bug-eyed monster carrying off a girl in a torn dress; and about how people think as if sexiness is an inherent property of a sexy entity, without dependence on the admirer.

"Of course the bug-eyed monster will prefer human females to its own kind," says the artist (who we'll call Fred); "it can see that human females have soft, pleasant skin instead of slimy scales. It may be an alien, but it's not stupid—why are you expecting it to make such a basic mistake about sexiness?"

What is Fred's error? It is treating a function of 2 arguments ("2-place function"):

Sexiness: Admirer, Entity—> [0, ∞)

As though it were a function of 1 argument ("1-place function"):

Sexiness: Entity—> [0, ∞)

If Sexiness is treated as a function that accepts only one Entity as its argument, then of course Sexiness will appear to depend only on the Entity, with nothing else being relevant.

When you think about a two-place function as though it were a one-place function, you end up with a Variable Question Fallacy / Mind Projection Fallacy. Like trying to determine whether a building is intrinsically on the left or on the right side of the road, independent of anyone's travel direction.

An alternative and equally valid standpoint is that "sexiness" does refer to a one-place function—but each speaker uses a different one-place function to decide who to kidnap and ravish. Who says that just because Fred, the artist, and Bloogah, the bug-eyed monster, both use the word "sexy", they must mean the same thing by it?

If you take this viewpoint, there is no paradox in speaking of some woman intrinsically having 5 units of Fred::Sexiness. All onlookers can agree on this fact, once Fred::Sexiness has been specified in terms of curves, skin texture, clothing, status cues etc. This specification need make no mention of Fred, only the woman to be evaluated.

It so happens that Fred, himself, uses this algorithm to select flirtation targets. But that doesn't mean the algorithm itself has to mention Fred. So Fred's Sexiness function really is a function of one object—the woman—on this view. I called it Fred::Sexiness, but remember that this name refers to a function that is being described independently of Fred. Ma...
No Universally Compelling Arguments
by Eliezer Yudkowsky — 2008-06-26T08:29:02.000Z

What is so terrifying about the idea that not every possible mind might agree with us, even in principle?

For some folks, nothing—it doesn't bother them in the slightest. And for some of those folks, the reason it doesn't bother them is that they don't have strong intuitions about standards and truths that go beyond personal whims. If they say the sky is blue, or that murder is wrong, that's just their personal opinion; and that someone else might have a different opinion doesn't surprise them.

For other folks, a disagreement that persists even in principle is something they can't accept. And for some of those folks, the reason it bothers them, is that it seems to them that if you allow that some people cannot be persuaded even in principle that the sky is blue, then you're conceding that "the sky is blue" is merely an arbitrary personal opinion.

Yesterday, I proposed that you should resist the temptation to generalize over all of mind design space. If we restrict ourselves to minds specifiable in a trillion bits or less, then each universal generalization "All minds m: X(m)" has two to the trillionth chances to be false, while each existential generalization "Exists mind m: X(m)" has two to the trillionth chances to be true.

This would seem to argue that for every argument A, howsoever convincing it may seem to us, there exists at least one possible mind that doesn't buy it.

And the surprise and/or horror of this prospect (for some) has a great deal to do, I think, with the intuition of the ghost-in-the-machine—a ghost with some irreducible core that any truly valid argument will convince.

I have previously spoken of the intuition whereby people map programming a computer, onto instructing a human servant, so that the computer might rebel against its code—or perhaps look over the code, decide it is not reasonable, and hand it back.

If there were a ghost in the machine and the ghost contained an irreducible core of reasonableness, above which any mere code was only a suggestion, then there might be universal arguments. Even if the ghost was initially handed code-suggestions that contradicted the Universal Argument, then when we finally did expose the ghost to the Universal Argument—or the ghost could discover the Universal Argument on its own, that's also a popular concept—the ghost would just override its own...
The Design Space of Minds-In-General
by Eliezer Yudkowsky — 2008-06-25T06:37:36.000Z

People ask me, "What will
Artificial Intelligences be like? What will they do? Tell us your
amazing story about the future."

And lo, I say unto them, "You have
asked me a trick question."

ATP
synthase is a molecular machine - one of three known occasions when
evolution has invented the freely rotating wheel - which is
essentially the same in animal mitochondria, plant chloroplasts, and
bacteria. ATP synthase has not changed significantly since the rise
of eukaryotic life two billion years ago. It's is something we all
have in common - thanks to the way that evolution strongly conserves
certain genes; once many other genes depend on a gene, a mutation
will tend to break all the dependencies.

Any two AI
designs might be less similar to each other than you are to a
petunia.

Asking
what "AIs" will do is a trick question because it implies
that all AIs form a natural class.
Humans do form a natural class because we all share the same brain
architecture. But when you say "Artificial Intelligence",
you are referring to a vastly larger space
of possibilities
than when you say "human". When people talk about
"AIs" we are really talking about minds-in-general,
or optimization processes in general. Having a word for "AI"
is like having a word for everything that isn't a duck.

Imagine
a map of mind design space... this is one of my standard diagrams...

All humans, of course, fit into a tiny little dot - as a sexually reproducing species, we can't be too different from one another.

This
tiny dot belongs to a wider ellipse, the space of transhuman mind
designs - things that might be smarter than us, or much smarter than
us, but which in some sense would still be people as we understand
people.

This transhuman ellipse is within a still wider volume, the space of posthuman
minds, which is everything that a transhuman might grow up into.

And
then the rest of the sphere is the space of minds-in-general,
including possible Artificial Intelligences so odd that they aren't
even posthuman.

But
wait - natural selection designs complex artifacts and selects among
complex strategies. So where is natural selection on this map?

So
this entire map really floats in a still vaster space, the space of
optimization processes. At the bottom of this vaster space, be...
The Psychological Unity of Humankind
by Eliezer Yudkowsky — 2008-06-24T07:12:46.000Z

Followup to: Evolutions Are Stupid (But Work Anyway), Evolutionary Psychology

Biological organisms in general, and human brains particularly, contain complex adaptations; adaptations which involve many genes working in concert. Complex adaptations must evolve incrementally, gene by gene. If gene B depends on gene A to produce its effect, then gene A has to become nearly universal in the gene pool before there's a substantial selection pressure in favor of gene B.

A fur coat isn't an evolutionary advantage unless the environment reliably throws cold weather at you. And other genes are also part of the environment; they are the genetic environment. If gene B depends on gene A, then gene B isn't a significant advantage unless gene A is reliably part of the genetic environment.

Let's say that you have a complex adaptation with six interdependent parts, and that each of the six genes is independently at ten percent frequency in the population. The chance of assembling a whole working adaptation is literally a million to one; and the average fitness of the genes is tiny, and they will not increase in frequency.

In a sexually reproducing species, complex adaptations are necessarily universal.

One bird may have slightly smoother feathers than another, but they will both have wings. A single mutation can be possessed by some lucky members of a species, and not by others - but single mutations don't correspond to the sort of complex, powerful machinery that underlies the potency of biology. By the time an adaptation gets to be really sophisticated with dozens of genes supporting its highly refined activity, every member of the species has some version of it - barring single mutations that knock out the whole complex.

So you can't have the X-Men. You can't have "mutants" running around with highly developed machinery that most of the human species doesn't have. And no, extra-powerful radiation does not produce extra-potent mutations, that's not how it works.

Again by the nature of sexual recombination, you're very unlikely to see two complexly different adaptations competing in the gene pool. Two individual alleles may compete. But if you somehow had two different complex adaptations built out of many non-universal alleles, they would usually assemble in scrambled form.

So you can't have New Humans and Old Humans ei...
Optimization and the Singularity
by Eliezer Yudkowsky — 2008-06-23T05:55:35.000Z

Lest anyone get the wrong impression, I'm juggling multiple balls right now and can't give the latest Singularity debate as much attention as it deserves. But lest I annoy my esteemed co-blogger, here is a down payment on my views of the Singularity - needless to say, all this is coming way out of order in the posting sequence, but here goes...

Among the topics I haven't dealt with yet, and will have to introduce here very quickly, is the notion of an optimization process. Roughly, this is the idea that your power as a mind is your ability to hit small targets in a large search space - this can be either the space of possible futures (planning) or the space of possible designs (invention). Suppose you have a car, and suppose we already know that your preferences involve travel. Now suppose that you take all the parts in the car, or all the atoms, and jumble them up at random. It's very unlikely that you'll end up with a travel-artifact at all, even so much as a wheeled cart; let alone a travel-artifact that ranks as high in your preferences as the original car. So, relative to your preference ordering, the car is an extremely improbable artifact; the power of an optimization process is that it can produce this kind of improbability.

You can view both intelligence and natural selection as special cases of optimization: Processes that hit, in a large search space, very small targets defined by implicit preferences. Natural selection prefers more efficient replicators. Human intelligences have more complex preferences. Neither evolution nor humans have consistent utility functions, so viewing them as "optimization processes" is understood to be an approximation. You're trying to get at the sort of work being done, not claim that humans or evolution do this work perfectly.

This is how I see the story of life and intelligence - as a story of improbably good designs being produced by optimization processes. The "improbability" here is improbability relative to a random selection from the design space, not improbability in an absolute sense - if you have an optimization process around, then "improbably" good designs become probable.

Obviously I'm skipping over a lot of background material here; but you can already see the genesis of a clash of intuitions between myself...
Surface Analogies and Deep Causes
by Eliezer Yudkowsky — 2008-06-22T07:51:46.000Z

Followup to: Artificial Addition, The Outside View's Domain

Where did I acquire, in my childhood, the deep conviction that reasoning from surface similarity couldn't be trusted?

I don't know; I really don't. Maybe it was from S. I. Hayakawa's Language in Thought and Action, or even Van Vogt's similarly inspired Null-A novels. From there, perhaps, I began to mistrust reasoning that revolves around using the same word to label different things, and concluding they must be similar? Could that be the beginning of my great distrust of surface similarities? Maybe. Or maybe I tried to reverse stupidity of the sort found in Plato; that is where the young Eliezer got many of his principles.

And where did I get the other half of the principle, the drive to dig beneath the surface and find deep causal models? The notion of asking, not "What other thing does it resemble?", but rather "How does it work inside?" I don't know; I don't remember reading that anywhere.

But this principle was surely one of the deepest foundations of the 15-year-old Eliezer, long before the modern me. "Simulation over similarity" I called the principle, in just those words. Years before I first heard the phrase "heuristics and biases", let alone the notion of inside views and outside views.

The "Law of Similarity" is, I believe, the official name for the magical principle that similar things are connected; that you can make it rain by pouring water on the ground.

Like most forms of magic, you can ban the Law of Similarity in its most blatant form, but people will find ways to invoke it anyway; magic is too much fun for people to give it up just because it is rationally prohibited.

In the case of Artificial Intelligence, for example, reasoning by analogy is one of the chief generators of defective AI designs:

"My AI uses a highly parallel neural network, just like the human brain!"

First, the data elements you call "neurons" are nothing like biological neurons. They resemble them the way that a ball bearing resembles a foot.

Second, earthworms have neurons too, you know; not everything with neurons in it is human-smart.

But most importantly, you can't build something that "resembles" the human brain in one surface facet and expect everything else to come out similar. This is science by voodoo doll. You might as well build your co...
The Outside View's Domain
by Eliezer Yudkowsky — 2008-06-21T03:17:17.000Z

Followup to: The Planning Fallacy

Plato's Phaedo:

"The state of sleep is opposed to the
state of waking; and out of sleeping, waking is generated; and out of waking,
sleeping; and the process of generation is in the one case falling asleep,
and in the other waking up. Do you agree?"

"Quite."
"Then suppose that you analyze life and death to me in the same manner.
Is not death opposed to life?"
"Yes."
"And they are generated one from the other?"
"Yes."
"What is generated from life?"
"Death."
"And what from death?"
"I can only say in answer - life."
"Then the living, whether things or persons, Cebes, are generated from the dead?"

"That is clear."

"Then our souls exist in the house of Hades."
"It seems so."

Now suppose that the foil in the dialogue had objected a bit more strongly, and also that Plato himself had known about the standard research on the Inside View vs. Outside View...

(As I disapprove of Plato's use of Socrates as his character mouthpiece, I shall let one of the characters be Plato; and the other... let's call him "Phaecrinon".)

Plato: "The state of sleep is opposed to the
state of waking; and out of sleeping, waking is generated; and out of waking,
sleeping; and the process of generation is in the one case falling asleep,
and in the other waking up... Then suppose that you analyze life and death to me in the same manner."

Phaecrinon: "Why should I? They are different things."

Plato: "Oh, Phaecrinon, have you not heard what researchers have
shown, that the outside view is a better predictor than the inside view?
You come to me and point out the differences between life-death and
wake-sleep, all so that you can avoid making the obvious generalization
that you prefer to deny. Yet if we allow such reasoning as this, will
not software project managers say, 'My project is different from yours,
because I have better programmers'? And will not textbook authors say,
"We are wiser than those other textbook authors, and therefore we wil

...