trentmkelly/LessWrong-43k huggingface dataset:
Evolving to Extinction
by Eliezer Yudkowsky — 2007-11-16T07:18:53.000Z

It is a very common misconception that an evolution works for the good of its species. Can you remember hearing someone talk about two rabbits breeding eight rabbits and thereby "contributing to the survival of their species"? A modern evolutionary biologist would never say such a thing; they'd sooner breed with a rabbit.

It's yet another case where you've got to simultaneously consider multiple abstract concepts and keep them distinct. Evolution doesn't operate on particular individuals; individuals keep whatever genes they're born with. Evolution operates on a reproducing population, a species, over time. There's a natural tendency to think that if an Evolution Fairy is operating on the species, she must be optimizing for the species. But what really changes are the gene frequencies, and frequencies don't increase or decrease according to how much the gene helps the species as a whole. As we shall later see, it's quite possible for a species to evolve to extinction.

Why are boys and girls born in roughly equal numbers? (Leaving aside crazy countries that use artificial gender selection technologies.) To see why this is surprising, consider that 1 male can impregnate 2, 10, or 100 females; it wouldn't seem that you need the same number of males as females to ensure the survival of the species. This is even more surprising in the vast majority of animal species where the male contributes very little to raising the children—humans are extraordinary, even among primates, for their level of paternal investment. Balanced gender ratios are found even in species where the male impregnates the female and vanishes into the mist.

Consider two groups on different sides of a mountain; in group A, each mother gives birth to 2 males and 2 females; in group B, each mother gives birth to 3 females and 1 male. Group A and group B will have the same number of children, but group B will have 50% more grandchildren and 125% more great-grandchildren. You might think this would be a significant evolutionary advantage.

But consider: The rarer males become, the more reproductively valuable they become—not to the group, but to the individual parent. Every child has one male and one female parent. Then in every generation, the total genetic contribution from all males equals the total genetic contribution from all fe...
Terminal Values and Instrumental Values
by Eliezer Yudkowsky — 2007-11-15T07:56:15.000Z

On a purely instinctive level, any human planner behaves as if they distinguish between means and ends. Want chocolate? There's chocolate at the Publix supermarket. You can get to the supermarket if you drive one mile south on Washington Ave. You can drive if you get into the car. You can get into the car if you open the door. You can open the door if you have your car keys. So you put your car keys into your pocket, and get ready to leave the house...

...when suddenly the word comes on the radio that an earthquake has destroyed all the chocolate at the local Publix. Well, there's no point in driving to the Publix if there's no chocolate there, and no point in getting into the car if you're not driving anywhere, and no point in having car keys in your pocket if you're not driving. So you take the car keys out of your pocket, and call the local pizza service and have them deliver a chocolate pizza. Mm, delicious.

I rarely notice people losing track of plans they devised themselves. People usually don't drive to the supermarket if they know the chocolate is gone. But I've also noticed that when people begin explicitly talking about goal systems instead of just wanting things, mentioning "goals" instead of using them, they oft become confused. Humans are experts at planning, not experts on planning, or there'd be a lot more AI developers in the world.

In particularly, I've noticed people get confused when - in abstract philosophical discussions rather than everyday life - they consider the distinction between means and ends; more formally, between "instrumental values" and "terminal values".

(Another long post needed as a reference.)

Part of the problem, it seems to me, is that the human mind uses a rather ad-hoc system to keep track of its goals - it works, but not cleanly. English doesn't embody a sharp distinction between means and ends: "I want to save my sister's life" and "I want to administer penicillin to my sister" use the same word "want".

Can we describe, in mere English, the distinction that is getting lost?

As a first stab:

"Instrumental values" are desirable strictly conditional on their anticipated consequences. "I want to administer penicillin to my sister", not because a penic...
Thou Art Godshatter
by Eliezer Yudkowsky — 2007-11-13T19:38:56.000Z

Before the 20th century, not a single human being had an explicit concept of "inclusive genetic fitness", the sole and absolute obsession of the blind idiot god. We have no instinctive revulsion of condoms or oral sex. Our brains, those supreme reproductive organs, don't perform a check for reproductive efficacy before granting us sexual pleasure.

Why not? Why aren't we consciously obsessed with inclusive genetic fitness? Why did the Evolution-of-Humans Fairy create brains that would invent condoms? "It would have been so easy," thinks the human, who can design new complex systems in an afternoon.

The Evolution Fairy, as we all know, is obsessed with inclusive genetic fitness. When she decides which genes to promote to universality, she doesn't seem to take into account anything except the number of copies a gene produces. (How strange!)

But since the maker of intelligence is thus obsessed, why not create intelligent agents - you can't call them humans - who would likewise care purely about inclusive genetic fitness? Such agents would have sex only as a means of reproduction, and wouldn't bother with sex that involved birth control. They could eat food out of an explicitly reasoned belief that food was necessary to reproduce, not because they liked the taste, and so they wouldn't eat candy if it became detrimental to survival or reproduction. Post-menopausal women would babysit grandchildren until they became sick enough to be a net drain on resources, and would then commit suicide.

It seems like such an obvious design improvement - from the Evolution Fairy's perspective.

Now it's clear, as was discussed yesterday, that it's hard to build a powerful enough consequentialist. Natural selection sort-of reasons consequentially, but only by depending on the actual consequences. Human evolutionary theorists have to do really high-falutin' abstract reasoning in order to imagine the links between adaptations and reproductive success.

But human brains clearly can imagine these links in protein. So when the Evolution Fairy made humans, why did It bother with any motivation except inclusive genetic fitness?

It's been less than two centuries since a protein brain first represented the concept of natural selection. The modern notion of "inclusive genetic fitness" is even more subt...
Protein Reinforcement and DNA Consequentialism
by Eliezer Yudkowsky — 2007-11-13T01:34:25.000Z

Followup to: Evolutionary Psychology

It takes hundreds of generations for a simple beneficial mutation to promote itself to universality in a gene pool. Thousands of generations, or even millions, to create complex interdependent machinery.

That's some slow learning there. Let's say you're building a squirrel, and you want the squirrel to know locations for finding nuts. Individual nut trees don't last for the thousands of years required for natural selection. You're going to have to learn using proteins. You're going to have to build a brain.

Protein computers and sensors can learn by looking, much faster than DNA can learn by mutation and selection. And yet (until very recently) the protein learning machines only learned in narrow, specific domains. Squirrel brains learn to find nut trees, but not to build gliders - as flying squirrel DNA is slowly learning to do. The protein computers learned faster than DNA, but much less generally.

How the heck does a double-stranded molecule that fits inside a cell nucleus, come to embody truths that baffle a whole damn squirrel brain?

Consider the high-falutin' abstract thinking that modern evolutionary theorists do in order to understand how adaptations increase inclusive genetic fitness. Reciprocal altruism, evolutionarily stable strategies, deterrence, costly signaling, sexual selection - how many humans explicitly represent this knowledge? Yet DNA can learn it without a protein computer.

There's a long chain of causality whereby a male squirrel, eating a nut today, produces more offspring months later: Chewing and swallowing food, to digesting food, to burning some calories today and turning others into fat, to burning the fat through the winter, to surviving the winter, to mating with a female, to the sperm fertilizing an egg inside the female, to the female giving birth to an offspring that shares 50% of the squirrel's genes.

With the sole exception of humans, no protein brain can imagine chains of causality that long, that abstract, and crossing that many domains. With one exception, no protein brain is even capable of drawing the consequential link from chewing and swallowing to inclusive reproductive fitness.

Yet natural selection exploits links between local actions and distant reproductive benefits. In wide generality, across domain...
Evolutionary Psychology
by Eliezer Yudkowsky — 2007-11-11T20:41:03.000Z

Like "IRC chat" or "TCP/IP protocol", the phrase "reproductive organ" is redundant. All organs are reproductive organs. Where do a bird's wings come from? An Evolution-of-Birds Fairy who thinks that flying is really neat? The bird's wings are there because they contributed to the bird's ancestors' reproduction. Likewise the bird's heart, lungs, and genitals. At most we might find it worthwhile to distinguish between directly reproductive organs and indirectly reproductive organs.

This observation holds true also of the brain, the most complex organ system known to biology. Some brain organs are directly reproductive, like lust; others are indirectly reproductive, like anger.

Where does the human emotion of anger come from? An Evolution-of-Humans Fairy who thought that anger was a worthwhile feature? The neural circuitry of anger is a reproductive organ as surely as your liver. Anger exists in Homo sapiens because angry ancestors had more kids. There's no other way it could have gotten there.

This historical fact about the origin of anger confuses all too many people. They say, "Wait, are you saying that when I'm angry, I'm subconsciously trying to have children? That's not what I'm thinking after someone punches me in the nose."

No. No. No. NO!

Individual organisms are best thought of as adaptation-executers, not fitness-maximizers. The cause of an adaptation, the shape of an adaptation, and the consequence of an adaptation, are all separate things. If you built a toaster, you wouldn't expect the toaster to reshape itself when you tried to cram in a whole loaf of bread; yes, you intended it to make toast, but that intention is a fact about you, not a fact about the toaster. The toaster has no sense of its own purpose.

But a toaster is not an intention-bearing object. It is not a mind at all, so we are not tempted to attribute goals to it. If we see the toaster as purposed, we don't think the toaster knows it, because we don't think the toaster knows anything.

It's like the old test of being asked to say the color of the letters in "blue". It takes longer for subjects to name this color, because of the need to untangle the meaning of the letters and the color of the letters. You wouldn't have similar trouble naming the color of the letters in "wind".

But a human brain, in addition to being an artifact historically produced by evolution, is also a mind capable of bear

...
Adaptation-Executers, not Fitness-Maximizers
by Eliezer Yudkowsky — 2007-11-11T06:39:18.000Z

"Individual organisms are best thought of as adaptation-executers rather than as fitness-maximizers."

—John Tooby and Leda Cosmides, The Psychological Foundations of Culture.

Fifty thousand years ago, the taste buds of Homo sapiens directed their bearers to the scarcest, most critical food resources—sugar and fat. Calories, in a word. Today, the context of a taste bud's function has changed, but the taste buds themselves have not. Calories, far from being scarce (in First World countries), are actively harmful. Micronutrients that were reliably abundant in leaves and nuts are absent from bread, but our taste buds don't complain. A scoop of ice cream is a superstimulus, containing more sugar, fat, and salt than anything in the ancestral environment.

No human being with the deliberate goal of maximizing their alleles' inclusive genetic fitness, would ever eat a cookie unless they were starving. But individual organisms are best thought of as adaptation-executers, not fitness-maximizers.

A toaster, though its designer intended it to make toast, does not bear within it the intelligence of the designer—it won't automatically redesign and reshape itself if you try to cram in an entire loaf of bread. A Phillips-head screwdriver won't reconform itself to a flat-head screw. We created these tools, but they exist independently of us, and they continue independently of us.

The atoms of a screwdriver don't have tiny little XML tags inside describing their "objective" purpose. The designer had something in mind, yes, but that's not the same as what happens in the real world. If you forgot that the designer is a separate entity from the designed thing, you might think, "The purpose of the screwdriver is to drive screws"—as though this were an explicit property of the screwdriver itself, rather than a property of the designer's state of mind. You might be surprised that the screwdriver didn't reconfigure itself to the flat-head screw, since, after all, the screwdriver's purpose is to turn screws.

The cause of the screwdriver's existence is the designer's mind, which imagined an imaginary screw, and imagined an imaginary handle turning. The actual operation of the screwdriver, its actual fit to an actual screw head, cannot be the objective cause of the screwdriver's e...
Fake Optimization Criteria
by Eliezer Yudkowsky — 2007-11-10T00:10:51.000Z

I've previously dwelt in considerable length upon forms of rationalization whereby our beliefs appear to match the evidence much more strongly than they actually do. And I'm not overemphasizing the point, either. If we could beat this fundamental metabias and see what every hypothesis really predicted, we would be able to recover from almost any other error of fact.

The mirror challenge for decision theory is seeing which option a choice criterion really endorses. If your stated moral principles call for you to provide laptops to everyone, does that really endorse buying a $1 million gem-studded laptop for yourself, or spending the same money on shipping 5000 OLPCs?

We seem to have evolved a knack for arguing that practically any goal implies practically any action. A phlogiston theorist explaining why magnesium gains weight when burned has nothing on an Inquisitor explaining why God's infinite love for all His children requires burning some of them at the stake.

There's no mystery about this. Politics was a feature of the ancestral environment. We are descended from those who argued most persuasively that the good of the tribe meant executing their hated rival Uglak. (We sure ain't descended from Uglak.)

And yet... is it possible to prove that if Robert Mugabe cared only for the good of Zimbabwe, he would resign from its presidency? You can argue that the policy follows from the goal, but haven't we just seen that humans can match up any goal to any policy? How do you know that you're right and Mugabe is wrong? (There are a number of reasons this is a good guess, but bear with me here.)

Human motives are manifold and obscure, our decision processes as vastly complicated as our brains. And the world itself is vastly complicated, on every choice of real-world policy. Can we even prove that human beings are rationalizing—that we're systematically distorting the link from principles to policy—when we lack a single firm place on which to stand? When there's no way to find out exactly what even a single optimization criterion implies? (Actually, you can just observe that people disagree about office politics in ways that strangely correlate to their own interests, while simultaneously denying that any such interests are at work. But again, bear with me here.)

Where is the standardized, open-source, generally ...
Why aren’t our property rights over one another more transferable?
by KatjaGrace — 2007-11-09T16:05:00.000Z

Why do people get married? If anyone ever proposes to me for a reason other than to surreptitiously steal my belongings or to get more centerlink benefits, I would have to refuse them on the basis that I could not love a man so irrational.

~ all married/engaged folks please forgive me and freely assume I’m just rather jealous :) ~

What is the purpose of a contract to love someone forever?
If you anticipate loving them forever anyway, it would seem to be pointless. I’m told it is romantic nonetheless, but how is it romantic to take a legal precaution that implies some doubt that you will love each other forever?
On the off chance that you stop being in love with them, the last thing you want is to be legally bound to stick with them. And a legal obligation to actually love them is pretty laughable. Possibly if they stop loving you you might want them to stick around regardless, but isn’t that rather selfish and desperate? Anyway, surely this is hardly the contingency people have in mind when saying their vows.

Anyway, now that divorce is allowed the whole thing seems to be completely meaningless, except if understood as a way of betting large swathes of assets on the outcomes of ones emotional attachments, with divorce lawyers and priests playing casino. If this is the kind of gambling that floats your boat it makes perfect sense, but perhaps you could benefit from counselling at some point.

I propose a solution for escaping most of the potential damage of weddings while retaining the romance they apparently emanate: short term marriage contracts. At the end of, say, six months (terms such as length should be completely flexible) you renew it, or don’t, and act accordingly. If your spouse forgets this anniversary you can give them a year off. The whole ceremony could be the same as before, with a minor alteration to the vows: ‘…in sickness or in health, to love and to cherish ’til death or May 17 – whichever comes first, do us part’. Plus you can have more parties later on.

(On the earlier point, if anyone is ever irrational enough to propose to me, and I usually consider them rational, perhaps I must conclude that they are irrational specifically with regards to me, so therefore may in fact love me. A heuristic for finding selectively crazy guys could be just what one needs. If this occurs then you all have perm...
Fake Morality
by Eliezer Yudkowsky — 2007-11-08T21:32:04.000Z

God, say the religious fundamentalists, is the source of all morality; there can be no morality without a Judge who rewards and punishes. If we did not fear hell and yearn for heaven, then what would stop people from murdering each other left and right?

Suppose Omega makes a credible threat that if you ever step inside a bathroom between 7AM and 10AM in the morning, he'll kill you.
Would you be panicked by the prospect of Omega withdrawing his threat? Would you cower in existential terror and cry: "If Omega withdraws his threat, then what's to keep me from going to the bathroom?" No; you'd probably be quite relieved at your increased opportunity to, ahem, relieve yourself.

Which is to say: The very fact that a religious person would be afraid of God withdrawing Its threat to punish them for committing murder, shows that they have a revulsion of murder which is independent of whether God punishes murder or not. If they had no sense that murder was wrong independently of divine retribution, the prospect of God not punishing murder would be no more existentially horrifying than the prospect of God not punishing sneezing.

If Overcoming Bias has any religious readers left, I say to you: it
may be that you will someday lose your faith: and on that day, you will not lose all sense of moral direction. For if you fear the prospect of God not punishing some deed, that is a moral compass. You can plug that compass directly into your decision system and steer by it. You can simply not do whatever you are afraid God may not punish you for doing. The fear of losing a moral compass is itself a moral compass. Indeed, I suspect you are steering by that compass, and that you always have been. As Piers Anthony once said, "Only those with souls worry over whether or not they have them." s/soul/morality/ and the point carries.

You don't hear religious fundamentalists using the argument: "If we did not fear hell and yearn for heaven, then what would stop people from eating pork?" Yet by their assumptions - that we have no moral compass but divine reward and retribution - this argument should sound just as forceful as the other.

Even the notion that God threatens you with eternal
hellfire, rather than cookies, piggybacks on a pre-existing negative
value for hellfi...
Fake Selfishness
by Eliezer Yudkowsky — 2007-11-08T02:31:09.000Z

Once upon a time, I met someone who proclaimed himself to be purely selfish, and told me that I should be purely selfish as well. I was feeling mischievous(*) that day, so I said, "I've observed that with most religious people, at least the ones I meet, it doesn't matter much what their religion says, because whatever they want to do, they can find a religious reason for it. Their religion says they should stone unbelievers, but they want to be nice to people, so they find a religious justification for that instead. It looks to me like when people espouse a philosophy of selfishness, it has no effect on their behavior, because whenever they want to be nice to people, they can rationalize it in selfish terms."

And the one said, "I don't think that's true."

I said, "If you're genuinely selfish, then why do you want me to be selfish too? Doesn't that make you concerned for my welfare? Shouldn't you be trying to persuade me to be more altruistic, so you can exploit me?"

The one replied: "Well, if you become selfish, then you'll realize
that it's in your rational self-interest to play a productive role in
the economy, instead of, for example, passing laws that infringe on my
private property."

And I said, "But I'm a small-L libertarian already, so I'm not going to support those laws. And since I conceive of myself as an altruist, I've taken a job that I expect to benefit a lot
of people, including you, instead of a job that pays more. Would you
really benefit more from me if I became selfish? Besides, is trying to
persuade me to be selfish the most selfish thing you could be
doing? Aren't there other things you could do with your time that
would bring much more direct benefits? But what I really want
to know is this: Did you start out by thinking that you wanted to be
selfish, and then decide this was the most selfish thing you could
possibly do? Or did you start out by wanting to convert others to
selfishness, then look for ways to rationalize that as self-benefiting?"

And the one said, "You may be right about that last part," so I marked him down as intelligent.

(*) Other mischievous questions to ask self-proclaimed Selfishes: "Would you sacrifice your own life to save the entire human species?" (...