trentmkelly/LessWrong-43k huggingface dataset:
Excluding the Supernatural
by Eliezer Yudkowsky — 2008-09-12T00:12:37.000Z

Occasionally, you hear someone claiming that creationism should not be taught in schools, especially not as a competing hypothesis to evolution, because creationism is a priori and automatically excluded from scientific consideration, in that it invokes the "supernatural".

So... is the idea here, that creationism could be true, but even if it were true, you wouldn't be allowed to teach it in science class, because science is only about "natural" things?

It seems clear enough that this notion stems from the desire to avoid a confrontation between science and religion. You don't want to come right out and say that science doesn't teach Religious Claim X because X has been tested by the scientific method and found false. So instead, you can... um... claim that science is excluding hypothesis X a priori. That way you don't have to discuss how experiment has falsified X a posteriori.

Of course this plays right into the creationist claim that Intelligent Design isn't getting a fair shake from science—that science has prejudged the issue in favor of atheism, regardless of the evidence. If science excluded Intelligent Design a priori, this would be a justified complaint!

But let's back up a moment. The one comes to you and says: "Intelligent Design is excluded from being science a priori, because it is 'supernatural', and science only deals in 'natural' explanations."

What exactly do they mean, "supernatural"? Is any explanation invented by someone with the last name "Cohen" a supernatural one? If we're going to summarily kick a set of hypotheses out of science, what is it that we're supposed to exclude?

By far the best definition I've ever heard of the supernatural is Richard Carrier's: A "supernatural" explanation appeals to ontologically basic mental things, mental entities that cannot be reduced to nonmental entities.

This is the difference, for example, between saying that water rolls downhill because it wants to be lower, and setting forth differential equations that claim to describe only motions, not desires. It's the difference between saying that a tree puts forth leaves because of a tree spirit, versus examining plant biochemistry. Cognitive science takes the fight against supernaturalism into the realm of the mind.

Why is this an excellent definition of the supernatural? I refer you to...
Rationality Quotes 17
by Eliezer Yudkowsky — 2008-09-11T03:29:56.000Z

"We take almost all of the decisive steps in our lives as a result of slight inner adjustments of which we are barely conscious."
-- Austerlitz

"In both poker and life, you can't read people any better than they can read themselves. You can, if you’re good, very accurately determine if they think their hand is good, or if they think they know the answer to your legal question. But you can't be sure if reality differs from their perception."
-- Matt Maroon

"We should not complain about impermanence, because without impermanence, nothing is possible."
-- Thich Nhat Hanh

"I've never been happy. I have a few memories, early in life, and it sounds dramatic to say, but when I reflect on my life, the best I've ever had were brief periods when things were simply less painful."
-- [Anonymous]

Q: What are the "intuitive and metaphyscal arts"?
A: The gods alone know. Probably the old tired con-acts of fortune-telling and putting the hex on your neighbor's goat, glossed up with gibberish borrowed from pop science tracts in the last two centuries.
-- The Aleph Anti-FAQ

"If you build a snazzy alife sim ... you'd be a kind of bridging `first cause', and might even have the power to intervene in their lives - even obliterate their entire experienced cosmos - but that wouldn't make you a god in any interesting sense. Gods are ontologically distinct from creatures, or they're not worth the paper they're written on."
-- Damien Broderick

"NORMAL is a setting on a washing-machine."
-- Nikolai Kingsley
Points of Departure
by Eliezer Yudkowsky — 2008-09-09T21:18:08.000Z

Followup to: Anthropomorphic Optimism

If you've watched Hollywood sci-fi involving supposed robots, androids, or AIs, then you've seen AIs that are depicted as "emotionless". In the olden days this was done by having the AI speak in a monotone pitch - while perfectly stressing the syllables, of course. (I could similarly go on about how AIs that disastrously misinterpret their mission instructions, never seem to need help parsing spoken English.) You can also show that an AI is "emotionless" by having it notice an emotion with a blatant somatic effect, like tears or laughter, and ask what it means (though of course the AI never asks about sweat or coughing).

If you watch enough Hollywood sci-fi, you'll run into all of the following situations occurring with supposedly "emotionless" AIs:

An AI that malfunctions or otherwise turns evil, instantly acquires all of the negative human emotions - it hates, it wants revenge, and feels the need to make self-justifying speeches.

Conversely, an AI that turns to the Light Side, gradually acquires a full complement of human emotions.

An "emotionless" AI suddenly exhibits human emotion when under exceptional stress; e.g. an AI that displays no reaction to thousands of deaths, suddenly showing remorse upon killing its creator.

An AI begins to exhibit signs of human emotion, and refuses to admit it.

Now, why might a Hollywood scriptwriter make those particular mistakes?

These mistakes seem to me to bear the signature of modeling an Artificial Intelligence as an emotionally repressed human.

At least, I can't seem to think of any other simple hypothesis that explains the behaviors 1-4 above. The AI that turns evil has lost its negative-emotion-suppressor, so the negative emotions suddenly switch on. The AI that turns from mechanical agent to good agent, gradually loses the emotion-suppressor keeping it mechanical, so the good emotions rise to the surface. Under exceptional stress, of course the emotional repression that keeps the AI "mechanical" will immediately break down and let the emotions out. But if the stress isn't so exceptional, the firmly repressed AI will deny any hint of the emotions leaking out - that conflicts with the AI's self-image of itself as being emotionless.

It's not that the Hollywood scriptwriters are expl...
Singularity Summit 2008
by Eliezer Yudkowsky — 2008-09-08T23:26:47.000Z

FYI all: The Singularity Summit 2008 is coming up, 9am-5pm October 25th, 2008 in San Jose, CA. This is run by my host organization, the Singularity Institute. Speakers this year include Vernor Vinge, Marvin Minsky, the CTO of Intel, and the chair of the X Prize Foundation.

Before anyone posts any angry comments: yes, the registration costs
actual money this year. The Singularity Institute has run free events before, and will run free events in the future. But while past Singularity Summits have been media successes, they haven't been fundraising successes up to this point. So Tyler Emerson et. al. are trying it a little differently. TANSTAAFL.

Lots of speakers talking for short periods this year. I'm intrigued by that format. We'll see how it goes.

Press release:

SAN JOSE, CA
– Singularity Summit 2008: Opportunity, Risk, Leadership takes place
October 25 at the intimate Montgomery Theater in San Jose, CA, the
Singularity Institute for Artificial Intelligence announced. Now
in its third year, the Singularity Summit gathers the smartest people
around to explore the biggest idea of our time: the Singularity.

Keynotes will include Ray Kurzweil, updating his predictions in The Singularity is Near,
and Intel CTO Justin Rattner, who will examine the Singularity’s
plausibility. At the Intel Developer Forum on August 21, 2008, he
explained why he thinks the gap between humans and machines will close
by 2050. “Rather than look back, we’re going to look forward 40 years,”
said Rattner. “It’s in that future where many people think that machine
intelligence will surpass human intelligence.”

“The acceleration of technological progress has been the central
feature of this century,” said computer scientist Dr. Vernor Vinge in a
seminal paper in 1993. “We are on the edge of change comparable to the
rise of human life on Earth. The precise cause of this change is the
imminent creation by technology of entities with greater than human
intelligence.”

Singularity Summit 2008 will feature an impressive lineup:

Dr. Ruzena Bajcsy, pioneering AI and robotics researcher

Dr. Eric Baum, AI researcher, author of What is Thought?

Marshall Brain, founder of HowStuffWorks.com, author of Robotic Nation

Dr. Cynthia Breazeal, robotics professor at MIT, creator of Kismet

Dr. Peter Diamandis, chair and CEO of X PRIZE Foundation

Esther D

...
Rationality Quotes 16
by Eliezer Yudkowsky — 2008-09-08T03:36:50.000Z

"I read a lot of fantasy and have wondered sometimes, not so much what I would do in a fantasy setting, but what the book characters would do in the real world."
-- Romana

"That's the thing that's always fascinated me about Go. It is essentially an extremely simple game gone terribly, terribly wrong."
-- Amorymeltzer

"Dealing with the sheer of volume of "stuff" available on the internet is like being a crackhead with OCD. In the course of one hour I've tweaked my fantasy baseball lineup, posted on this message board, read Yahoo news, answered my latest e-mail, downloaded guidance criteria for PAHs in soils in NY State, checked the discography of a couple of bands, sent a deliverable to a client, and checked the weather. If that isn't superstimulus I don't know what is. It's amazing how much I can do, yet accomplish so little."
-- Misanthropic

"We don't have thoughts, we are thoughts. Thoughts are not responsible for the machinery that happens to think them."
-- John K Clark

"I have known more people whose lives have been ruined by getting a Ph.D. in physics than by drugs."
-- Jonathan I. Katz

"There's no difference between a pessimist who says, "Oh, it's hopeless, so don't bother doing anything," and an optimist who says, "Don't bother doing anything, it's going to turn out fine anyway." Either way, nothing happens."
-- Yvon Chouinard

"Life moved ever outward into infinite possibilities and yet all things were perfect and finished in every single moment, their end attained."
-- David Zindell, Neverness
Let’s discuss the weather
by KatjaGrace — 2008-09-07T12:54:00.000Z

Wondering your opinion, not particularly trying to change it:

1. How likely is our avoiding dangerous climate change by getting enough international cooperation to cut global carbon emissions enough in time?

2. How much of a difference can I make to this?

3. If the chances in 1. are small, why don’t we try something like geoengineering?

4. If 1 and 2 are small and 3 feasible (so the world isn’t about to end), why is cutting emissions an important issue to work on? (given the opportunity costs: there are lots of other critical issues)
Rationality Quotes 15
by Eliezer Yudkowsky — 2008-09-06T18:53:12.000Z

"Who thinks they're not open-minded? Our hypothetical prim miss from the suburbs thinks she's open-minded. Hasn't she been taught to be? Ask anyone, and they'll say the same thing: they're pretty open-minded, though they draw the line at things that are really wrong."
-- Paul Graham

"In the same way that we need statesmen to spare us the abjection of exercising power, we need scholars to spare us the abjection of learning."
-- Jean Baudrillard

"Because giftedness is not to be talked about, no one tells high-IQ children explicitly, forcefully and repeatedly that their intellectual talent is a gift. That they are not superior human beings, but lucky ones. That the gift brings with it obligations to be worthy of it."
-- Charles Murray

"The popular media can only handle ideas expressible in proto-language, not ideas requiring nested phrase-structure syntax for their exposition."
-- Ben Goertzel

"The best part about math is that, if you have the right answer and someone disagrees with you, it really is because they're stupid."
-- Quotes from Honors Linear Algebra

"Long-Term Capital Management had faith in diversification. Its history serves as ample notification that eggs in different baskets can and do all break at the same time."
-- Craig L. Howe

"Accountability is about one person taking responsibility. If two people are accountable for the same decision, no one is really accountable."
-- Glyn Holton
Rationality Quotes 14
by Eliezer Yudkowsky — 2008-09-05T20:16:33.000Z

"As for the little green men... they don't want us to know about them, so they refrain from making contact... then they do silly aerobatics displays within radar range of military bases... with their exterior lights on... if that's extraterrestrial intelligence, I'm not sure I want to know what extraterrestrial stupidity looks like."
-- Russell Wallace

"Characterizing male status-seeking as egotistical is like characterizing bonobo promiscuity as unchaste."
-- Liza May

"Introducing a technology is not a neutral act--it is profoundly revolutionary. If you present a new technology to the world you are effectively legislating a change in the way we all live. You are changing society, not some vague democratic process. The individuals who are driven to use that technology by the disparities of wealth and power it creates do not have a real choice in the matter. So the idea that we are giving people more freedom by developing technologies and then simply making them available is a dangerous illusion."
-- Karl Schroeder

"Hans Riesel held a Mersenne record for 14 days in the 50's, calculated using the first Swedish computer. My old highschool computing teacher had worked as a student on the system and had managed to crush his foot when a byte fell out of its rack and onto him."
-- Anders Sandberg

"Gentlemen, I do not mind being contradicted, and I am unperturbed when I am attacked, but I confess I have slight misgivings when I hear myself being explained."
-- Lord Balfour, to the English Parliament
The Truly Iterated Prisoner's Dilemma
by Eliezer Yudkowsky — 2008-09-04T18:00:00.000Z

Followup to: The True Prisoner's Dilemma

For everyone who thought that the rational choice in yesterday's True Prisoner's Dilemma was to defect, a follow-up dilemma:

Suppose that the dilemma was not one-shot, but was rather to be repeated exactly 100 times, where for each round, the payoff matrix looks like this:

Humans: C

Humans: D

Paperclipper: C

(2 million human lives saved, 2 paperclips gained)

(+3 million lives, +0 paperclips)

Paperclipper: D

(+0 lives, +3 paperclips)

(+1 million lives, +1 paperclip)

As most of you probably know, the king of the classical iterated Prisoner's Dilemma is Tit for Tat, which cooperates on the first round, and on succeeding rounds does whatever its opponent did last time. But what most of you may not realize, is that, if you know when the iteration will stop, Tit for Tat is - according to classical game theory - irrational.

Why? Consider the 100th round. On the 100th round, there will be no future iterations, no chance to retaliate against the other player for defection. Both of you know this, so the game reduces to the one-shot Prisoner's Dilemma. Since you are both classical game theorists, you both defect.

Now consider the 99th round. Both of you know that you will both defect in the 100th round, regardless of what either of you do in the 99th round. So you both know that your future payoff doesn't depend on your current action, only your current payoff. You are both classical game theorists. So you both defect.

Now consider the 98th round...

With humanity and the Paperclipper facing 100 rounds of the iterated Prisoner's Dilemma, do you really truly think that the rational thing for both parties to do, is steadily defect against each other for the next 100 rounds?
The True Prisoner's Dilemma
by Eliezer Yudkowsky — 2008-09-03T21:34:28.000Z

It occurred to me one day that the standard visualization of the Prisoner's Dilemma is fake.

The core of the Prisoner's Dilemma is this symmetric payoff matrix:

1: C

1: D

2: C

(3, 3)

(5, 0)

2: D

(0, 5)

(2, 2)

Player 1, and Player 2, can each choose C or D. 1 and 2's utility for the final outcome is given by the first and second number in the pair. For reasons that will become apparent, "C" stands for "cooperate" and D stands for "defect".

Observe that a player in this game (regarding themselves as the first player) has this preference ordering over outcomes: (D, C) > (C, C) > (D, D) > (C, D).

D, it would seem, dominates C: If the other player chooses C, you prefer (D, C) to (C, C); and if the other player chooses D, you prefer (D, D) to (C, D). So you wisely choose D, and as the payoff table is symmetric, the other player likewise chooses D.

If only you'd both been less wise! You both prefer (C, C) to (D, D). That is, you both prefer mutual cooperation to mutual defection.

The Prisoner's Dilemma is one of the great foundational issues in decision theory, and enormous volumes of material have been written about it. Which makes it an audacious assertion of mine, that the usual way of visualizing the Prisoner's Dilemma has a severe flaw, at least if you happen to be human.

The classic visualization of the Prisoner's Dilemma is as follows: you
are a criminal, and you and your confederate in crime have both been
captured by the authorities.

Independently, without communicating, and
without being able to change your mind afterward, you have to decide
whether to give testimony against your confederate (D) or remain silent
(C).

Both of you, right now, are facing one-year prison sentences;
testifying (D) takes one year off your prison sentence, and adds two years
to your confederate's sentence.

Or maybe you and some stranger are, only once, and without knowing the other player's history, or finding out who the player was afterward, deciding whether to play C or D, for a payoff in dollars matching the standard chart.

And, oh yes - in the classic visualization you're supposed to pretend that you're entirely
selfish, that you don't care about your confederate criminal, or the player in
the other room.

It's this last specification that makes the classic visualization, in my ...
Rationality Quotes 13
by Eliezer Yudkowsky — 2008-09-02T16:00:00.000Z

"You can only compromise your principles once. After then you don't have any."
-- Smug Lisp Weeny

"If you want to do good, work on the technology, not on getting power."
-- John McCarthy

"If you’re interested in being on the right side of disputes, you will refute your opponents’ arguments. But if you’re interested in producing truth, you will fix your opponents’ arguments for them. To win, you must fight not only the creature you encounter; you must fight the most horrible thing that can be constructed from its corpse."
-- Black Belt Bayesian

"I normally thought of "God!" as a disclaimer, or like the MPAA rating you see just before a movie starts: it told me before I continued into conversation with that person, that that person had limitations to their intellectual capacity or intellectual honesty."
-- Mike Barskey

"It is the soldier, not the reporter, who has given us freedom of the press. It is the soldier, not the poet, who has given us freedom of speech. It is the soldier, not the campus organizer, who has given us the freedom to demonstrate. It is the soldier, not the lawyer, who has given us the right to a fair trial. It is the soldier, who salutes the flag, who serves under the flag, and whose coffin is draped by the flag, who allows the protester to burn the flag."
-- Father Dennis Edward O'Brien, USMC
Rationality Quotes 12
by Eliezer Yudkowsky — 2008-09-01T20:00:00.000Z

"Even if I had an objective proof that you don't find it unpleasant when you stick your hand in a fire, I still think you’d pull your hand out at the first opportunity."
-- John K Clark

"So often when one level of delusion goes away, another one more subtle comes in its place."
-- Rational Buddhist

"Your denial of the importance of objectivity amounts to announcing your intention to lie to us. No-one should believe anything you say."
-- John McCarthy

"How exactly does one 'alter reality'? If I eat an apple have I altered reality? Or maybe you mean to just give the appearance of altering reality."
-- JoeDad

"Promoting less than maximally accurate beliefs is an act of sabotage. Don't do it to anyone unless you'd also slash their tires."
-- Black Belt Bayesian
Brief Break
by Eliezer Yudkowsky — 2008-08-31T16:00:00.000Z

I've been feeling burned on Overcoming Bias lately, meaning that I take too long to write my posts, which decreases the amount of recovery time, making me feel more burned, etc.

So I'm taking at most a one-week break. I'll post small units of rationality quotes each day, so as to not quite abandon you. I may even post some actual writing, if I feel spontaneous, but definitely not for the next two days; I have to enforce this break upon myself.

When I get back, my schedule calls for me to finish up the Anthropomorphism sequence, and then talk about Marcus Hutter's AIXI, which I think is the last brain-malfunction-causing subject I need to discuss. My posts should then hopefully go back to being shorter and easier.

Hey, at least I got through over a solid year of posts without taking a vacation.
Dreams of Friendliness
by Eliezer Yudkowsky — 2008-08-31T01:20:52.000Z

Continuation of: Qualitative Strategies of Friendliness

Yesterday I described three classes of deep problem with qualitative-physics-like strategies for building nice AIs - e.g., the AI is reinforced by smiles, and happy people smile, therefore the AI will tend to act to produce happiness. In shallow form, three instances of the three problems would be:

Ripping people's faces off and wiring them into smiles;

Building lots of tiny agents with happiness counters set to large numbers;

Killing off the human species and replacing it with a form of sentient life that has no objections to being happy all day in a little jar.

And the deep forms of the problem are, roughly:

A superintelligence will search out alternate causal pathways to its goals than the ones you had in mind;

The boundaries of moral categories are not predictively natural entities;

Strong optimization for only some humane values, does not imply a good total outcome.

But there are other ways, and deeper ways, of viewing the failure of qualitative-physics-based Friendliness strategies.

Every now and then, someone proposes the Oracle AI strategy: "Why not just have a superintelligence that answers human questions, instead of acting autonomously in the world?"

Sounds pretty safe, doesn't it? What could possibly go wrong?

Well... if you've got any respect for Murphy's Law, the power of superintelligence, and human stupidity, then you can probably think of quite a few things that could go wrong with this scenario. Both in terms of how a naive implementation could fail - e.g., universe tiled with tiny users asking tiny questions and receiving fast, non-resource-intensive answers - and in terms of what could go wrong even if the basic scenario worked.

But let's just talk about the structure of the AI.

When someone reinvents the Oracle AI, the most common opening remark runs like this:

"Why not just have the AI answer questions, instead of trying to do anything? Then it wouldn't need to be Friendly. It wouldn't need any goals at all. It would just answer questions."

To which the reply is that the AI needs goals in order to decide how to think: that is, the AI has to act as a powerful optimization process in order to plan its acquisition of knowledge, effectively distill sensory information, pluck "answers" to particular questions out of the space of all possib...