trentmkelly/LessWrong-43k huggingface dataset:
Lawful Uncertainty
by Eliezer Yudkowsky — 2008-11-10T21:06:32.000Z

In Rational Choice in an Uncertain World, Robyn Dawes describes an experiment by Tversky:1

Many psychological experiments were conducted in the late 1950s and early 1960s in which subjects were asked to predict the outcome of an event that had a random component but yet had base-rate predictability—for example, subjects were asked to predict whether the next card the experimenter turned over would be red or blue in a context in which 70% of the cards were blue, but in which the sequence of red and blue cards was totally random.

In such a situation, the strategy that will yield the highest proportion of success is to predict the more common event. For example, if 70% of the cards are blue, then predicting blue on every trial yields a 70% success rate.

What subjects tended to do instead, however, was match probabilities—that is, predict the more probable event with the relative frequency with which it occurred. For example, subjects tended to predict 70% of the time that the blue card would occur and 30% of the time that the red card would occur. Such a strategy yields a 58% success rate, because the subjects are correct 70% of the time when the blue card occurs (which happens with probability .70) and 30% of the time when the red card occurs (which happens with probability .30); (.70×.70) + (.30×.30) = .58.

In fact, subjects predict the more frequent event with a slightly higher probability than that with which it occurs, but do not come close to predicting its occurrence 100% of the time, even when they are paid for the accuracy of their predictions . . . For example, subjects who were paid a nickel for each correct prediction over a thousand trials . . . predicted [the more common event] 76% of the time.

Do not think that this experiment is about a minor flaw in gambling strategies. It compactly illustrates the most important idea in all of rationality.

Subjects just keep guessing red, as if they think they have some way of predicting the random sequence. Of this experiment Dawes goes on to say, “Despite feedback through a thousand trials, subjects cannot bring themselves to believe that the situation is one in which they cannot predict.”

But the error must go deeper than that. Even if subjects think they’ve come up with a hypothesis, they don’t have to actually bet on that prediction in order to test their hypothesis. They can say, “Now if this hypothesis is corr...
Ask OB: Leaving the Fold
by Eliezer Yudkowsky — 2008-11-09T18:08:52.000Z

Followup to: Crisis of Faith

I thought this comment from "Jo" deserved a bump to the front page:

"So
here I am having been raised in the Christian faith and trying not to
freak out over the past few weeks because I've finally begun to wonder
whether I believe things just because I was raised with them. Our
family is surrounded by genuinely wonderful people who have poured
their talents into us since we were teenagers, and our social structure
and business rests on the tenets of what we believe. I've been trying
to work out how I can 'clear the decks' and then rebuild with whatever
is worth keeping, yet it's so foundational that it will affect my
marriage (to a pretty special man) and my daughters who, of course,
have also been raised to walk the Christian path.

Is there anyone who's been in this position - really, really invested in a faith and then walked away?"

Handling this kind of situation has to count as part of the art. But I haven't gone through anything like this. Can anyone with experience advise Jo on what to expect, what to do, and what not to do?
Lawful Creativity
by Eliezer Yudkowsky — 2008-11-08T19:54:22.000Z

Previously in Series: Recognizing Intelligence

Creativity, we've all been told, is about Jumping Out Of The System, as Hofstadter calls it (JOOTSing for short). Questioned assumptions, violated expectations.

Fire is dangerous: the rule of fire is to run away from it. What must have gone through the mind of the first hominid to domesticate fire? The rule of milk is that it spoils quickly and then you can't drink it - who first turned milk into cheese? The rule of computers is that they're made with vacuum tubes, fill a room and are so expensive that only corporations can own them. Wasn't the transistor a surprise...

Who, then, could put laws on creativity? Who could bound it, who could circumscribe it, even with a concept boundary that distinguishes "creativity" from "not creativity"? No matter what system you try to lay down, mightn't a more clever person JOOTS right out of it? If you say "This, this, and this is 'creative'" aren't you just making up the sort of rule that creative minds love to violate?

Why, look at all the rules that smart people have violated throughout history, to the enormous profit of humanity. Indeed, the most amazing acts of creativity are those that violate the rules that we would least expect to be violated.

Is there not even creativity on the level of how to think? Wasn't the invention of Science a creative act that violated old beliefs about rationality? Who, then, can lay down a law of creativity?

But there is one law of creativity which cannot be violated...

Ordinarily, if you took a horse-and-buggy, unstrapped the horses, put a large amount of highly combustible fluid on board, and then set fire to the fluid, you would expect the buggy to burn. You certainly wouldn't expect the buggy to move forward at a high rate of speed, for the convenience of its passengers.

How unexpected was the internal combustion engine! How surprising! What a creative act, to violate the rule that you shouldn't start a fire inside your vehicle!

But now suppose that I unstrapped the horses from a buggy, put gasoline in the buggy, and set fire to the gasoline, and it did just explode.

Then there would be no element of "creative surprise" about that. More experienced engineers would just shake their heads wisely and say, "That's why we use horses, kiddo."

Creativity is surprisi...
Recognizing Intelligence
by Eliezer Yudkowsky — 2008-11-07T23:22:35.000Z

Previously in series: Building Something Smarter

Humans in Funny Suits inveighed against the problem of "aliens" on TV shows and movies who think and act like 21st-century middle-class Westerners, even if they have tentacles or exoskeletons. If you were going to seriously ask what real aliens might be like, you would try to make fewer assumptions - a difficult task when the assumptions are invisible.

I previously spoke of how you don't have to start out by assuming any particular goals, when dealing with an unknown intelligence. You can use some of your evidence to deduce the alien's goals, and then use that hypothesis to predict the alien's future achievements, thus making an epistemological profit.

But could you, in principle, recognize an alien intelligence without even hypothesizing anything about its ultimate ends - anything about the terminal values it's trying to achieve?

This sounds like it goes against my suggested definition of intelligence, or even optimization. How can you recognize something as having a strong ability to hit narrow targets in a large search space, if you have no idea what the target is?

And yet, intuitively, it seems easy to imagine a scenario in which we could recognize an alien's intelligence while having no concept whatsoever of its terminal values - having no idea where it's trying to steer the future.

Suppose I landed on an alien planet and discovered what seemed to be a highly sophisticated machine, all gleaming chrome as the stereotype demands. Can I recognize this machine as being in any sense well-designed, if I have no idea what the machine is intended to accomplish? Can I guess that the machine's makers were intelligent, without guessing their motivations?

And again, it seems like in an intuitive sense I should obviously be able to do so. I look at the cables running through the machine, and find large electrical currents passing through them, and discover that the material is a flexible high-temperature high-amperage superconductor. Dozens of gears whir rapidly, perfectly meshed...

I have no idea what the machine is doing. I don't even have a hypothesis as to what it's doing. Yet I have recognized the machine as the product of an alien intelligence. Doesn't this show that "optimization process" is not an indispensable notion to "intelligence"?

But you can't pos...
Dying for a donation
by KatjaGrace — 2008-11-07T01:17:00.000Z

The most outstanding feature of organ markets is that most people hate the idea. This is a curiosity deserving a second glance. There are organ shortages almost everywhere, with people dying on waiting lists hourly. To sentence them to death based on a cursory throb of disgust is not just uncivilised but murderous.

First I should get some technical details out of the way. An organ market can involve buying from living donors, or selling rights to organs after death, or both. Organs needn’t go to the rich preferentially; like any treatment, that depends on the healthcare system. The supply of organs available won’t decrease – if free donations dropped as a result of sales, the price would rise until either enough people sold organs or relatives and friends felt morally obliged to donate them anyway. A regulated market needn’t lead to an increase in stolen Chinese organ imports. It would lower the price here, making smuggling less worthwhile, while stopping Australians going on desperate holidays to seek organs in the under-regulated Third World.

That they ‘commodify the human body’ is the main objection to organ markets. They certainly do that, but why is commodification terrible? Well, a commodity is generally an object subordinated to the goal of making money. Treating other humans in that way leads to abominable actions. Slavery and organ theft are examples of human commodification that rightly repulse us. This doesn’t generalise however. The horror in these examples is that people are being made miserable because they don’t want to be sold. This is a completely different scenario to people voluntarily commodifying themselves.

After all, if commodifying people is inherently wrong, why allow paid labour? Renting out a portion of your time, mind and body to a company or government is surely commodification in the same vein. Or is selling body parts just too much commodification? It doesn’t seem so to me – you can lose more of your most personal possession, your limited lifespan, working than you would selling a kidney. Regardless of how we personally answer that question, there is no reason for the public to decide where the line on commodification should be drawn rather than the people choosing to be involved.

Perhaps anyone who wants to commodify themselves must necessarily be insane and unable to make good choices. To decide that somebody with an alternative ...
Back Up and Ask Whether, Not Why
by Eliezer Yudkowsky — 2008-11-06T19:20:14.000Z

Followup to: The Bottom Line

A recent conversation reminded me of this simple, important, and difficult method:

When someone asks you "Why are you doing X?",
And you don't remember an answer previously in mind,
Do not ask yourself "Why am I doing X?".

For example, if someone asks you
"Why are you using a QWERTY keyboard?" or "Why haven't you invested in stocks?"
and you don't remember already considering this exact question and deciding it,
do not ask yourself "Why am I using a QWERTY keyboard?" or "Why aren't I invested in stocks?"

Instead, try to blank your mind - maybe not a full-fledged crisis of faith, but at least try to prevent your mind from knowing the answer immediately - and ask yourself:

"Should I do X, or not?"

Should I use a QWERTY keyboard, or not? Should I invest in stocks, or not?

When you finish considering this question, print out a traceback of the arguments that you yourself considered in order to arrive at your decision, whether that decision is to X, or not X. Those are your only real reasons, nor is it possible to arrive at a real reason in any other way.

And this is also writing advice: because I have sometimes been approached by people who say "How do I convince people to wear green shoes? I don't know how to argue it," and I reply, "Ask yourself honestly whether you should wear green shoes; then make a list of which thoughts actually move you to decide one way or another; then figure out how to explain or argue them, recursing as necessary."
Hanging Out My Speaker's Shingle
by Eliezer Yudkowsky — 2008-11-05T14:00:00.000Z

I was recently invited to give a talk on heuristics and biases at Jane Street Capital, one of the top proprietary trading firms ("proprietary" = they trade their own money). When I got back home, I realized that (a) I'd successfully managed to work through the trip, and (b) it'd been very pleasant mentally, a nice change of pace. (One of these days I have to blog about what I discovered at Jane Street - it turns out they've got their own rationalist subculture going.)

So I've decided to hang out my shingle as a speaker at financial companies.

You may be thinking: "Perhaps, Eliezer, this is not the best of times."

Well... I do have hopes that, among the firms interested in having me as a speaker, a higher-than-usual percentage will have come out of the crash okay. I checked recently to see if this were the case for Jane Street Capital, and it was.

But more importantly - your competitors are learning the secrets of rationality! Are you?

Or maybe I should frame it as: "Not doing too well this year? Drop the expensive big-name speakers. I can give a fascinating and useful talk and I won't charge you as much."

And just to offer a bit of a carrot - if I can monetize by speaking, I'm much less likely to try charging for access to my future writings. No promises, but something to keep in mind. So do recommend me to your friends as well.

I expect that, as I speak, the marginal value of money to my work will go down; the more I speak, the more my price will go up. If my (future) popular book on rationality becomes a hit, I'll upgrade to big-name fees. And later in my life, if all goes as planned, I'll be just plain not available.

So I'm offering you, my treasured readers, a chance to get me early. I would suggest referencing this page when requesting me as a speaker. Emails will be answered in the order they arrive.
Today's Inspirational Tale
by Eliezer Yudkowsky — 2008-11-04T16:15:24.000Z

At a Foresight Gathering some years ago, a Congressman was in attendance, and he spoke to us and said the following:

"Everyone in this room who's signed up for cryonics, raise your hand."

Many hands went up.

"Now everyone who knows the name of your representative in the House, raise your hand."

Fewer hands went up.

"And you wonder why you don't have any political influence."

Rationalists would likewise do well to keep this lesson in mind.

(I should also mention that voting is a Newcomblike problem. As I don't believe rational agents should defect in the 100fold iterated prisoner's dilemma, I don't buy the idea that rational agents don't vote .)

(See also Stop Voting For Nincompoops. It's more applicable to primaries than to the general election. But a vote for a losing candidate is not "thrown away"; it sends a message to mainstream candidates that you vote, but they have to work harder to appeal to your interest group to get your vote. Readers in non-swing states especially should consider what message they're sending with their vote before voting for any candidate, in any election, that they don't actually like.)
Complexity and Intelligence
by Eliezer Yudkowsky — 2008-11-03T20:27:23.000Z

Followup to: Building Something Smarter , Say Not "Complexity", That Alien Message

One of the Godel-inspired challenges to the idea of self-improving minds is based on the notion of "complexity".

Now "complexity", as I've previously mentioned, is a dangerous sort of word. "Complexity" conjures up the image of a huge machine with incomprehensibly many gears inside - an impressive sort of image. Thanks to this impressiveness, "complexity" sounds like it could be explaining all sorts of things - that all sorts of phenomena could be happening because of "complexity".

It so happens that "complexity" also names another meaning, strict and mathematical: the Kolmogorov complexity of a pattern is the size of the program code of the shortest Turing machine that produces the pattern as an output, given unlimited tape as working memory.

I immediately note that this mathematical meaning, is not the same as that intuitive image that comes to mind when you say "complexity". The vast impressive-looking collection of wheels and gears? That's not what the math term means.

Suppose you ran a Turing machine with unlimited tape, so that, starting from our laws of physics, it simulated our whole universe - not just the region of space we see around us, but all regions of space and all quantum branches. (There's strong indications our universe may be effectively discrete, but if not, just calculate it out to 3^^^3 digits of precision.)

Then the "Kolmogorov complexity" of that entire universe - throughout all of space and all of time, from the Big Bang to whatever end, and all the life forms that ever evolved on Earth and all the decoherent branches of Earth and all the life-bearing planets anywhere, and all the intelligences that ever devised galactic civilizations, and all the art and all the technology and every machine ever built by those civilizations...

...would be 500 bits, or whatever the size of the true laws of physics when written out as equations on a sheet of paper.

The Kolmogorov complexity of just a single planet, like Earth, would of course be much higher than the "complexity" of the entire universe that contains it.

"Eh?" you say. "What's this?" you say. "How can a single planet contain more wheels and gears, more complexity, than the whole vast turning universe that embeds it? Wouldn't one planet contain fewer books...
Building Something Smarter
by Eliezer Yudkowsky — 2008-11-02T17:00:00.000Z

Previously in series: Efficient Cross-Domain Optimization

Once you demystify "intelligence" far enough to think of it as searching possible chains of causality, across learned domains, in order to find actions leading to a future ranked high in a preference ordering...

...then it no longer sounds quite as strange, to think of building something "smarter" than yourself.

There's a popular conception of AI as a tape-recorder-of-thought, which only plays back knowledge given to it by the programmers - I deconstructed this in Artificial Addition, giving the example of the machine that stores the expert knowledge Plus-Of(Seven, Six) = Thirteen instead of having a CPU that does binary arithmetic.

There's multiple sources supporting this misconception:

The stereotype "intelligence as book smarts", where you memorize disconnected "facts" in class and repeat them back.

The idea that "machines do only what they are told to do", which confuses the idea of a system whose abstract laws you designed, with your exerting moment-by-moment detailed control over the system's output.

And various reductionist confusions - a computer is "mere transistors" or "only remixes what's already there" (just as Shakespeare merely regurgitated what his teachers taught him: the alphabet of English letters - all his plays are merely that).

Since the workings of human intelligence are still to some extent unknown, and will seem very mysterious indeed to one who has not studied much cognitive science, it will seem impossible for the one to imagine that a machine could contain the generators of knowledge.

The knowledge-generators and behavior-generators are black boxes, or even invisible background frameworks. So tasking the imagination to visualize "Artificial Intelligence" only shows specific answers, specific beliefs, specific behaviors, impressed into a "machine" like being stamped into clay. The frozen outputs of human intelligence, divorced of their generator and not capable of change or improvement.

You can't build Deep Blue by programming a good chess move for every possible position. First and foremost, you don't know exactly which chess positions the AI will encounter. You would have to record a specific move for zillions of positions, more than you could consider in a lifetim...
BHTV: Jaron Lanier and Yudkowsky
by Eliezer Yudkowsky — 2008-11-01T17:04:06.000Z

My Bloggingheads.tv interview with Jaron Lanier is up. Reductionism, zombies, and questions that you're not allowed to answer:

This ended up being more of me interviewing Lanier than a dialog, I'm afraid. I was a little too reluctant to interrupt. But you at least get a chance to see the probes I use, and Lanier's replies to them.

If there are any BHTV heads out there who read Overcoming Bias and have something they'd like to talk to me about, do let me or our kindly producers know.
Mundane Magic
by Eliezer Yudkowsky — 2008-10-31T16:00:00.000Z

As you may recall from some months earlier, I think that part of the rationalist ethos is binding yourself emotionally to an absolutely lawful reductionistic universe—a universe containing no ontologically basic mental things such as souls or magic—and pouring all your hope and all your care into that merely real universe and its possibilities, without disappointment.

There's an old trick for combating dukkha where you make a list of things you're grateful for, like a roof over your head.

So why not make a list of abilities you have that would be amazingly cool if they were magic, or if only a few chosen individuals had them?

For example, suppose that instead of one eye, you possessed a magical second eye embedded in your forehead. And this second eye enabled you to see into the third dimension—so that you could somehow tell how far away things were—where an ordinary eye would see only a two-dimensional shadow of the true world. Only the possessors of this ability can accurately aim the legendary distance-weapons that kill at ranges far beyond a sword, or use to their fullest potential the shells of ultrafast machinery called "cars".

"Binocular vision" would be too light a term for this ability. We'll only appreciate it once it has a properly impressive name, like Mystic Eyes of Depth Perception.

So here's a list of some of my favorite magical powers:

Vibratory Telepathy. By transmitting invisible vibrations through the very air itself, two users of this ability can share thoughts. As a result, Vibratory Telepaths can form emotional bonds much deeper than those possible to other primates.

Psychometric Tracery. By tracing small fine lines on a surface, the Psychometric Tracer can leave impressions of emotions, history, knowledge, even the structure of other spells. This is a higher level than Vibratory Telepathy as a Psychometric Tracer can share the thoughts of long-dead Tracers who lived thousands of years earlier. By reading one Tracery and inscribing another simultaneously, Tracers can duplicate Tracings; and these replicated Tracings can even contain the detailed pattern of other spells and magics. Thus, the Tracers wield almost unimaginable power as magicians; but Tracers can get in trouble trying to use complicated Traceries that they could not have Traced themselves.

Multidimens

...