trentmkelly/LessWrong-43k huggingface dataset:
Tell Your Rationalist Origin Story... at Less Wrong
by Eliezer Yudkowsky — 2009-02-27T04:06:15.000Z

(A beta version of Less Wrong is now live, no old posts imported as yet. Some of the plans for what to do with Less Wrong relative to OB have been revised by further discussion among Robin, Nick, and myself, but for now we're just seeing what happens once LW is up - whether it's stable, what happens to the tone of comments once threading and voting is enabled, etcetera.
Posting by non-admins is disabled for now - today we're just testing out registration, commenting, threading, etcetera.)

To
break up the awkward silence at the start of a recent Overcoming Bias
meetup, I asked everyone present to tell their rationalist origin story
- a key event or fact that played a role in their becoming rationalists. This worked surprisingly well.

I think I've already told enough of my own origin story on Overcoming Bias:
how I was digging in my parents' yard as a kid and found a tarnished
silver amulet inscribed with Bayes's Theorem, and how I wore it to bed
that night and dreamed of a woman in white, holding a leather-bound
book called Judgment Under Uncertainty: Heuristics and Biases (eds. D. Kahneman, P. Slovic, and A. Tversky, 1982)... but there's no need to go into that again.

So, seriously... how did you originally go down that road?

Continue reading "Tell Your Rationalist Origin Story" at Less Wrong »
Issues, Bugs, and Requested Features
by Eliezer Yudkowsky — 2009-02-26T16:45:28.803Z

[Edit: Issues, Bugs, and Requested Features should be tracked at Google Code, not here -- matt, 2010-04-23]

Less Wrong is still under construction. Please post any bugs or issues with Less Wrong to this thread. Try to keep each comment thread a clean discussion of each bug or issue.

Requested features... sure, go ahead, but bear in mind we may not be able to implement for a while.
Markets are Anti-Inductive
by Eliezer Yudkowsky — 2009-02-26T00:55:33.000Z

I suspect there's a Pons Asinorum of probability between the bettor who thinks that you make money on horse races by betting on the horse you think will win, and the bettor who realizes that you can only make money on horse races if you find horses whose odds seem poorly calibrated relative to superior probabilistic guesses.

There is, I think, a second Pons Asinorum associated with more advanced finance, and it is the concept that markets are an anti-inductive environment.

Let's say you see me flipping a coin. It is not necessarily a fair coin. It's a biased coin, and you don't know the bias. I flip the coin nine times, and the coin comes up "heads" each time. I flip the coin a tenth time. What is the probability that it comes up heads?

If you answered "ten-elevenths, by Laplace's Rule of Succession", you are a fine scientist in ordinary environments, but you will lose money in finance.

In finance the correct reply is, "Well... if everyone else also saw the coin coming up heads... then by now the odds are probably back to fifty-fifty."

Recently on Hacker News I saw a commenter insisting that stock prices had nowhere to go but down, because the economy was in such awful shape. If stock prices have nowhere to go but down, and everyone knows it, then trades won't clear - remember, for every seller there must be a buyer - until prices have gone down far enough that there is once again a possibility of prices going up.

So you can see the bizarreness of someone saying, "Real estate prices
have gone up by 10% a year for the last N years, and we've never
seen a drop." This treats the market like it was the mass of an
electron or something. Markets are anti-inductive. If, historically, real estate prices have always gone up, they will
keep rising until they can go down.

To get an excess return - a return that pays premium interest over the going rate for that level of riskiness - you need to know something that other market participants don't, or they will rush in and bid up whatever you're buying (or bid down whatever you're selling) until the returns match prevailing rates.

If the economy is awful and everyone knows it, no one's going to buy at
a price that doesn't take into account that knowledge.

If there's an
obviou...
Tell Your Rationalist Origin Story
by Eliezer Yudkowsky — 2009-02-25T17:16:11.626Z

To break up the awkward silence at the start of a recent Overcoming Bias meetup, I asked everyone present to tell their rationalist origin story - a key event or fact that played a role in their first beginning to aspire to rationality. This worked surprisingly well (and I would recommend it for future meetups).

I think I've already told enough of my own origin story on Overcoming Bias: how I was digging in my parents' yard as a kid and found a tarnished silver amulet inscribed with Bayes's Theorem, and how I wore it to bed that night and dreamed of a woman in white, holding an ancient leather-bound book called Judgment Under Uncertainty: Heuristics and Biases (eds. D. Kahneman, P. Slovic, and A. Tversky, 1982)... but there's no need to go into that again.

So, seriously... how did you originally go down that road?

Added: For some odd reason, many of the commenters here seem to have had a single experience in common - namely, at some point, encountering Overcoming Bias... But I'm especially interested in what it takes to get the transition started - crossing the first divide. This would be very valuable knowledge if it can be generalized. If that did happen at OB, please try to specify what was the crucial "Aha!" insight (down to the specific post if possible).
Formative Youth
by Eliezer Yudkowsky — 2009-02-24T23:02:35.000Z

Followup to: Against Maturity

"Rule of thumb: Be skeptical of things you learned before you could read. E.g., religion."
-- Ben Casnocha

Looking down on others is fun, and if there's one group we adults can all enjoy looking down on, it's children. At least I assume this is one of the driving forces behind the incredible disregard for... but don't get me started.

Inconveniently, though, most of us were children at one point or another during our lives. Furthermore, many of us, as adults, still believe or choose certain things that we happened to believe or choose as children. This fact is incongruent with the general fun of condescension - it means that your life is being run by a child, even if that particular child happens to be your own past self.

I suspect that most of us therefore underestimate the degree to which our youths were formative - because to admit that your youth was formative is to admit that the course of your life was not all steered by Incredibly Deep Wisdom and uncaused free will.

To give a concrete example, suppose you asked me, "Eliezer, where does your altruism originally come from? What was the very first step in the chain that made you amenable to helping others?"

Then my best guess would be "Watching He-Man and similar TV shows as a very young and impressionable child, then failing to compartmentalize the way my contemporaries did." (Same reason my Jewish education didn't take; I either genuinely believed something, or didn't believe it at all. (Not that I'm saying that I believed He-Man was fact; just that the altruistic behavior I picked up wasn't compartmentalized off into some safely harmless area of my brain, then or later.))

It's my understanding that most people would be reluctant to admit this sort of historical fact, because it makes them sound childish - in the sense that they're still being governed by the causal history of a child.

But I find myself skeptical that others are governed by their childhood causal histories so much less than myself - especially when there's a simple alternative explanation: they're too embarrassed to admit it.

A lovely excuse, of course, is that we at first ended up in a certain place for childish reasons, and then we went back and redid the calculations as adults, and what do you know, it magically ended up ...
About Less Wrong
by Eliezer Yudkowsky — 2009-02-23T23:30:48.747Z

Edit: This post refers to the original version of LessWrong, which ran between February 2009 and March 2018. The About page referring to the period from March 2018 to the present can be found here.

Over the last decades, new experiments have changed science's picture of the way we think - the ways we succeed or fail to obtain the truth, or fulfill our goals. The heuristics and biases program, in cognitive psychology, has exposed dozens of major flaws in human reasoning. Social psychology shows how we succeed or fail in groups. Probability theory and decision theory have given us new mathematical foundations for understanding minds.

Less Wrong is devoted to refining the art of human rationality - the art of thinking. The new math and science deserves to be applied to our daily lives, and heard in our public voices.

Less Wrong consists of three areas: The main community blog, the Less Wrong wiki and the Less Wrong discussion area.

Less Wrong is a partially moderated community blog that allows general authors to contribute posts as well as comments. Users vote posts and comments up and down (with code based on Reddit's open source). "Promoted" posts (appearing on the front page) are chosen by the editors on the basis of substantive new content, clear argument, good writing, popularity, and importance.

We suggest submitting links with a short description. Recommended books should have longer descriptions. Links will not be promoted unless they are truly excellent - the "promoted" posts are intended as a filtered stream for the casual/busy reader.

The Less Wrong discussion area is for topics not yet ready or not suitable for normal top level posts. To post a new discussion, select "Post to: Less Wrong Discussion" from the Create new article page. Comment on discussion posts as you would elsewhere on the site.

Votes on posts are worth ±10 points on the main site and ±1 point in the discussion area. Votes on comments are worth ±1 point. Users with sufficient karma can publish posts. You need 20+ points to post to the main area and 2+ points to post to the discussion area. You can only down vote up to four times your current karma (thus if you never comment, you cannot downvote). Comments voted to -3 or lower will be collapsed by default for most readers (if you log in, you can change this setting in your Preferences). Please keep this in mi...
On Not Having an Advance Abyssal Plan
by Eliezer Yudkowsky — 2009-02-23T20:20:31.000Z

"Even though he could foresee the problem then, we can see it equally well now. Therefore, if he could foresee the solution then, we should be able to see it now. After all, Seldon was not a magician. There are no trick methods of escaping a dilemma that he can see and we can't."
-- Salvor Hardin

Years ago at the Singularity Institute, the Board was entertaining a proposal to expand somewhat. I wasn't sure our funding was able to support the expansion, so I insisted that - if we started running out of money - we decide in advance who got fired and what got shut down, in what order.

Even over the electronic aether, you could hear the uncomfortable silence.

"Why can't we decide that at the time, if the worst happens?" they said, or something along those lines.

"For the same reason that when you're buying a stock you think will go up, you decide how far it has to decline before it means you were wrong," I said, or something along those lines; this being far back enough in time that I would still have used stock-trading in a rationality example. "If we can make that decision during a crisis, we ought to be able to make it now. And if I can't trust that we can make this decision in a crisis, I can't trust this to go forward."

People are really, really reluctant to plan in advance for the abyss. But what good reason is there not to? How can you be worse off from knowing in advance what you'll do in the worse cases?

I have been trying fairly hard to keep my mouth shut about the current economic crisis. But still -

Why didn't various governments create and publish a plan for what they would do in the event of various forms of financial collapse, before it actually happened?

Never mind hindsight on the real-estate bubble - there are lots of things that could potentially trigger financial catastrophes. I'm willing to bet the American government knows what it will do in terms of immediate rescue operations if an atomic bomb goes off in San Francisco. But if the US government had any advance idea of under which circumstances it would nationalize Fannie Mae or guarantee Bear Stearns's counterparties, this plan was not very much in evidence as various government officials gave every...
Fairness vs. Goodness
by Eliezer Yudkowsky — 2009-02-22T20:22:00.000Z

It seems that back when the Prisoner's Dilemma was still being worked out, Merrill Flood and Melvin Drescher tried a 100-fold iterative PD on two smart but unprepared subjects, Armen Alchian of UCLA and John D. Williams of RAND.

The kicker being that the payoff matrix was asymmetrical, with dual cooperation awarding JW twice as many points as AA:

(AA, JW)	JW: D	JW: C
AA: D	(0, 0.5)	(1, -1)
AA: C	(-1, 2)	(0.5, 1)

The resulting 100 iterations, with a log of comments written by both players, make for fascinating reading.

JW spots the possibilities of cooperation right away, while AA is slower to catch on.

But once AA does catch on to the possibilities of cooperation, AA goes on throwing in an occasional D... because AA thinks the natural meeting point for cooperation is a fair outcome, where both players get around the same number of total points.

JW goes on trying to enforce (C, C) - the option that maximizes total utility for both players - by punishing AA's attempts at defection. JW's log shows comments like "He's crazy. I'll teach him the hard way."

Meanwhile, AA's log shows comments such as "He won't share. He'll punish me for trying!"

I confess that my own sympathies lie with JW, and I don't think I would have played AA's game in AA's shoes. This would seem to indicate that I'm more of a utilitarian than a fair-i-tarian. Life doesn't always hand you fair games, and the best we can do for each other is play them positive-sum.

Though I might have been somewhat more sympathetic to AA, if the (C, C) outcome had actually lost him points, and only (D, C) had made it possible for him to gain them back. For example, this is also a Prisoner's Dilemma:

(AA, JW)	JW: D	JW: C
AA: D	(-2, 2)	(2, 0)
AA: C	(-5, 6)	(-1, 4)

Theoretically, of course, utility functions are invariant up to affine transformation, so a utility's absolute sign is not meaningful. But this is not always a good metaphor for real life.

Of course what we want in this case, societally speaking, is for JW to slip AA a bribe under the table. That way we can maximize social utility while letting AA go on making a profit. But if AA starts out with a negative number in (C,
C), how much do we want AA to demand in bribes - from our global, societal perspective?

The whole affair makes for an int...
Rationality Quotes 27
by Eliezer Yudkowsky — 2009-02-22T01:55:48.000Z

"Believing this statement will make you happier."
-- Ryan Lortie

"Make changes based on your strongest opportunities, not your most convenient ones."
-- MegaTokyo

"The mind is a cruel, lying, unreliable bastard that can't be trusted with even an ounce of responsibility. If you were dating the mind, all your friends would take you aside, and tell you that you can really do better, and being alone isn't all that bad, anyway. If you hired the mind as a babysitter, you would come home to find all but one of your children in critical condition, and the remaining one crowned 'King of the Pit'."
-- Lore Sjoberg

"Getting bored is a non-trivial cerebral transformation that doubtlessly took many millions of years for nature to perfect."
-- Lee Corbin

"The views expressed here do not necessarily represent the unanimous view of all parts of my mind."
-- Malcolm McMahon

"The boundary between these two classes is more porous than I've made it sound. I'm always running into regular dudes--construction workers, auto mechanics, taxi drivers, galoots in general--who were largely aliterate until something made it necessary for them to become readers and start actually thinking about things. Perhaps they had to come to grips with alcoholism, perhaps they got sent to jail, or came down with a disease, or suffered a crisis in religious faith, or simply got bored. Such people can get up to speed on particular subjects quite rapidly. Sometimes their lack of a broad education makes them over-apt to go off on intellectual wild goose chases, but, hey, at least a wild goose chase gives you some exercise."
-- Neal Stephenson, In the Beginning was the Command Line
Wise Pretensions v.0
by Eliezer Yudkowsky — 2009-02-20T17:02:28.000Z

Followup to: Pretending to be Wise

For comparison purposes, here's an essay with similar content to yesterday's "Pretending to be Wise", which I wrote in 2006 in a completely different style, edited down slightly (content has been deleted but not added). Note that the 2006 concept of "pretending to be Wise" hasn't been narrowed down as much compared to the 2009 version; also when I wrote it, I was in more urgent need of persuasive force.

I thought it would be an interesting data point to check whether this essay seems more convincing than yesterday's, following Robin's injuction "to avoid emotion, color, flash, stories, vagueness, repetition, rambling, and even eloquence" - this seems like rather the sort of thing he might have had in mind.

And conversely the stylistic change also seems like the sort of thing Orwell might have had in mind, when Politics and the English Language compared: "I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all." Versus: "Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account." That would be the other side of it.

At any rate, here goes Eliezer2006...

I do not fit the stereotype of the Wise. I am not Gandalf, Ged, or Gandhi. I do not sit amidst my quiet garden, staring deeply into the truths engraved in a flower or a drop of dew; speaking courteously to all who come before me, and answering them gently regardless of how they speak to me.

If I tried to look Wise, and succeeded, I would receive more respect from my fellows. But there would be a price.

To pretend to be Wise means that you must always appear to give people the benefit of the doubt. Thus people will admire you for your courtesy. But this is not always true.

To pretend to be Wise, you must always pretend that both sides have merit, and solemnly refuse to judge between them. For if you took one side or another, why then, you would no longer be one of the aloof Wise, but merely another partisan, on a level with all the other mere bicke...
Pretending to be Wise
by Eliezer Yudkowsky — 2009-02-19T22:30:22.000Z

The hottest place in Hell is reserved for those who in time of crisis remain neutral.

—Dante Alighieri, famous hell expert

—John F. Kennedy, misquoter

Belief is quantitative, and just as it is possible to make overconfident assertions relative to ones anticipations, it is possible to make under confident assertions relative to ones anticipations. One can wear the attire of uncertainty, or profess an agnosticism that isn’t really there. Here, I'll single out a special case of improper uncertainty: the display of neutrality or suspended judgment in order to signal maturity, impartiality, or a superior vantage point.

An example would be the case of my parents, who respond to theological questions like “Why does ancient Egypt, which had good records on many other matters, lack any records of Jews having ever been there?” with “Oh, when I was your age, I also used to ask that sort of question, but now I’ve grown out of it.”

Another example would be the principal who, faced with two children who were caught fighting on the playground, sternly says: “It doesn’t matter who started the fight, it only matters who ends it.” Of course it matters who started the fight. The principal may not have access to good information about this critical fact, but if so, the principal should say so, not dismiss the importance of who threw the first punch. Let a parent try punching the principal, and we’ll see how far “It doesn’t matter who started it” gets in front of a judge. But to adults it is just inconvenient that children fight, and it matters not at all to their convenience which child started it. It is only convenient that the fight end as rapidly as possible.

A similar dynamic, I believe, governs the occasions in international diplomacy where Great Powers sternly tell smaller groups to stop that fighting right now. It doesn’t matter to the Great Power who started it—who provoked, or who responded disproportionately to provocation—because the Great Power’s ongoing inconvenience is only a function of the ongoing conflict. Oh, can’t Israel and Hamas just get along?

This I call “pretending to be Wise.” Of course there are many ways to try and signal wisdom. But trying to signal wisdom by refusing to make guesses—refusing to sum up evidence—refusing to pass judgment—refusing to take sides—staying above the fray and looking down with a lofty and condescending gaze—which is to ...
Against Maturity
by Eliezer Yudkowsky — 2009-02-18T23:34:51.000Z

I remember the moment of my first break with Judaism. It was in kindergarten, when I was being forced to memorize and recite my first prayer. It was in Hebrew. We were given a transliteration, but not a translation. I asked what the prayer meant. I was told that I didn't need to know - so long as I prayed in Hebrew, it would work even if I didn't understand the words. (Any resemblance to follies inveighed against in my writings is not coincidental.)

Of course I didn't accept this, since it was blatantly stupid, and I figured that God had to be at least as smart as I was. So when I got home, I asked my parents, and they didn't bother arguing with me. They just said, "You're too young to argue with; we're older and wiser; adults know best; you'll understand when you're older."

They were right about that last part, anyway.

Of course there were plenty of places my parents really did know better, even in the realms of abstract reasoning. They were doctorate-bearing folks and not stupid. I remember, at age nine or something silly like that, showing my father a diagram full of filled circles and trying to convince him that the indeterminacy of particle collisions was because they had a fourth-dimensional cross-section and they were bumping or failing to bump in the fourth dimension.

My father shot me down flat. (Without making the slightest effort to humor me or encourage me. This seems to have worked out just fine. He did buy me books, though.)

But he didn't just say, "You'll understand when you're older." He said that physics was math and couldn't even be talked about
without math. He talked about how everyone he met tried to invent
their own theory of physics and how annoying this was. He may even
have talked about the futility of "providing a mechanism", though I'm
not actually sure if I originally got that off him or Baez.

You see the pattern developing here. "Adulthood" was what my parents appealed to when they couldn't verbalize any object-level justification. They had doctorates and were smart; if there was a good reason, they usually would at least try to explain it to me. And it gets worse...

The most fearsome damage wreaked upon my pa...