trentmkelly/LessWrong-43k huggingface dataset:
When Truth Isn't Enough
by Scott Alexander — 2009-03-22T20:23:50.562Z

Continuation of: The Power of Positivist Thinking

Consider this statement:

The ultra-rich, who control the majority of our planet's wealth, spend their time at cocktail parties and salons while millions of decent hard-working people starve.

A soft positivist would be quite happy with this proposition. If we define "the ultra-rich" as, say, the richest two percent of people, then a quick look at the economic data shows they do control the majority of our planet's wealth. Checking up on the guest lists for cocktail parties and customer data for salons, we find that these two activities are indeed disproportionately enjoyed by the rich, so that part of the statement also seems true enough. And as anyone who's been to India or Africa knows, millions of decent hard-working people do starve, and there's no particular reason to think this isn't happening at the same time as some of these rich people attend their cocktail parties. The positivist scribbles some quick calculations on the back of a napkin and certifies the statement as TRUE. She hands it the Official Positivist Seal of Approval and moves on to her next task.
But the truth isn't always enough. Whoever's making this statement has a much deeper agenda than a simple observation on the distribution of wealth and preferred recreational activities of the upper class, one that the reduction doesn't capture.

Philosophers like to speak of the denotation and the connotation of a word. Denotations (not to be confused with dennettations, which are much more fun) are simple and reducible. To capture the denotation of "old", we might reduce it to something testable like "over 65". Is Methusaleh old? He's over 65, so yes, he is. End of story.
Connotations0 are whatever's left of a word when you subtract the denotation. Is Methusaleh old? How dare you use that word! He's a "senior citizen!" He's "elderly!" He's "in his golden years." Each of these may share the same denotation as "old", but the connotation is quite different.
There is, oddly enough, a children's game about connotations and denotations1. It goes something like this:

I am intelligent. You are clever. He's an egghead.
I am proud. You are arrogant. He's full of himself.
I have perseverance. You are stubborn. He is pig-headed.
I am patriotic. You're a nationalist. He is jingoistic.

Politicians like this game too. Their version goes:

I care about the poor. You

...
Eliezer Yudkowsky Facts
by steven0461 — 2009-03-22T20:17:21.220Z

Eliezer Yudkowsky was once attacked by a Moebius strip. He beat it to death with the other side, non-violently.

Inside Eliezer Yudkowsky's pineal gland is not an immortal soul, but another brain.

Eliezer Yudkowsky's favorite food is printouts of Rice's theorem.

Eliezer Yudkowsky's favorite fighting technique is a roundhouse dustspeck to the face.

Eliezer Yudkowsky once brought peace to the Middle East from inside a freight container, through a straw.

Eliezer Yudkowsky once held up a sheet of paper and said, "A blank map does not correspond to a blank territory". It was thus that the universe was created.

If you dial Chaitin's Omega, you get Eliezer Yudkowsky on the phone.

Unless otherwise specified, Eliezer Yudkowsky knows everything that he isn't telling you.

Somewhere deep in the microtubules inside an out-of-the-way neuron somewhere in the basal ganglia of Eliezer Yudkowsky's brain, there is a little XML tag that says awesome.

Eliezer Yudkowsky is the Muhammad Ali of one-boxing.

Eliezer Yudkowsky is a 1400 year old avatar of the Aztec god Aixitl.

The game of "Go" was abbreviated from "Go Home, For You Cannot Defeat Eliezer Yudkowsky".

When Eliezer Yudkowsky gets bored, he pinches his mouth shut at the 1/3 and 2/3 points and pretends to be a General Systems Vehicle holding a conversation among itselves. On several occasions he has managed to fool bystanders.

Eliezer Yudkowsky has a swiss army knife that has folded into it a corkscrew, a pair of scissors, an instance of AIXI which Eliezer once beat at tic tac toe, an identical swiss army knife, and Douglas Hofstadter.

If I am ignorant about a phenomenon, that is not a fact about the phenomenon; it just means I am not Eliezer Yudkowsky.

Eliezer Yudkowsky has no need for induction or deduction. He has perfected the undiluted master art of duction.

There was no ice age. Eliezer Yudkowsky just persuaded the planet to sign up for cryonics.

There is no spacetime symmetry. Eliezer Yudkowsky just sometimes holds the territory upside down, and he doesn't care.

Eliezer Yudkowsky has no need for doctors. He has implemented a Universal Curing Machine in a system made out of five marbles, three pieces of plastic, and some of MacGyver's fingernail clippings.

Before Bruce Schneier goes to sleep, he scans his computer for uploaded copies of Eliezer Yudkowsky.

If you know more Eliezer Yudkowsky facts, post them in the com...
Cached Selves
by AnnaSalamon — 2009-03-22T19:34:19.719Z

by Anna Salamon and Steve Rayhawk (joint authorship)

Related to: Beware identity

Update, 2021: I believe a large majority of the priming studies failed replication, though I haven't looked into it in depth. I still personally do a great many of the "possible strategies" listed at the bottom; and they subjectively seem useful to me; but if you end up believing that it should not be on the basis of the claimed studies.

A few days ago, Yvain introduced us to priming, the effect where, in Yvain’s words, "any random thing that happens to you can hijack your judgment and personality for the next few minutes."

Today, I’d like to discuss a related effect from the social psychology and marketing literatures: “commitment and consistency effects”, whereby any random thing you say or do in the absence of obvious outside pressure, can hijack your self-concept for the medium- to long-term future.

To sum up the principle briefly: your brain builds you up a self-image. You are the kind of person who says, and does... whatever it is your brain remembers you saying and doing. So if you say you believe X... especially if no one’s holding a gun to your head, and it looks superficially as though you endorsed X “by choice”... you’re liable to “go on” believing X afterwards. Even if you said X because you were lying, or because a salesperson tricked you into it, or because your neurons and the wind just happened to push in that direction at that moment.

For example, if I hang out with a bunch of Green Sky-ers, and I make small remarks that accord with the Green Sky position so that they’ll like me, I’m liable to end up a Green Sky-er myself. If my friends ask me what I think of their poetry, or their rationality, or of how they look in that dress, and I choose my words slightly on the positive side, I’m liable to end up with a falsely positive view of my friends. If I get promoted, and I start telling my employees that of course rule-following is for the best (because I want them to follow my rules), I’m liable to start believing in rule-following in general.

All familiar phenomena, right? You probably already discount other peoples’ views of their friends, and you probably already know that other people mostly stay stuck in their own bad initial ideas. But if you’re like me, you might not have looked carefully into the mechanisms behind these ph...
You're Calling *Who* A Cult Leader?
by Eliezer Yudkowsky — 2009-03-22T06:57:46.809Z

Followup to: Why Our Kind Can't Cooperate, Cultish Countercultishness

I used to be a lot more worried that I was a cult leader before I started reading Hacker News. (WARNING: Do not click that link if you do not want another addictive Internet habit.)

From time to time, on a mailing list or IRC channel or blog which I ran, someone would start talking about "cults" and "echo chambers" and "coteries". And it was a scary accusation, because no matter what kind of epistemic hygeine I try to practice myself, I can't look into other people's minds. I don't know if my long-time readers are agreeing with me because I'm making sense, or because I've developed creepy mind-control powers. My readers are drawn from the nonconformist crowd—the atheist/libertarian/technophile/sf-reader/Silicon-Valley/early-adopter cluster—and so they certainly wouldn't admit to worshipping me even if they were.

And then I ran into Hacker News, where accusations in exactly the same tone were aimed at the site owner, Paul Graham.

Hold on. Paul Graham gets the same flak I do?

Paul Graham has written a word or two about rationality... in a much more matter-of-fact style.

Paul Graham does not ask his readers for donations. He is independently wealthy.

Paul Graham is not dabbling in mad-science-grade AI. He runs Y Combinator, a seed-stage venture fund.

Paul Graham is not trying to save the world. He's trying to help a new generation of entrepreneurs.

I've never heard of Paul Graham saying or doing a single thing that smacks of cultishness. Not one.

He just wrote some great essays (that appeal especially to the nonconformist crowd), and started an online forum where some people who liked those essays hang out (among others who just wandered into that corner of the Internet).

So when I read someone:

Comparing the long hours worked by Y Combinator startup founders to the sleep-deprivation tactic used in cults;

Claiming that founders were asked to move to the Bay Area startup hub as a cult tactic of separation from friends and family;

...well, that outright broke my suspension of disbelief.

Something is going on here which has more to do with the behavior of nonconformists in packs than whether or not you can make a plausible case for cultishness or even cultishness risk factors.

But there are aspects of this phenomenon that ...
Don't Revere The Bearer Of Good Info
by CarlShulman — 2009-03-21T23:22:50.348Z

Follow-up to: Every Cause Wants To Be A Cult, Cultish Countercultishness

One of the classic demonstrations of the Fundamental Attribution Error is the 'quiz study' of Ross, Amabile, and Steinmetz (1977). In the study, subjects were randomly assigned to either ask or answer questions in quiz show style, and were observed by other subjects who were asked to rate them for competence/knowledge. Even knowing that the assignments were random did not prevent the raters from rating the questioners higher than the answerers. Of course, when we rate individuals highly the affect heuristic comes into play, and if we're not careful that can lead to a super-happy death spiral of reverence. Students can revere teachers or science popularizers (even devotion to Richard Dawkins can get a bit extreme at his busy web forum) simply because the former only interact with the latter in domains where the students know less. This is certainly a problem with blogging, where the blogger chooses to post in domains of expertise.

Specifically, Eliezer's writing at Overcoming Bias has provided nice introductions to many standard concepts and arguments from philosophy, economics, and psychology: the philosophical compatibilist account of free will, utility functions, standard biases, and much more. These are great concepts, and many commenters report that they have been greatly influenced by their introductions to them at Overcoming Bias, but the psychological default will be to overrate the messenger. This danger is particularly great in light of his writing style, and when the fact that a point is already extant in the literature, and is either being relayed or reinvented, isn't noted. To address a few cases of the latter: Gary Drescher covered much of the content of Eliezer's Overcoming Bias posts (mostly very well), from timeless physics to Newcomb's problems to quantum mechanics, in a book back in May 2006, while Eliezer's irrealist meta-ethics would be very familiar to modern philosophers like Don Loeb or Josh Greene, and isn't so far from the 18th century philosopher David Hume.

If you're feeling a tendency to cultish hero-worship, reading such independent prior analyses is a noncultish way to diffuse it, and the history of science suggests that this procedure will be applicable to almost anyone you're tempted to revere. Wallace invented the idea of evolution through natural selectio

...
The Power of Positivist Thinking
by Scott Alexander — 2009-03-21T20:55:37.146Z

Related to: No Logical Positivist I, Making Beliefs Pay Rent, How An Algorithm Feels From Inside, Disguised Queries

Call me non-conformist, call me one man against the world, but...I kinda like logical positivism.
The logical positivists were a dour, no-nonsense group of early 20th-century European philosophers. Indeed, the phrase "no-nonsense" seems almost invented to describe the Positivists. They liked nothing better then to reject the pet topics of other philosophers as being untestable and therefore meaningless. Is the true also the beautiful? Meaningless! Is there a destiny to the affairs of humankind? Meaningless? What is justice? Meaningless! Are rights inalienable? Meaningless!
Positivism became stricter and stricter, defining more and more things as meaningless, until someone finally pointed out that positivism itself was meaningless by the positivists' definitions, at which point the entire system vanished in a puff of logic. Okay, it wasn't that simple. It took several decades and Popper's falsifiabilism to seal its coffin. But vanish it did. It remains one of the least lamented theories in the history of philosophy, because if there is one thing philosophers hate it's people telling them they can't argue about meaningless stuff.
But if we've learned anything from fantasy books, it is that any cabal of ancient wise men destroyed by their own hubris at the height of their glory must leave behind a single ridiculously powerful artifact, which in the right hands gains the power to dispel darkness and annihilate the forces of evil.
The positivists left us the idea of verifiability, and it's time we started using it more.

Eliezer, in No Logical Positivist I, condemns the positivist notion of verifiability for excluding some perfectly meaningful propositions. For example, he says, it may be that a chocolate cake formed in the center of the sun on 8/1/2008, then disappeared after one second. This statement seems to be meaningful; that is, there seems to be a difference between it being true or false. But there's no way to test it (at least without time machines and sundiver ships, which we can't prove are possible) so the logical positivists would dismiss it as nonsense.
I am not an expert in logical positivism; I have two weeks studying positivism in an undergrad philosophy class under my belt, and little more. If Eliezer says that is how the positivists inte...
Individual Rationality Is a Matter of Life and Death
by patrissimo — 2009-03-21T19:22:02.606Z

On at least two occasions - one only a year past - my life was at serious risk because I was not thinking clearly. Both times, I was lucky (and once, the car even survived!). As a gambler I don't like counting on luck, and I'd much rather be rational enough to avoid serious mistakes. So when I checked the top-ranked posts here and saw Robin's Rational Me or We? arguing against rationality as a martial art I was dumbfounded. To me, individual rationality is a matter of life and death[1].

In poker, much attention is given to the sexy art of reading your opponent, but the true veteran knows that far more important is the art of reading and controlling yourself. It is very rare that a situation comes up where a "tell" matters, and each of my opponents is only in an occasional hand. I and my irrationalities, however, are in every decision in every hand. This is why self-knowledge and self-discipline are first-order concerns in poker, while opponent reading is second or perhaps even third.

And this is why Robin's post is so wrong[2]. Our minds and their irrationalities are part of every second of our lives, every moment we experience, and every decision that we make. And contra to Robin's security metaphor, few of our decisions can be outsourced. My two bad decisions regarding motor vehicles, for example, could not have easily been outsourced to a group rationality mechanism[3]. Only a tiny percentage of the choices I make every day can be punted to experts.

We have long since left the Hobbesian world where physical security depends on individual skills, but when it comes to rationality, we are all "isolated survivalist Einsteins". We are in a world where our individual mental skills are constantly put to the test. And even when we can rely on experts, it is our individual choices (influenced by the quality of our minds) that determine our success in life. (How long would a professor's reputation last if he never did any original work?)

So while I respect and admire Robin's interest in improving institutions, I believe that his characterization of the relative merits of individual and collective mechanisms is horridly wrong. To have more and better rational collective institutions is a speculative, long-term goal with limited scope (albeit in some very important areas). Lea...
Mind Control and Me
by Patrick — 2009-03-21T17:31:33.340Z

Reading Eliezer Yudkowsky's works have always inspired an insidious feeling in me, sort of a cross between righteousness, contempt, the fun you get from understanding something new and gravitas. It's a feeling that I have found to be pleasurable, or at least addictive enough to go through all of his OB posts, and the feeling makes me less skeptical and more obedient than I normally would be. For instance, in an act of uncharacteristic generosity, I decided to make a charitable donation on Eliezer's advice.

Now this is probably a good idea, because the charity is probably going to help guys like me later on in life and of course it's the Right Thing to Do. But the bottom line is that I did something I normally wouldn't have because Eliezer told me to. My sociopathic selfishness was acting as canary in the mine of my psyche.

Now this could be because Eliezer has creepy mind control powers, but I get similar feelings when reading other people, such as George Orwell, Richard Stallman or Paul Graham. I even have a friend who can inspire that insidious feeling in me. So it's a personal problem, one that I'm not sure I want to remove, but I would like to understand it better.

There are probably buttons being pushed by the style and the sort of ideas in the work that help to create the feeling, and I'll probably try to go over an essay or two and dissect it. However, I'd like to know who and at what times, if anyone at all, I should let create such feelings in me. Can I trust anyone that much, even if they aren't aware that they're doing it?

I don't know if anyone else here has similar brain overrides, or if I'm just crazy, but it's possible that such brain overrides could be understood much more thoroughly and induced in more people. So what are the ethics of mind control (for want of a better term) and how much effort should we put in to stopping such feelings from occuring?

Edit Mar 22: Decided to remove the cryonics example due to factual inaccuracies.
Tolerate Tolerance
by Eliezer Yudkowsky — 2009-03-21T07:34:12.259Z

One of the likely characteristics of someone who sets out to be a "rationalist" is a lower-than-usual tolerance for flaws in reasoning. This doesn't strictly follow. You could end up, say, rejecting your religion, just because you spotted more or deeper flaws in the reasoning, not because you were, by your nature, more annoyed at a flaw of fixed size. But realistically speaking, a lot of us probably have our level of "annoyance at all these flaws we're spotting" set a bit higher than average.

That's why it's so important for us to tolerate others' tolerance if we want to get anything done together.

For me, the poster case of tolerance I need to tolerate is Ben Goertzel, who among other things runs an annual AI conference, and who has something nice to say about everyone. Ben even complimented the ideas of M*nt*f*x, the most legendary of all AI crackpots. (M*nt*f*x apparently started adding a link to Ben's compliment in his email signatures, presumably because it was the only compliment he'd ever gotten from a bona fide AI academic.) (Please do not pronounce his True Name correctly or he will be summoned here.)

But I've come to understand that this is one of Ben's strengths—that he's nice to lots of people that others might ignore, including, say, me—and every now and then this pays off for him.

And if I subtract points off Ben's reputation for finding something nice to say about people and projects that I think are hopeless—even M*nt*f*x—then what I'm doing is insisting that Ben dislike everyone I dislike before I can work with him.

Is that a realistic standard? Especially if different people are annoyed in different amounts by different things?

But it's hard to remember that when Ben is being nice to so many idiots.

Cooperation is unstable, in both game theory and evolutionary biology, without some kind of punishment for defection. So it's one thing to subtract points off someone's reputation for mistakes they make themselves, directly. But if you also look askance at someone for refusing to castigate a person or idea, then that is punishment of non-punishers, a far more dangerous idiom that can lock an equilibrium in place even if it's harmful to everyone involved.

The danger of punishing nonpunishers is something I remind myself of, say, every time Robin Hanson points out a flaw in some academic trope and yet ...
Support That Sounds Like Dissent
by jimrandomh — 2009-03-20T22:28:03.357Z

Related to: Why Our Kind Can't Cooperate

Eliezer described a scene that's familiar to all of us:

Imagine that you're at a conference, and the speaker gives a 30-minute talk. Afterward, people line up at the microphones for questions. The first questioner objects to the graph used in slide 14 using a logarithmic scale; he quotes Tufte on The Visual Display of Quantitative Information. The second questioner disputes a claim made in slide 3. The third questioner suggests an alternative hypothesis that seems to explain the same data...

An outsider might conclude that this presentation went poorly, because all of the people who spoke afterwards seemed to disagree. Someone who had been to a few conferences would understand that this is normal; only the people who disagree speak up, while the the rest stay silent, because taking the mic to say "me too!" isn't a productive use of everyone's time. If you polled the audience, you might expect to find a few vocal dissenters against a silent majority. This is not what you would find.

Consider this situation more closely. A series of people step up, and say things which sound like disagreement. But do they really disagree? The first questioner is only quibbling with a bit of the presentation; he hasn't actually disagreed with the main point. The second questioner has challenged one of the claims from the presentation, but ignored the rest. The third questioner has proposed an alternative hypothesis which might be true, but that doesn't mean the alternative hypothesis is true, or even that the questioner thinks it's likely. If you stopped and asked these questioners whether they agreed with the main thrust of the presentation, they would probably say that they do. Why, then, does it sound like everyone disagrees?

In our community, we hold writing and arguments to a high standard, so when we see something that's imperfect, we speak up. Posting merely to indicate agreement ("me too") is strongly discouraged. In practice, this often translates into nit-picking: pointing out minor issues in reasoning or presentation that are easy to fix. We have lots of practice nit-picking, because we do it all the time with our own writings. We remove or rework weak arguments, expand on points that need clarification and tweak explanations with every draft. Revising a draft, however, does not mean questioning its prem...