trentmkelly/LessWrong-43k huggingface dataset:
On the Power of Intelligence and Rationality
by alyssavance — 2009-12-23T10:49:44.496Z

As Eliezer and many others on Less Wrong have said, the way the human species rose to dominate the Earth was through our intelligence- and not through our muscle power, biochemical weapons, or superior resistance to environmental hazards. Given our overwhelming power over other species, and the fact that many former top predators are now extinct or endangered, we should readily accept that general intelligence is a game-changing power on the species level.

Similarly, one of the key ingredients in the birth of the modern era was the discovery of science, and its counterpart, the discovery of the art of Traditional Rationality. Armed with these, the nations of Western Europe managed to dominate the entire rest of the world, even though, when they began their explorations in the 15th century, the Chinese were more advanced in many respects. Given how Western Europe, and the cultures derived from it, has so completely surpassed the rest of the world in terms of wealth and military might, we should readily accept that science and rationality is a game-changing power on the civilization level.

However, neither of these imply that intelligence, science, and rationality, as a practical matter, are the best way to get things done by individual people operating in the year 2009. We can easily see that many things which work on the species level, or the civilization level, do not work for individuals and small groups. For instance, until the discovery of nuclear weapons, armed conflict was often a primary means of settling disputes between nation-states. However, if you tried to settle your dispute with your neighbor, or your company's dispute with its competitor, using armed force, it would achieve nothing except getting you thrown in prison.

People are crazy and the world is mad, but it does not necessarily follow that we should try to solve our own problems primarily by becoming more sane. Plenty of people achieve many of their goals despite being completely nuts. Adolf Hitler, for example, achieved a large fraction of his (extremely ambitious!) goals, despite having numerous beliefs that most of us would recognize as making no sense whatsoever.

We know, as a matter of historical fact, that Adolf Hitler and his Nazi Party, despite being generally incompetent, unintelligent, irrational, superstitious and just plain insane, managed to take over a country of tens of millio...
Two Truths and a Lie
by Psychohistorian — 2009-12-23T06:34:55.204Z

Response to Man-with-a-hammer syndrome.

It's been claimed that there is no way to spot Affective Death Spirals, or cultish obsession with the One Big Idea of Everything. I'd like to posit a simple way to spot such error, with the caveat that it may not work for every case.

There's an old game called Two Truths and a Lie. I'd bet almost everyone's heard of it, but I'll summarize it just in case. A person makes three statements, and the other players must guess which of those statements is false. The statement-maker gets points for fooling people, people get points for not being fooled. That's it. I'd like to propose a rationalist's version of this game that should serve as a nifty check on certain Affective Death Spirals, runaway Theory-Of-Everythings, and Perfectly General Explanations. It's almost as simple.

Say you have a theory about human behaviour. Get a friend to do a little research and assert three factual claims about how people behave that your theory would realistically apply to. At least one of these claims must be false. See if you can explain every claim using your theory before learning which one's false.

If you can come up with a convincing explanation for all three statements, you must be very cautious when using your One Theory. If it can explain falsehoods, there's a very high risk you're going to use it to justify whatever prior beliefs you have. Even worse, you may use it to infer facts about the world, even though it is clearly not consistent enough to do so reliably. You must exercise the utmost caution in applying your One Theory, if not abandon reliance on it altogether. If, on the other hand, you can't come up with a convincing way to explain some of the statements, and those turn out to be the false ones, then there's at least a chance you're on to something.

Come to think of it, this is an excellent challenge to any proponent of a Big Idea. Give them three facts, some of which are false, and see if their Idea can discriminate. Just remember to be ruthless when they get it wrong; it doesn't prove their idea is totally wrong, only that reliance upon it would be.

Edited to clarify: My argument is not that one should simply abandon a theory altogether. In some cases, this may be justified, if all the theory has going for it is its predictive power, and you show it lacks that, toss it. But in the case of broad, complex theories that ac...
The 9/11 Meta-Truther Conspiracy Theory
by Eliezer Yudkowsky — 2009-12-22T18:59:05.811Z

Date: September 11th, 2001.
Personnel: Unknown [designate A], Unknown [designate B], Unknown [designate C].

A: It's done. The plane targeted at Congress was crashed by those on-board, but the Pentagon and Trade Center attacks occurred just as scheduled.

B: Congress seems sufficiently angry in any case. I don't think the further steps of the plan will meet with any opposition. We should gain the governmental powers we need, and the stock market should move as expected.

A: Good. Have you prepared the conspiracy theorists to accuse us?

B: Yes. All is in readiness. The first accusations will fly within the hour.

C: Er...

A: What is it?

C: Sorry, I know I'm a bit new to this sort of thing, but why are we sponsoring conspiracy theorists? Aren't they our arch-nemeses, tenaciously hunting down and exposing our lies?

A: No, my young apprentice, just the opposite. As soon as you pull off a conspiracy, the first thing you do is start a conspiracy theory about it. Day one.

C: What? We want to be accused of deliberately ignoring intelligence and assassinating that one agent who tried to forward specific information -

A: No, of course not! What you do in a case like this is start an accusation so ridiculous that nobody else wants to be associated with the accusers. You create a low-prestige conspiracy theory and staff it with as many vocal lunatics as you can. That way no one wants to be seen as affiliating with the conspiracy theorists by making a similar accusation.

C: That works? I know I'm not the brightest fish in the barrel - sometimes, hanging around you guys, I feel almost as dumb as I pretend to be - but even I know that "The world's stupidest man may say the sun is shining, but that doesn't make it dark out."

B: Works like a charm, in my experience. Like that business with the Section Magenta aircraft. All you need is a bunch of lunatics screaming about aliens and no one respectable will dream of reporting a "flying saucer" after that.

C: So what did you plan for the 9/11 cover conspiracy theory, by the way? Are the conspiracy theorists going to say the Jews were behind it? Can't get much lower-prestige than anti-Semitism!

B: You've got the right general idea, but you're not t...
lessmeta
by PlaidX — 2009-12-22T17:57:21.810Z

The social bookmarking site metafilter has a sister site called metatalk, which works the same way but is devoted entirely to talking about metafilter itself. Arguments about arguments, discussions about discussions, proposals for changes in site architecture, etc.

Arguments about arguments are often less productive than the arguments they are about, but they CAN be quite productive, and there's certainly a place for them. The only thing wrong with them is when they obstruct the discussion that spawned them, and so the idea of splitting off metatalk into its own site is really quite a clever one.

Lesswrong's problem is a peculiar one. It is ENTIRELY devoted to meta-arguments, to the extent that people have to shoehorn anything else they want to talk about into a cleverly (or not so cleverly) disguised example of some more meta topic. It's a kite without a string.

Imagine if you had been around the internet, trying to have a rational discussion about topic X, but unable to find an intelligent venue, and then stumbling upon lesswrong. "Aha!" you say. "Finally a community making a concerted effort to be rational!"

But to your dismay, you find that the ONLY thing they talk about is being rational, and a few other subjects that have been apparently grandfathered in. It's not that they have no interest in topic X, there's just no place on the site they're allowed to talk about it.

What I propose is a "non-meta" sister site, where people can talk and think about anything BESIDES talking and thinking. Well, you know what I mean.

Yes?
Karma Changes
by Eliezer Yudkowsky — 2009-12-22T00:17:14.517Z

As recently (re-)suggested by Kaj Sotala, posts now have much larger effects on karma than comments: Each up or down vote on a post is worth 10 karma.

Negative votes on posts have had karma effects all along, but for some reason Reddit's code imposed a display cap (not an actual cap) of 0. This violates a basic user interface principle: things with important effects should have visible effects. Since this just got 10x more important, we now show negative post totals rather than "0". This also provides some feedback to posters that was previously missing. Note that downvoting a post costs 10 karma from your downvote cap of 4x current karma.

The minimum karma to start posting has been raised to 50.

Thanks to our friends at Tricycle for implementing this request!
The Correct Contrarian Cluster
by Eliezer Yudkowsky — 2009-12-21T22:01:56.781Z

Followup to: Contrarian Status Catch-22

Suppose you know someone believes that the World Trade Center was rigged with explosives on 9/11. What else can you infer about them? Are they more or less likely than average to believe in homeopathy?

I couldn't cite an experiment to verify it, but it seems likely that:

There are persistent character traits which contribute to someone being willing to state a contrarian point of view.

All else being equal, if you know that someone advocates one contrarian view, you can infer that they are more likely than average to have other contrarian views.

All sorts of obvious disclaimers can be included here. Someone who expresses an extreme-left contrarian view is less likely to have an extreme-right contrarian view. Different character traits may contribute to expressing contrarian views that are counterintuitive vs. low-prestige vs. anti-establishment etcetera. Nonetheless, it seems likely that you could usefully distinguish a c-factor, a general contrarian factor, in people and beliefs, even though it would break down further on closer examination; there would be a cluster of contrarian people and a cluster of contrarian beliefs, whatever the clusters of the subcluster.

(If you perform a statistical analysis of contrarian ideas and you find that they form distinct subclusters of ideologies that don't correlate with each other, then I'm wrong and no c-factor exists.)

Now, suppose that someone advocates the many-worlds interpretation of quantum mechanics. What else can you infer about them?

Well, one possible reason for believing in the many-worlds interpretation is that, as a general rule of cognitive conduct, you investigated the issue and thought about it carefully; and you learned enough quantum mechanics and probability theory to understand why the no-worldeaters advocates call their theory the strictly simpler one; and you're reflective enough to understand how a deeper theory can undermine your brain's intuition of an apparently single world; and you listen to the physicists who mock many-worlds and correctly assess that these physicists are not to be trusted. Then you believe in many-worlds out of general causes that would operate in other cases - you probably have a high correct contrarian factor - and we can infer that you're more likely to be an atheist.

It's also possibl...
Mandating Information Disclosure vs. Banning Deceptive Contract Terms
by David_J_Balan — 2009-12-20T20:55:58.088Z

Economists are very into the idea of mutually beneficial exchange. The standard argument is that if two parties voluntarily agree to a deal, then they must be better off with the deal than without it, otherwise they wouldn't have agreed. And if the terms of that deal don't harm any third parties,* then the deal must be welfare-improving, and any regulatory restrictions on making it must be bad.
One objection to this argument is that it's not always clear what is and what is not "voluntary." I once has a well-published economist friend argue that there are no gradations of voluntariness: either a deal was made under some kind of compulsion or it wasn't. I asked him if he would be OK letting his then pre-adolescent son make any schoolyard deal he wanted as long as it was not made under any overt threat, and I think (but am not totally sure) that he has since backed off this position. So there is an argument for purely paternalistic restrictions on freedom of contract.
Another objection, one which economists tend to take more seriously, relates to information. Specifically, there is the idea that maybe one party to the contract is not fully informed about its terms. For this reason, many economists are willing to entertain policies by which firms are required to disclose certain information, and to do so in a way that is comprehensible to consumers. So for example we now have "Schumer boxes" that govern the ways in which credit card companies present certain information in promotional materials. This seems to many people to be a reasonable remedy: if the problem was that one side of the transaction was ignorant, then a regulation that eliminates that ignorance, while at the same time not interfering with their freedom to engage in mutually beneficial exchange, must be a good thing.

I think this reasonable-sounding position is largely wrong. The standard asymmetric information stories with rational agents are stories in which the uniformed party knows that it is uninformed, which influences the contract terms that it is willing to accept, which in turn either causes beneficial exchange not to happen (the "lemons" problem) or causes contract terms to be distorted away from the efficient ones. They are generally not stories about uninformed consumers not understanding that they are uninformed and blithely marching into traps as a result. But this is what we ac...
Sufficiently Advanced Sanity
by Eliezer Yudkowsky — 2009-12-20T18:11:17.677Z

Reply to: Shalmanese's Third Law

From an unpublished story confronting Vinge's Law, written in 2004, as abstracted a bit:

"If you met someone who was substantially saner than yourself, how would you know?"

"The obvious mistake that sounds like deep wisdom is claiming that sanity looks like insanity to the insane. I would expect to discover sanity that struck me as wonderfully and surprisingly sane, sanity that shocked me but that I could verify on deeper examination, sanity that sounded wrong but that I could not actually prove to be wrong, and sanity that seemed completely bizarre."

"Like a history of 20th-century science, presented to a scientist from 1900. Much of the future history would sound insane, and easy to argue against. It would take a careful mind to realize none of it was more inconsistent with present knowledge than the scientific history of the 19th century with the knowledge of 1800. Someone who wished to dismiss the whole affair as crackpot would find a thousand excuses ready to hand, plenty of statements that sounded obviously wrong. Yet no crackpot could possibly fake the parts that were obviously right. That is what it is like to meet someone saner. They are not infallible, are not future histories of anything. But no one could counterfeit the wonderfully and surprisingly sane parts; they would need to be that sane themselves."

Spot the Bayesian problem, anyone? It's obvious to me today, but not to the me of 2004. Eliezer2004 would have seen the structure of the Bayesian problem the moment I pointed it out to him, but he might not have assigned it the same importance I would without a lot of other background.
Fundamentally Flawed, or Fast and Frugal?
by Kaj_Sotala — 2009-12-20T15:10:15.714Z

Whenever biases are discussed around here, it tends to happen under the following framing: human cognition is a dirty, jury-rigged hack, only barely managing to approximate the laws of probability even in a rough manner. We have plenty of biases, many of them a result of adaptations that evolved to work well in the Pleistocene, but are hopelessly broken in a modern-day environment.

That's one interpretation. But there's also a different interpretation: that a perfect Bayesian reasoner is computationally intractable, and our mental algorithms make for an excellent, possibly close to an optimal, use of the limited computational resources we happen to have available. It's not that the programming would be bad, it's simply that you can't do much better without upgrading the hardware. In the interest of fairness, I will be presenting this view by summarizing a classic 1996 Psychological Review article, "Reasoning the Fast and Frugal Way: Models of Bounded Rationality" by Gerd Gigerenzer and Daniel G. Goldstein. It begins by discussing two contrasting views: the Enlightenment ideal of the human mind as the perfect reasoner, versus the heuristics and biases program that considers human cognition as a set of quick-and-dirty heuristics.

Many experiments have been conducted to test the validity of these two views, identifying a host of conditions under which the human mind appears more rational or irrational. But most of this work has dealt with simple situations, such as Bayesian inference with binary hypotheses, one single piece of binary data, and all the necessary information conveniently laid out for the participant (Gigerenzer & Hoffrage, 1995). In many real-world situations, however, there are multiple pieces of information, which are not independent, but redundant. Here, Bayes’ theorem and other “rational” algorithms quickly become mathematically complex and computationally intractable, at least for ordinary human minds. These situations make neither of the two views look promising. If one would apply the classical view to such complex real-world environments, this would suggest that the mind is a supercalculator like a Laplacean Demon (Wimsatt, 1976)— carrying around the collected works of Kolmogoroff, Fisher, or Neyman—and simply neds a memory jog, like the slave in Plato’s Meno. On the other hand, the heuristics-and-b

...
Any sufficiently advanced wisdom is indistinguishable from bullshit
by Shalmanese — 2009-12-20T10:09:02.908Z

In the grand tradition of sequences, I'm going to jot this down real quick because it's required for the next argument I'm going to make.

Shalmanese's 3rd law is "Any sufficiently advanced wisdom is indistinguishable from bullshit". Shalmanese's first law is "As the length of any discussion involving the metric system approaches infinity, the likelihood approaches 1 of there being a reference to The Simpsons episode about 40 rods to the hogshead" so judge it by the company it keeps.

Imagine you got to travel back in time to meet yourself from 10 years ago and impart as much wisdom as possible on your past-self in 6 hours. You're bound by the Time Enforcement Committee not to reveal that you are the future-self of your past-self and it never occurs to your past-self that this ugly thing in front of them could ever be you. As far as the past-self is concerned, it's just a moderately interesting person they're having a conversation with.

There would be 3 broad sets that your discussions would fall in: Beliefs that you both mutually agree on, Beliefs that you are able to convince your past-self through reason and Beliefs which make the past-self regard your future-self as being actively stupid for holding. It's this third category which I'm going to term Advanced Wisdom.

For everybody, those beliefs are going to be specific to the individual. Maybe you used to be devoutly religious and now you're staunchly atheist. Perhaps you were once radically marxist and now you're a staunch libertarian. For me, it was the wisdom of the advice to "be yourself". I have no doubt that I would get precisely nowhere convincing my past-self that "be yourself" is a piece of wisdom. Anything I could ever possibly say to him, he had already heard many times before and convinced himself was utter bullshit. If even my actual self couldn't convince myself of something, what hope is there that any rational argument could have penetrated?

If I were to have my future-self visit my present-self now, I would have no doubt that he would also present me with some pieces of advanced wisdom I thought were bullshit. The problem is, sufficiently advanced wisdom is indistinguishable from bullshit. There is no possible test that can separate the two. You might be told something is advanced wisdom and keep the openest mind possible about it and investigate it in all the various ways and perhaps even be co...
The Contrarian Status Catch-22
by Eliezer Yudkowsky — 2009-12-19T22:40:51.201Z

It used to puzzle me that Scott Aaronson still hasn't come to terms with the obvious absurdity of attempts to make quantum mechanics yield a single world.

I should have realized what was going on when I read Scott's blog post "The bullet-swallowers" in which Scott compares many-worlds to libertarianism. But light didn't dawn until my recent diavlog with Scott, where, at 50 minutes and 20 seconds, Scott says:

"What you've forced me to realize, Eliezer, and I thank you for this: What I'm uncomfortable with is not the many-worlds interpretation itself, it's the air of satisfaction that often comes with it."
-- Scott Aaronson, 50:20 in our Bloggingheads dialogue.

It doesn't show on my face (I need to learn to reveal my expressions more, people complain that I'm eerily motionless during these diavlogs) but at this point I'm thinking, Didn't Scott just outright concede the argument? (He didn't; I checked.) I mean, to me this sounds an awful lot like:

Sure, many-worlds is the simplest explanation that fits the facts, but I don't like the people who believe it.

And I strongly suspect that a lot of people out there who would refuse to identify themselves as "atheists" would say almost exactly the same thing:

What I'm uncomfortable with isn't the idea of a god-free physical universe, it's the air of satisfaction that atheists give off.

If you're a regular reader of Robin Hanson, you might essay a Hansonian explanation as follows:

Although the actual state of evidence favors many-worlds (atheism), I don't want to affiliate with other people who say so. They act all brash, arrogant, and offensive, and tend to believe and advocate other odd ideas like libertarianism. If I believed in many-worlds (atheism), that would make me part of this low-prestige group.

Or in simpler psychology:

I don't feel like I belong with the group that believes in many-worlds (atheism).

I think this might form a very general sort of status catch-22 for contrarian ideas.

When a correct contrarian idea comes along, it will have appealing qualities like simplicity and favorable evidence (in the case of factual beliefs) or high expected utility (in the case of policy proposals). When an appealing contrarian idea comes along, it will be initially supported by its appealing qualities, and opposed by the fact that it seems st...