trentmkelly/LessWrong-43k huggingface dataset:
Consider Representative Data Sets
by Vladimir_Nesov — 2009-05-06T01:49:21.389Z

In this article, I consider the standard biases in drawing factual conclusions that are not related to emotional reactions, and describe a simple model summarizing what goes wrong with the reasoning in these cases, that in turn suggests a way of systematically avoiding this kind of problems.

The following model is used to describe the process of getting from a question to a (potentially biased) answer for the purposes of this article. First, you ask yourself a question. Second, in the context of the question, a data set is presented before your mind, either directly, by you looking at the explicit statements of fact, or indirectly, by associated facts becoming salient to your attention, triggered by the explicit data items or by the question. Third, you construct an intuitive model of some phenomenon, that allows to see its properties, as a result of considering the data set. And finally, you pronounce the answer, that is read out as one of the properties of the model you've just constructed.

This description is meant to present mental paintbrush handles, to refer to the things you can see introspectively, and things you could operate consciously if you choose to.

Most of the biases in the considered class may be seen as particular ways in which you pay attention to a wrong data set, not representative of the phenomenon you model to get to the answer you seek. As a result, the intuitive model gets systematically wrong, and the answer read out from it gets biased. Below I review the specific biases, to identify the ways in which things go wrong in each particular case, and then I summarize the classes of mistakes of reasoning playing major roles in these biases and correspondingly the ways of avoiding those mistakes.

Correspondence Bias is a tendency to attribute to a person a disposition to behave in a particular way, based on observing an episode in which that person behaves in that way. The data set that gets considered consists only of the observed episode, while the target model is of the person's behavior in general, in many possible episodes, in many different possible contexts that may influence the person's behavior.

Hindsight bias is a tendency to overestimate the a priori probability of an event that has actually happened. The data set that gets considered overemphasizes the scenario that did happen, while the model that needs to be constructed, of the ...
How to use "philosophical majoritarianism"
by jimmy — 2009-05-05T06:49:45.419Z

The majority of people would hold more accurate beliefs if they simply believed the majority. To state this in a way that doesn't risk information cascades, we're talking about averaging impressions and coming up with the same belief.

To the degree that you come up with different averages of the impressions, you acknowledge that your belief was just your impression of the average, and you average those metaimpressions and get closer to belief convergence. You can repeat this until you get bored, but if you're doing it right, your beliefs should get closer and closer to agreement, and you shouldn't be able to predict who is going to fall on which side.
Of course, most of us are atypical cases, and as good rationalists, we need to update on this information. Even if our impressions were (on average) no better than the average, there are certain cases where we know that the majority is wrong. If we're going to selectively apply majoritarianism, we need to figure out the rules for when to apply it, to whom, and how the weighting works.
This much I think has been said again and again. I'm gonna attempt to describe how.

Imagine for a moment that you are a perfectly rational Bayesian, and you just need data.
First realize that "duplicate people" don't count double. If you make a maximum precision copy of someone, that doesn't make him any more likely to be right- clearly we can do better than averaging over all people with equal weighting. By the same idea, finding out that a certain train of thought leading to a certain belief is common shouldn't make you proportionally more confident in that idea. The only reason it might make you any more confident in it is the possibility that its truth leads to its proliferation and therefore its popularity is (weak) evidence.

This explains why we can dismiss the beliefs of the billions of theists. First of all, their beliefs are very well correlated so that all useful information can be learned through only a handful of theists. Second of all, we understand their arguments and we understand how they formed their beliefs-and have already taken them into account. The reason they continue to disagree is because the situation isn't symmetric - they don't understand the opposing arguments or the causal path that leads one to be a reductionist atheist.

No wonder "majoritarionism" doesn't seem to work here.

Since we'...
How David Beats Goliath
by JulianMorrison — 2009-05-05T01:25:03.586Z

From the New Yorker:

It was as if there were a kind of conspiracy in the basketball world about the way the game ought to be played, and Ranadivé thought that that conspiracy had the effect of widening the gap between good teams and weak teams. Good teams, after all, had players who were tall and could dribble and shoot well; they could crisply execute their carefully prepared plays in their opponent’s end. Why, then, did weak teams play in a way that made it easy for good teams to do the very things that made them so good?

[...]

David’s victory over Goliath, in the Biblical account, is held to be an anomaly. It was not. Davids win all the time. The political scientist Ivan Arreguín-Toft recently looked at every war fought in the past two hundred years between strong and weak combatants. The Goliaths, he found, won in 71.5 per cent of the cases. That is a remarkable fact. Arreguín-Toft was analyzing conflicts in which one side was at least ten times as powerful—in terms of armed might and population—as its opponent, and even in those lopsided contests the underdog won almost a third of the time.
[...] What happened, Arreguín-Toft wondered, when the underdogs likewise acknowledged their weakness and chose an unconventional strategy? He went back and re-analyzed his data. In those cases, David’s winning percentage went from 28.5 to 63.6.

[...]

Arreguín-Toft found the same puzzling pattern. When an underdog fought like David, he usually won. But most of the time underdogs didn’t fight like David. Of the two hundred and two lopsided conflicts in Arreguín-Toft’s database, the underdog chose to go toe to toe with Goliath the conventional way a hundred and fifty-two times—and lost a hundred and nineteen times.
Special Status Needs Special Support
by Eliezer Yudkowsky — 2009-05-04T22:59:59.183Z

I just recorded another BHTV with Adam Frank, though it's not out yet, and I had a thought that seems worth recording. At a certain point in the dialogue, Adam Frank was praising the wisdom and poetry in religion. I retorted, "Tolkien's got great poetry, and some parts that are wise and some that are unwise; but you don't see people wearing little rings around their neck in memory of Frodo."

(I don't remember whether this observation is original to me, so if anyone knows a prior source for this exact wording, please comment it!)

The general structure of this critique is that Frank wants to assign a special status to the Book of Job, but he gives a reason that would be equally applicable to The Lord of the Rings (good poetry and some wise parts). So if those are his real reasons, he should feel just the same way about God and Gandalf. Or if not that exact particular book, then some other work of poetic fiction that was always understood to be poetic fiction.

Later on I did demand of Adam Frank to say whether he thought the Book of Job ought to be assigned any different status from The Merchant of Venice, and Frank did reply "No". I'm not sure that he lives up to this reply, frankly. I strongly suspect he grants the two works a different emotional status. One is widely revered as Sacred Religious Truth while the other is merely a Great Work of Literature. Frank, while not a religious believer himself, does have different modes of thought for Sacred Truth and Great Literature and he knows that Job is supposed to be Sacred Truth.

When I challenged the sacredness of the Book of Job, Frank reacted by trying to praise Job's "great poetry", which positive affect then seems to justify the positive-affect sacred status via the affect heuristic / halo effect. But "great poetry" would apply to Tolkien as well; and yet if you talked about Tolkien the way that Frank talked about Job, most people would write you down as a hopeless fanboy/fangirl...

So the general form of the bias that I'm critiquing is to try and justify a special positive (negative) status by pointing to positive (negative) attributes, saying, "Therefore I can assign it this very positive status!", but the same attributes belong to many other works that you don't grant the special positive status.

Other places to watch out for this would be if, say, you thou...
Bead Jar Guesses
by Alicorn — 2009-05-04T18:59:56.768Z

Let's say Omega turns up and sets you a puzzle, since this seems to be what Omega does in his spare time. He has with him an opaque jar, which he says contains some solid-colored beads, and he's going to draw one bead out of the jar. He would like to know what your probability is that the bead will be red.

Well, now there is an interesting question. We'll bypass the novice mistake of calling it .5, of course; just because the options are binary (red or non-red) doesn't make them equally likely. It's not like you have any information. Assuming you don't think Omega is out to deliberately screw with you, you could say that the probability is .083 based on the fact that "red" is one of twelve basic color words in English. (If he had asked for the probability that the bead would be lilac, you'd be in a bit more trouble.) If you were obliged to make a bet that the bead is red, you would probably take the most conservative bet available (even if you're still assuming Omega isn't deliberately screwing with you), but .083 sounds okay.

But because you start with no information, it's very hard to gather more. Suppose Omega reaches into the jar and pulls out a red bead. Does your probability that the second bead will be red go up (obviously the beads come in red)? Does it go down (that might have been the only one, and however many red beads there were before, there are fewer now)? Does it stay the same (the beads are all - as far as you know - independent of one another; removing this one bead has an effect on the actual probabilities of what the next one will be, but it can't affect your epistemic probability)? What if he pulled out a gray bead first, instead of a red one? How many beads would he have to pull, and in what colors, for you to start making confident predictions?

So that's one kind of probability: the bead jar guess. It has a basis, but it's a terribly flimsy one, and guessing right (or wrong) doesn't help much to confirm or disconfirm the guess. Even if Omega had asked about the bead being lilac, and you'd dutifully given a tiny probability, it would not have surprised you to see a lilac bead emerge from the jar.

A non-bead-jar-guess probability yields surprise when it turns out to be true even if it's just the same size. Say your probability for lilac was .003.&...
Without models
by Richard_Kennaway — 2009-05-04T11:31:38.399Z

Followup to: What is control theory?

I mentioned in my post testing the water on this subject that control systems are not intuitive until one has learnt to understand them. The point I am going to talk about is one of those non-intuitive features of the subject. It is (a) basic to the very idea of a control system, and (b) something that almost everyone gets wrong when they first encounter control systems.

I'm going to address just this one point, not in order to ignore the rest, but because the discussion arising from my last post has shown that this is presently the most important thing.

There is a great temptation to think that to control a variable -- that is, to keep it at a desired value in spite of disturbing influences -- the controller must contain a model of the process to be controlled and use it to calculate what actions will have the desired effect. In addition, it must measure the disturbances or better still, predict them in advance and what effect they will have, and take those into account in deciding its actions.

In terms more familiar here, the temptation to think that to bring about desired effects in the world, one must have a model of the relevant parts of the world and predict what actions will produce the desired results.

However, this is absolutely wrong. This is not a minor mistake or a small misunderstanding; it is the pons asinorum of the subject.

Note the word "must". It is not disputed that one can use models and predictions, only that one must, that the task inherently requires it.

A control system can work without having any model of what it is controlling.

The designer will have a model. For the room thermostat, he must know that the heating should turn on when the room is too cold and off when it is too hot, rather than the other way around, and he must arrange that the source of heat is powerful enough. The controller he designs does not know that; it merely does that. (Compare the similar relationship between evolution and evolved organisms. How evolution works is not how the evolved organism works, nor is how a designer works how the designed system works.) For a cruise control, he must choose the parameters of the controller, taking into account the engine's response to the accelerator pedal. The resulting control system, however, contains no representation of that. According to the HowStuffWorks article, they typically use not...
Allais Hack -- Transform Your Decisions!
by MBlume — 2009-05-03T22:37:13.238Z

The Allais Paradox, though not actually a paradox, was a classic experiment which showed that decisions made by humans do not demonstrate consistent preferences. If you actually want to accomplish something, rather than simply feel good about your decisions, this is rather disturbing.

When something like the Allais Paradox is presented all in one go, it's fairly easy to see that the two cases are equivalent, and ensure that your decisions are consistent. But if I clone you right now, present one of you with gamble 1, and one of you with gamble 2, you might not fare so well. The question is how to consistently advance your own preferences even when you're only looking at one side of the problem.

Obviously, one solution is to actually construct a utility function in money, and apply it rigorously to all decisions. Logarithmic in your total net worth is usually a good place to start. Next you can assign a number of utilons to each year you live, a negative number to each day you are sick, a number for each sunrise you witness...

I would humbly suggest that a less drastic strategy might be to familiarize yourself with the ways in which you can transform a decision which should make no difference unto decision theory, and actually get in the habit of applying these transformations to decisions you make in real life.

So, let us say that I present you with Allais Gamble #2: choose between A: 34% chance of winning $24,000, and 66% chance of winning nothing, and B: 33% chance of winning $27,000, and 67% chance of winning nothing.

Before snapping to a judgment, try some of the following transforms:

Assume your decision matters:

The gamble, as given, contains lots of probability mass in which your decision will not matter one way or the other -- shave it off!

Two possible resulting scenarios:

A: $24,000 with certainty, B: 33/34 chance of $27,000

Or, less obviously: I spin a wheel with 67 notches, 34 marked A and 33 marked B. Choose A and win $24,000 if the wheel comes up A, nothing otherwise. Choose B and win $27,000 if the wheel comes up B, nothing otherwise.

Assume your decision probably doesn't matter:

Tiny movements away from certainty tend to be more strongly felt -- try shifting all your probabilities down and see how you feel about them.

A: 3.4% chance of winning $24,000, 96.6% chance of nothing. B: 3.3% chance of winning $27,000, 96.7% chance of nothing.

Convert potential wi...
Essay-Question Poll: Dietary Choices
by Alicorn — 2009-05-03T15:27:26.437Z

I have noticed that among philosophers, vegetarianism of one form or another is quite common. In fact, I became a vegetarian (technically a pescetarian) myself partly out of respect for an undergraduate philosophy professor. I am interested in finding out if there is a similar disproportion in the Less Wrong community.

I didn't request that this go into Yvain's survey because I want more information than just what animal products you do or don't eat; I'd also like to see nuances of the reasons behind your diet. There are a lot more shades than carnivore/vegetarian/vegan - if you want to be a vegetarian but are allergic to soy and gluten, that's a compelling reason to diversify protein sources, for instance. I'd also like to hear about if you avoid any plant foods (if you think they're farmed in a way that's environmentally destructive or that hurts people or if you have warm fuzzy feelings for plants, maybe). Here are some questions that come to mind:

What foods, if any, do you normally avoid for reasons other than pure culinary taste, cost, individual health concerns (allergies, diabetes, etc.) or ease of preparation? (Avoiding foods that are considered revolting or just non-food in your culture of origin, like balut or fried locusts, counts as "culinary taste".)

What are your reasons for avoiding those foods?

How strictly do you avoid them? For instance, will you eat them if you are served them while a guest at a meal, or if you are hungry and there is nothing else available? Do you check to see if they're in potentially questionable dishes at restaurants (and if so, do you trust what the server says?)

If you have children or plan to have children, will you expect or encourage them to avoid the same foods?

Do you try to convince your friends and family members to make dietary choices similar to yours? If so, have you ever succeeded?

If you avoid a class of foods with valuable nutritive content (as opposed to Twinkies), what do you replace it with to get complete nutrition?

What are your attitudes to people who are more restrictive in their diets than you are? Less restrictive?

What is the timeline of your dietary restrictions? (Transitions, lapses, increases or decreases in restrictiveness, etc.)

If you have not avoided these foods for your entire life, how much did you enjoy them when you a

...
Return of the Survey
by Scott Alexander — 2009-05-03T02:10:43.448Z

[UPDATE: Survey is now closed. Thanks to everyone who took it. Results soon. Ignore everything below.]

Last week, I asked people for help writing a survey. I've since taken some of your suggestions. Not all, because I wanted to keep the survey short, and because the survey software I'm using made certain types of questions inconvenient, but some. I hope no one's too angry about their contributions being left out.

Please note that, due to what was very possibly a bad decision on my part as to what would be most intuitive, I've requested all probabilities be in percentage format. So if you think something has a 1/2 chance of being true, please list 50 instead of .5.

Please take the survey now; it can be found here and it shouldn't take more than fifteen or twenty minutes. Unless perhaps you need to spend a lot of time determining your opinions on controversial issues, in which case it will be time well spent!

Several people, despite the BOLD ALL CAPS TEXT saying not to take the survey in the last post, went ahead and took the survey. Your results have been deleted. Please take it again. Thank you.

I'll leave this open for about a week, calculate some results, then send out the data. There is an option to make your data private at the bottom of the survey.

Thanks to everyone who takes this. If you want, post a comment saying you took it below, and I'll give you a karma point :)
What I Tell You Three Times Is True
by Scott Alexander — 2009-05-02T23:47:32.013Z

"The human brain evidently operates on some variation of the famous principle enunciated in 'The Hunting of the Snark': 'What I tell you three times is true.'"

-- Norbert Weiner, from Cybernetics

Ask for a high-profile rationalist, and you'll hear about Richard Dawkins or James Randi or maybe Peter Thiel. Not a lot of people would immediately name Scott Adams, creator of Dilbert. But as readers of his blog know, he's got a deep interest in rationality, and sometimes it shows up in his comics: for example, this one from last week. How many people can expose several million people to the phrase "Boltzmann brain hypothesis" and have them enjoy it?
So I was very surprised to find Adams was a believer in and evangelist of something that sounded a lot like pseudoscience. "Affirmations" are positive statements made with the belief that saying the statement loud enough and long enough will help it come true. For example, you might say "I will become a syndicated cartoonist" fifteen times before bed every night, thinking that this will in fact make you a syndicated cartoonist. Adams partially credits his success as a cartoonist to doing exactly this.
He admits "it sounds as if I believe in some sort of voodoo or magic", and acknowledges that "skeptics have suggested, and reasonably so, that this is a classic case of selective memory" but still swears that it works. He also has "received thousands of e-mails from people recounting their own experiences with affirmations. Most people seem to be amazed at how well they worked."
None of this should be taken too seriously without a controlled scientific study investigating it, of course. But is it worth the effort of a study, or should it be filed under "so stupid that it's not worth anyone's time to investigate further"?
I think there's a good case to be made from within a rationalist/scientific worldview that affirmations may in fact be effective for certain goals. Not miraculously effective, but not totally useless either.

To build this case, I want to provide evidence for two propositions. First, that whether we subconsciously believe we can succeed affects whether or not we succeed. Second, that repeating a statement verbally can make the subconscious believe it.
The link between belief in success and success has progressed beyond the motivational speaker stage and into the scientific evidence stage. The best-kn...