trentmkelly/LessWrong-43k huggingface dataset:
Real-Life Anthropic Weirdness
by Eliezer Yudkowsky — 2009-04-05T22:26:38.377Z

In passing, I said:

From a statistical standpoint, lottery winners don't exist - you would never encounter one in your lifetime, if it weren't for the selective reporting.

And lo, CronoDAS said:

Well... one of my grandmothers' neighbors, whose son I played with as a child, did indeed win the lottery. (AFAIK, it was a relatively modest jackpot, but he did win!)

To which I replied:

Well, yes, some of the modest jackpots are statistically almost possible, in the sense that on a large enough web forum, someone else's grandmother's neighbor will have won it. Just not your own grandmother's neighbor.

Sorry about your statistical anomalatude, CronoDAS - it had to happen to someone, just not me.

There's a certain resemblance here - though not an actual analogy - to the strange position your friend ends up in, after you test the Quantum Theory of Immortality.

For those unfamiliar with QTI, it's a simple simultaneous test of many-worlds plus a particular interpretation of anthropic observer-selection effects: You put a gun to your head and wire up the trigger to a quantum coinflipper. After flipping a million coins, if the gun still hasn't gone off, you can be pretty sure of the simultaneous truth of MWI+QTI.

But what is your watching friend supposed to think? Though his predicament is perfectly predictable to you - that is, you expected before starting the experiment to see his confusion - from his perspective it is just a pure 100% unexplained miracle. What you have reason to believe and what he has reason to believe would now seem separated by an uncrossable gap, which no amount of explanation can bridge. This is the main plausible exception I know to Aumann's Agreement Theorem.

Pity those poor folk who actually win the lottery! If the hypothesis "this world is a holodeck" is normatively assigned a calibrated confidence well above 10-8, the lottery winner now has incommunicable good reason to believe they are in a holodeck. (I.e. to believe that the universe is such that most conscious observers observe ridiculously improbable positive events.)

It's a sad situation to be in - but don't worry: it will always happen to someone else, not you.
Supporting the underdog is explained by Hanson’s Near/Far distinction
by Roko — 2009-04-05T20:22:02.593Z

Yvain can’t make head nor tails of the apparently near universal human tendency to root for the underdog. [Read Yvain’s post before going any further]..

He uses the following plausible-sounding story from a small hunter-gatherer tribe in our Era of Evolutionary Adaptedness to illustrate why support for the underdog seems to be an antiprediction of the standard theory of human evolutionary psychology:

Suppose Zug and Urk are battling it out for supremacy in the tribe. Urk comes up to you and says “my faction are hopelessly outnumbered and will probably be killed, and our property divided up amongst Zug’s supporters.” Those cave-men with genes that made them support the underdog would join Urk’s faction and be wiped out. Their genes would not make it very far in evolution’s ruthless race, unless we can think of some even stronger effect that might compensate for this.

Yvain cites an experiment where people supported either Israel or Palestine depending on who they saw as the underdog. This seems to contradict the claim that the human mind is well adapted to its EEA.

A lot of people tried to use the “truel” situation as an explanation: in a game of three players, it is rational for the weaker two to team up against the stronger one. But the choice of which faction to join is not a truel between three approximately equal players: as an individual you will have almost no impact upon which faction wins, and if you join the winning side you won’t necessarily be next on the menu: you will have about as much chance as anyone else in Zug’s faction of doing well if there is another mini-war. People who proffered this explanation are guilty of not being more surprised by fiction than reality. To start with, if this theory were correct, we would expect to see soldiers defecting away from the winning side in the closing stages of a war... which, to my knowledge, is the opposite of what happens.

SoulessAutomaton comes closest to the truth when he makes the following statement:

there may be a critical difference between voicing sympathy for the losing faction and actually joining it and sharing its misfortune.

Yes! Draw Distinctions!

I thought about what the answer to Yvain’s puzzle was before reading the comments – and decided that Robin’s Near/Far distinction is the answer.

All

...
Formalizing Newcomb's
by cousin_it — 2009-04-05T15:39:03.228Z

This post was inspired by taw urging us to mathematize Newcomb's problem and Eliezer telling me to post stuff I like instead of complaining.

To make Newcomb's problem more concrete we need a workable model of Omega. Let me count the ways:

1) Omega reads your decision from the future using a time loop. In this case the contents of the boxes are directly causally determined by your actions via the loop, and it's logical to one-box.

2) Omega simulates your decision algorithm. In this case the decision algorithm has indexical uncertainty on whether it's being run inside Omega or in the real world, and it's logical to one-box thus making Omega give the "real you" the million.

3) Omega "scans your brain and predicts your decision" without simulating you: calculates the FFT of your brainwaves or whatever. In this case you can intend to build an identical scanner, use it on yourself to determine what Omega predicted, and then do what you please. Hilarity ensues.

(NB: if Omega prohibits agents from using mechanical aids for self-introspection, this is in effect a restriction on how rational you're allowed to be. If so, all bets are off - this wasn't the deal.)

(Another NB: this case is distinct from 2 because it requires Omega, and thus your own scanner too, to terminate without simulating everything. A simulator Omega would go into infinite recursion if treated like this.)

4) Same as 3, but the universe only has room for one Omega, e.g. the God Almighty. Then ipso facto it cannot ever be modelled mathematically, and let's talk no more.

I guess this one is settled, folks. Any questions?
Voting etiquette
by gjm — 2009-04-05T14:28:31.031Z

Not all that surprisingly, there's quite a lot of discussion on LW about questions like

just what should get voted up or down?

what conclusions can one reasonably draw from getting downvoted?

should downvotes (or even upvotes) be accompanied by explanations?

should the way karma and voting work be changed?

This generally happens in dribs and drabs, typically in response to more specific questions of the form

Waaaa, how come my supremely insightful comment above is currently sitting at -69?

and therefore tends to clutter up discussions that are meant to be about something else. So maybe it's worth seeing if we can arrive at some sort of consensus about the general issues, at which point maybe we can write that up and refer newcomers to it.

(The outcome may be that we find that there's no consensus to be had. That would be useful information too.)

I'll kick things off with a few unfocused thoughts.

What voting is for: establishing the nearest thing we have to the consensus view of the LW community, so as to (1) help readers guess what might be most worth reading and (2) help writers adjust their writing (if they wish) to please the audience more. Note that these purposes are somewhat separate from ...

What karma is for: motivating people to participate, motivating people to participate well, giving readers an indication of which writers are most worth reading.

It seems to me that voting is working reasonably well -- I find a reasonable correlation between comment ratings and comment quality. I'm not convinced that karma is working so well; what's rewarded by the system is prolific posting at least as much as high-quality posting. Doing away with the auto-self-upvote (and making it impossible to upvote one's own comments) seems likely to be an improvement. Or maybe making each comment count for (say) 1/4 as much as an upvote.

Explanations for votes: Lots of comments get voted up; quite a lot get voted down. The practice of explaining votes (even just downvotes) would make for cluttered threads. Also: upvotes and downvotes are anonymous, which is largely a good thing. So, here's one possibility. (It might just be unnecessary complication). When you vote something up or down, you get the chance (or the obligation?) to write a brief explanation of why; it doesn't go into the thread as a comment, but gets associated with the comment you voted on (without your name at...
Off-Topic Discussion Thread: April 2009
by MBlume — 2009-04-05T03:23:33.076Z

Dale McGowan writes:

And it needs to go well beyond one greeter. EVERY MEMBER of EVERY GROUP should make it a point to chat up new folks—and each other, for that matter. And not just about the latest debunky book. Ask where he’s from, what she does for a living, whether he follows the Mets or the Yankees. You know, mammal talk.

In this spirit, I propose the creation of a fully off-topic discussion thread.

Here is our monthly place to discuss topics entirely unrelated to Less Wrong that (of course) have not appeared in recent posts.

ETA: There are two behaviors I would love to see associated with this thread. First of all, discussions often drift off-topic in the middle of a thread. In these cases "let's take this to the off-topic thread" would be an excellent response. Secondly, given who's doing the discussing, I could easily see, say, a discussion about recent developments in some webcomic blossoming into a LW-worthy insight, in which case someone could spawn a new thread.
Why Support the Underdog?
by Scott Alexander — 2009-04-05T00:01:29.756Z

One of the strangest human biases is the almost universal tendency to support the underdog.
I say "human" because even though Americans like to identify themselves as particular friends of the underdog, you can find a little of it everywhere. Anyone who's watched anime knows the Japanese have it. Anyone who's read the Bible knows the Israelites had it (no one was rooting for Goliath!) From mythology to literature to politics to sports, it keeps coming up.
I say "universal" because it doesn't just affect silly things like sports teams. Some psychologists did a study where they showed participants two maps of Israel: one showing it as a large country surrounding the small Palestinian enclaves, and the other showing it as a tiny island in the middle of the hostile Arab world. In the "Palestinians as underdogs" condition, 55% said they supported Palestine. In the "Israelis as underdogs" condition, 75% said they supported Israel. Yes, you can change opinion thirty points by altering perceived underdog status. By comparison, my informal experiments trying to teach people relevant facts about the region's history changed opinion approximately zero percent.

(Oh, and the Israelis and Palestinians know this. That's why the propaganda handbooks they give to their respective supporters - of course they give their supporters propaganda handbooks! - specifically suggest the supporters portray their chosen cause as an underdog. It's also why every time BBC or someone shows a clip about the region, they get complaints from people who thought it didn't make their chosen side seem weak enough!)

And there aren't many mitigating factors. Even when the underdog is obviously completely doomed, we still identify with them: witness Leonidas at Thermopylae. Even when the underdog is evil and the powerful faction is good, we can still feel a little sympathy for them; I remember some of my friends and I talking about bin Laden, and admitting that although he was clearly an evil terrorist scumbag, there was still something sort of awesome about a guy who could take on the entire western world from a cave somewhere.
I say "strangest" because I can't make heads or tails of why evolutionary psychology would allow it. Let's say Zug and Urk are battling it out for supremacy of your hunter-gatherer tribe. Urk comes to you and says "Hey, my faction is really weak. We don't have a chance against Zu...
The First London Rationalist Meetup
by taw — 2009-04-04T17:49:19.283Z

Here's a brief summarly of the first meetup. It took place on cafe on top of Waterstone bookstore on Saturday 2009-04-04, starting at 14:00 and lasting until about 17:15. Six people showed up: Tomasz (me), Michael, Will, Julian, Shane, and Marc.

We started with a game of estimation warewolf - there was 1 decider, and 3 participants, one of the dishonest. 2 people showed up later, so they joined the game as openly honest. I don't think the decider position was really necessary, we could simply randomly assign people to honest and dishonest set. Or if the dishonest guy was necessary, we were confused enough without any active help. The subject was "maize production of Mexico". We did our estimate twice. First based on what Mexicans are likely to eat. Our estimates were:

100 million Mexicans

3000 kcal / day / Mexican

30% of their calories from maize

extra 50% on top of the result for animal feed, biofuels and other uses

giving 44.3 mln tons of maize per year

Most of the time we discussed it first, and then took the median of our guesses as the estimate. The discussion was really the fun bit, I'll give a few examples later. There was a lot of joking about anchoring effect, but I don't think there was that much of it.

Then we hid the first estimate, and tried the same question again, in a different way, estimating:

Land area of Mexico was 3,125,000 square km. This estimate took surprisingly long time, with paths like "USA has 4 time zones, there are 15 degrees per time zone, north Mexican border is about half of south USA border, Mexico is shaped more or less like a triangle" etc., drawing pictures from memory and taking medians of them, guessing how many times different countries are bigger than each other, and guessing from territorial changes after Mexican-American wars. I guess everybody enjoyed it so much, because we all had so many bits of information vaguely related to what we were trying to guess, but no direct estimate.

Land used for growing plants is 1/2 of all land.

Land for growith maize is 1/3 of that.

10 plants per square meter.

500g per plant.

giving 2600 mln tons of maize per year

That was almost two orders of magnitude off. I was so sure that the second estimate failed any sanity check that I took a 10000:1 bet against it (conditional on our arithmetics being correct), taking 1p against 100 pound donation to the Singularity Institute. At this p...
Incremental Progress and the Valley
by Eliezer Yudkowsky — 2009-04-04T16:42:38.405Z

Yesterday I said: "Rationality is systematized winning"

"But," you protest, "the reasonable person doesn't always win!"

What do you mean by this? Do you mean that every week or two, someone who bought a lottery ticket with negative expected value, wins the lottery and becomes much richer than you? That is not a systematic loss; it is selective reporting by the media. From a statistical standpoint, lottery winners don't exist—you would never encounter one in your lifetime, if it weren't for the selective reporting.

Even perfectly rational agents can lose. They just can't know in advance that they'll lose. They can't expect to underperform any other performable strategy, or they would simply perform it.

"No," you say, "I'm talking about how startup founders strike it rich by believing in themselves and their ideas more strongly than any reasonable person would. I'm talking about how religious people are happier—"

Ah. Well, here's the the thing: An incremental step in the direction of rationality, if the result is still irrational in other ways, does not have to yield incrementally more winning.

The optimality theorems that we have for probability theory and decision theory, are for perfect probability theory and decision theory. There is no companion theorem which says that, starting from some flawed initial form, every incremental modification of the algorithm that takes the structure closer to the ideal, must yield an incremental improvement in performance. This has not yet been proven, because it is not, in fact, true.

"So," you say, "what point is there then in striving to be more rational? We won't reach the perfect ideal. So we have no guarantee that our steps forward are helping."

You have no guarantee that a step backward will help you win, either. Guarantees don't exist in the world of flesh; but contrary to popular misconceptions, judgment under uncertainty is what rationality is all about.

"But we have several cases where, based on either vaguely plausible-sounding reasoning, or survey data, it looks like an incremental step forward in rationality is going to make us worse off. If it's really all about winning—if you have something to protect more important than any ritual of cognition—then why take that step?"

Ah, and now we come to the meat of it.

I can't necessarily an...
On dollars, utility, and crack cocaine
by PhilGoetz — 2009-04-04T00:00:24.951Z

The lottery came up in a recent comment, with the claim that the expected return is negative - and the implicit conclusion that it's irrational to play the lottery. So I will explain why this is not the case.

It's convenient to reason using units of equivalent value. Dollars, for instance. A utility function u(U) maps some bag of goods U (which might be dollars) into a value or ranking. In general, u(kn) / u(n) < k. This is because a utility function is (typically) defined in terms of marginal utility. The marginal utility to you of your first dollar is much greater than the marginal utility to you of your 1,000,000th dollar. It increases the possible actions available to you much more than your 1,000,000th dollar does.

Utility functions are sigmoidal. A serviceable utility function over one dimension might be u(U) = k * ([1 / (1 + e-U)] - .5). It's steep around U=0, and shallow for U >> 0 and U << 0.

Sounds like I'm making a dry, academic mathematical point, doesn't it? But it's not academic. It's crucial. Because neglecting this point leads us to make elementary errors such as asserting that it isn't rational to play the lottery or become addicted to crack cocaine.

For someone with $ << 0, the marginal utility of $5 to them is minimal. They're probably never going to get out of debt; someone has a lien on their income and it's going to be taken from them anyway; and if they're $5 richer it might mean they'll lose $4 in government benefits. It can be perfectly reasonable, in terms of expected utility, for them to play the lottery.

Not in terms of expected dollars. Dollars are the input to the utility function.

Rationally, you might expect that u(U) = 0 for all U < 0. Because you can always kill yourself. Once your life is so bad that you'd like to kill yourself, it could make perfect sense to play the lottery, if you thought that winning it would help. Or to take crack cocaine, if it gives you a few short intervals over the next year that are worth living.

Why is this important?

Because we look at poor folks playing the lottery, and taking crack cocaine, and we laugh at them and say, Those fools don't deserve our help if they're going to make such stupid decisions.

When in reality, some of them may be making <EDITED> much more rat...
First London Rationalist Meeting upcoming
by taw — 2009-04-03T22:28:05.105Z

It's an extremely short notice, but we're going to have the first meeting tomorrow, that is - Saturday (2009-04-04) 14:00, in cafe on top of the Waterstones bookstore near the Piccadilly Circuis Tube station.

If you want to know more, email me (Tomasz.Wegrzanowski@gmail.com) for details. Or just come straight away.

Hopefully we can get it going, and the second meeting with be better organized.
Another Call to End Aid to Africa
by Eliezer Yudkowsky — 2009-04-03T18:55:52.000Z

Dambisa Moyo, an African economist, has joined her voice to the other African economists [e.g. James Shikwati] calling for a full halt to Western aid. Her book is called Dead Aid and it asserts a direct cause-and-effect relationship between $1 trillion of aid and the rise in African poverty rates from 11% to 66%.

Though it's an easy enough signal to fake, I find it noteworthy that Moyo - in this interview at least - repeatedly pleads for some attention to "logic and evidence":

"I think the whole aid model is couched in pity. I don’t want to cast
aspersions as to where that pity comes from. But I do think it’s based
on pity because based on logic and evidence, it is very clear that aid
does not work. And yet if you speak to some of the biggest supporters
of aid, whether they are academics or policy makers or celebrities,
their whole rationale for giving more aid to Africa is not couched in
logic or evidence; it’s based largely on emotion and pity."

I was just trying to think of when was the last time I heard a Western politician - or even a mainstream Western economist in any public venue - draw an outright battle line between logic and pity. Oh, there are plenty of demagogues who claim the evidence is on their side, but they won't be so outright condemning of emotion - it's not a winning tactic. Even I avoid drawing a battle line so stark.

Moyo says she's gotten a better reception in Africa than in the West. Maybe you need to see your whole continent wrecked by emotion and pity before "logic and evidence" start to sound appealing.
Winning is Hard
by  — 2009-04-03T17:02:31.856Z

Let us say you are playing Steve Omohundro's meal choosing game 1, however the negatives are a bit harsher and more realistic than just a dodgy soufle. You are given two choices on the menu, oysters and fugu. Your goal avoid death, sickness and eat tasty food. You don't know much about either, although you do know that shellfish has made you ill in the past so you give it a lower expected utility (pretend you don't know what fugu is).
Eating the poorly prepared fugu kills you dead every time, do not pass go, do not update your utility values of choosing an option (although the utility of it would be 0, if you were allowed to update). Eating oysters gives you a utility of 1.
So how do we win in this situation? In a way it is easy: Don't eat the fugu! But by what principled fashion should you choose not to eat the fugu? Microeconomics is not enough, with negative expected utility from shellfish you would pick the fugu! Also you do not get to update your utilities when you eat the fugu, so your expected utilities can't converge with experience. So we are in a bit of a pickle.
Can humans solve these kinds of problems, if so how do we do it? The answer is poorly, in a patch work fashion and we get information on the fugu type problems from our genome and culture. For example we avoid bitter things, are scared of snakes and are careful if we are up high are because our ancestors had to have had thes bits of information (and more) to avoid death. They got them by chance, which isn't exactly principled. But all these are still needed for winning. We can also get the information culturally, but that can leave us open to taboos against harmless things such as eating pork, which we might be foolish to test ourselves. It is hardly principled either.
So in this kind of scenario it is not sufficient to be economically rational to win, you have to have a decent source of knowledge. Getting a decent source of knowledge is hard.
1 See the appendix of the Nature of Self-Improving Artificial Intelligence starting page 37