trentmkelly/LessWrong-43k huggingface dataset:
The mind-killer
by Paul Crowley — 2009-05-02T16:49:19.539Z

Can we talk about changing the world? Or saving the world?

I think few here would give an estimate higher than 95% for the probability that humanity will survive the next 100 years; plenty might put a figure less than 50% on it. So if you place any non-negligible value on future generations whose existence is threatened, reducing existential risk has to be the best possible contribution to humanity you are in a position to make. Given that existential risk is also one of the major themes of Overcoming Bias and of Eliezer's work, it's striking that we don't talk about it more here.

One reason of course was the bar until yesterday on talking about artificial general intelligence; another factor are the many who state in terms that they are not concerned about their contribution to humanity. But I think a third is that many of the things we might do to address existential risk, or other issues of concern to all humanity, get us into politics, and we've all had too much of a certain kind of argument about politics online that gets into a stale rehashing of talking points and point scoring.

If we here can't do better than that, then this whole rationality discussion we've been having comes to no more than how we can best get out of bed in the morning, solve a puzzle set by a powerful superintelligence in the afternoon, and get laid in the evening. How can we use what we discuss here to be able to talk about politics without spiralling down the plughole?

I think it will help in several ways that we are a largely community of materialists and expected utility consequentialists. For a start, we are freed from the concept of "deserving" that dogs political arguments on inequality, on human rights, on criminal sentencing and so many other issues; while I can imagine a consequentialism that valued the "deserving" more than the "undeserving", I don't get the impression that's a popular position among materialists because of the Phineas Gage problem. We need not ask whether the rich deserve their wealth, or who is ultimately to blame for a thing; every question must come down only to what decision will maximize utility.

For example, framed this way inequality of wealth is not justice or injustice. The consequentialist defence of the market recognises that because of the diminishing marginal utility of wealth, today's unequal distribution of wealth has a cost in utility...
TED Talks for Less Wrong
by Cyan — 2009-05-02T03:32:09.269Z

Dan Ariely talks about pain and cheating. In a nutshell: people report less pain when (i) they experience the strongest pain first; (ii) they experience less pain for a longer interval rather than more pain for a shorter interval; (iii) they can take breaks. The data falsifies the common intuition that people will prefer short, high intensity pain. In general, people tend to cheat more when (i) they obtain things other than actual cash; (ii) they observe in-group members cheating successfully; they tend to cheat less when (i) they take away cash; (ii) they observe out-group members cheating successfully; (iii) they experience priming with moral concepts such as the Ten Commandments.

Post yours in comments. I've put a couple with the theme "how brains work" down there.
Second London Rationalist Meeting upcoming - Sunday 14:00
by taw — 2009-05-02T00:17:55.558Z

The second meeting will take place on Sunday (2009-05-03) 14:00, in cafe on top of the Waterstones bookstore near the Piccadilly Circus Tube station.

If you want to know more, email me (Tomasz.Wegrzanowski@gmail.com) for details. Or just come straight away.

If you're wondering what it will be like - here's summary of the first meeting.
Conventions and Confusing Continuity Conundrums
by Psy-Kosh — 2009-05-01T01:41:14.146Z

The next "How Not to be Stupid" may be a bit delayed for a couple of reasons.

First, there appears to be a certain unstated continuity assumption in the material I've been working from that would probably be relevant for the next posting. As I said in the intro post, I'm working from Stephen Omhunduro's vulnerability argument, but filling in what I viewed as missing bits, generalizing one or two things, and so on. Anyways, the short of it is I thought that I was able to how to derive the relevant continuity condition, but turns out I need to think about that a bit more carefully.

If I remain stumped on that bit, I'll just post and explicitly state the assumption, pointing it out as a potential problem area that needs to be dealt with one way or another. ie, either solved somehow, or demonstrate that it actually is invalid (thus causing some issues for decision theory...)

Also, I'm going to be at Penguicon the next few days.

Actually, I think I'll right now state the continuity condition I need, let others play with it too:

Basically, I need it to be that if there's a preference ranking A > C > B, there must exist some p such that the p*A + (1-p)*B lottery ranks equal to C. (That is, that the mixing lotteries correspond to a smooth spectrum of preferences between, well, the things they are a mixing of rather than having any discontinuous jumps.)

Anyways, I hope we can put this little trouble bit behind us and resume climbing the shining path of awakening to nonstupidity. :)
Rationalist Role in the Information Age
by byrnema — 2009-04-30T18:24:24.865Z

In response to Marketing rationalism, Bystander Apathy, Step N1 in Extreme Rationality: It Could Be Great, and Rationality: Common Interest of Many Causes.

The problem that motivates this post is:
“Given a controversial question in which there are good and bad arguments on both sides, as well as unreliable and conflicting information, how do you determine the answer when you’re not yourself an expert in the subject?”

Well into the information age, we are still not pooling our resources in the most efficient way to get to the bottom of things. It would be enormously useful to develop some kind of group strategy to answer questions that have solutions somewhere in there.

The idea I'm presenting is a way to apply our intellectual (and rational) resources in a niche way, that I will shortly describe, to facilitate public (non-expert) understanding of real world problems.

The Niche and the Need

Science, obviously, does the best job of solving problems. I'm confident that epidemiologists are effectively and efficiently working on the best models for pandemics as I write this post.

And journalists do a pretty good job of what they do: providing information about what the scientists are doing. What's best about how journalists do that is that they always provide the source of the information, so that a rational person can judge the truth-value of that information. A qualified and rational person can then put the information in perspective.

Alas, people are not so good at interpreting the information: they are neither rational in weighting the information nor qualified to put it in perspective. (Presumably, the epidemiologists are too busy to do this.)

Interpreting information correctly is a service that rationalists could provide.

How we could do it:

1. Information culled from various sources would be posted in the first half of a post titled P and updated continuously.

2. As a rational group, within threads, we would discuss interpretations and implications of the available information.

3. Only consensus views would be presented in the second half of the post P, updated continuously to prioritize the most relevant information on top.

Why we would do it

1. It would be objectively and enormously useful to cull useful and consensus interpretations. It is a time-consuming ta

...
Rationalistic Losing
by MrHen — 2009-04-30T12:48:52.437Z

Playing to learn

I like losing. I don't even think that losing is necessarily evil. Personally, I believe this has less to do with a desire to lose and more to do with curiosity about the game-space.

Technically, my goals are probably shifted into some form of meta-winning — I like to understand winning or non-winning moves, strategies, and tactics. Actually winning is icing on the cake. The cake is learning as much as I can about whatever subject in which I am competing. I can do that if I win; I can do that if I lose.

I still prefer winning and I want to win and I play to win, but I also like losing. When I dive into a competition I will like the outcome. No matter what happens I will be happy because I will either (a) win or (b) lose and satiate my curiosity. Of course, learning is also possible while watching someone else lose and this generally makes winning more valuable than losing (I can watch them lose). It also provides a solid reason to watch and study other people play (or play myself and watch me "lose").

The catch is that the valuable knowledge contained within winning has diminishing returns. When I fight I either (a) win or (b) lose and, as a completely separate event, (c) may have an interesting match to study. Ideally I get (a) and (c) but the odds of (c) get lower the more I dominate because my opponents could lose in a known fashion (by me winning in an "old" method). (c) should always be found next to (b). If there is a reason I lost I should learn the reason. If I knew the reason I should not have lost. Because of this, (c) offsets the negative of (b) and losing is valuable. This makes winning and losing worth the effort. When I lose, I win.

Personally, I find (c) so valuable that I start getting bored when I no longer see anything to learn. If I keep winning over and over and never learn anything from the contest I have to find someone stronger to play or start losing creatively so that I can start learning again. Both of these solutions set up scenarios where I am increasing my chances to lose. Mathematically, this starts to make sense if the value of knowledge gained and the penalty of losing combine into something greater than winning without learning anything. (c - b > a) My hunches tell me that I value winning too little and curiosity is starting to curb my desire to win. I am not playing to win; I am playing to learn.

skip to...
How Not to be Stupid: Adorable Maybes
by Psy-Kosh — 2009-04-29T19:15:01.162Z

Previous: Know What You Want

Ah wahned yah, ah wahned yah about the titles. </some enchanter named Tim>

(Oh, a note: the idea here is to establish general rules for what sorts of decisions one in principle ought to make, and how one in principle ought to know stuff, given that one wants to avoid Being Stupid. (in the sense described in earlier posts) So I'm giving some general and contrived hypothetical situations to throw at the system to try to break it, to see what properties it would have to have to not automatically fail.)

Okay, so assuming you buy the argument in favor of ranked preferences, let's see what else we can learn by considering sources of, ahem, randomness:

Suppose that either via indexical uncertainty, or it turns out there really is some nondeterminism in the universe, or there's some source of bits such that the only thing you're able to determine about it is that the ratio of 1s it puts out to total bits is p. You're not able to determine anything else about the pattern of bits, they seem unconnected to each other. In other words, you've got some source of uncertainty that leaves you only knowing that some outcomes happen more often than others, and potentially you know something about the precise relative rates of those outcomes.

I'm trying here to avoid actually assuming epistemic probabilities. (If I've inserted an invisible assumption for such that I didn't notice, let me know.) Instead I'm trying to construct a situation in which that specific situation can be accepted as at least validly describable by something resembling probabilities (propensity or frequencies. (frequencies? aieeee! Burn the heretic, or at least flame them without mercy! :))) So, for whatever reason, suppose the universe or your opponent or whatever has access to such a source of bits. Let's consider some of the implications of this.

For instance, suppose you prefer A > B.

Now, suppose you are somehow presented with the following choice: Choose B, or choose a situation in which if, at a specific instance, the source outputs a 1, A will occur. Otherwise, B occurs. We'll call this sort of situation a p*A + (1-p)*B lottery, or simply p*A + (1-p)*B

So, which should you prefer? B or the above lottery? (assume there's no other cost other than declaring your choice. Or just wanting the choice. It's not a "pay for a lottery ticket" scenario yet. Just a "...
Fire and Motion
by rwallace — 2009-04-29T16:06:11.967Z

Related to: Extreme Rationality: It's Not That Great

On the recent topics of "rationality is all very well but how do we translate understanding into winning?" and "isn't akrasia the most common limiting factor?", one of the best (non-recent) articles on practical rationality that I've come across is:

http://www.joelonsoftware.com/articles/fog0000000339.html

Interestingly, it uses a different kind of martial art as a metaphor. I conjecture it to be the sort of metaphor that just works well for humans.

(Most of Spolsky's posts are good reading even if you're not a programmer. I'm not in the New York real estate market but I still enjoyed his posts on that topic. He's just that good a writer.)
Fighting Akrasia: Incentivising Action
by Gordon Seidoh Worley — 2009-04-29T13:48:56.070Z

Related To: Incremental Progress and the Valley, Silver Chairs, Paternalism, and Akrasia, How a pathological procrastinator can lose weight

Akrasia can strike anywhere, but one place it doesn't seem to strike too often or too severely, assuming you are employed, is in the work place. You may not want to do something, and it might take considerable willpower to perform a task, but unless you want to get fired you can't always play Solitaire. The reason is clear to most working folks: you have to do your job to keep it, and not keeping your job is often worse than performing an undesirable task, so you suck it up and find the willpower to make it through the day. So one question we might ask is, how can we take this motivational method and put it to our own use?

First, let's look at the mechanics of the method. You have to perform a task and some exterior entity will pay you unless you fail utterly to perform the task. Notice that this is quite different from working for prizes, where you receive pay in exchange for performing a particular task. Financially they may appear the same, but from the inside of the human mind they are quite different. In the former case you are motivated by a potential loss, whereas in the later you are motivated by a potential gain. Since losses carry more weight than gains, in general the former model will provide more motivation than the latter, keeping in mind that loss aversion is a statistical property of human thought and there may be exceptions.

This suggests that certain techniques will work better more often than others. For example, if you run a website about rationality and need programming work done for it, you have a couple of options. You can wait for someone to volunteer their time, you can offer a prize for implementing certain features for the site, or you can offer to pay someone to do it on the condition that if they don't meet certain deadlines they won't get paid or will be paid a lesser amount. If you aren't so lucky as to have someone come along who will volunteer their time and do a fantastic job for free, you are faced with accepting mediocre free work, offering prizes, or paying someone. Since prizes are usually inefficient, it appears that offering to pay someone is the best option, so long as you are able to stipulate that t...
How to come up with verbal probabilities
by jimmy — 2009-04-29T08:35:01.709Z

Unfortunately, we are kludged together, and we can't just look up our probability estimates in a register somewhere when someone asks us "How sure are you?".
The usual heuristic for putting a number on the strength of beliefs is to ask "When you're this sure about something, what fraction of the time do you expect to be right in the long run?". This is surely better than just "making up" numbers with no feel for what they mean, but still has it's faults. The big one is that unless you've done your calibrating, you may not have a good idea of how often you'd expect to be right.
I can think of a few different heuristics to use when coming up with probabilities to assign.
1) Pretend you have to bet on it. Pretend that someone says "I'll give you ____ odds, which side do you want?", and figure out what the odds would have to be to make you indifferent to which side you bet on. Consider the question as if though you were actually going to put money on it . If this question is covered on a prediction market, your answer is given to you.

2) Ask yourself how much evidence someone would have to give you before you're back to 50%. Since we're trying to update according to bayes law, knowing how much evidence it takes to bring you to 50% tells you the probability you're implicitely assigning.

For example, pretend someone said something like "I can guess peoples names by their looks". If he guesses the first name right, and it's a common name, you'll probably write it off as fluke. The second time you'll probably think he knew the people or is somehow fooling you, but conditional on that, you'd probably say he's just lucky. By bayes law, this suggests that you put the prior probability of him pulling this stunt at 0.1%<p<3%, and less than 0.1% prior probability of him having his claimed skill. If it takes 4 correct calls to bring you to equally unsure either way, then thats about 0.03^4 if they're common names, or one in a million1...
There's a couple neat things about this trick. One is that it allows you to get an idea of what your subconscious level of certainty is before you ever think of it. You can imagine your immediate reaction to "Why yes, my name is Alex, how did you know" as well as your carefully deliberated response to the same data (if they're much different, be wary of belief in belief). The other nea...
Wednesday depends on us.
by byrnema — 2009-04-29T03:47:50.833Z

In response to Theism, Wednesday, and Not Being Adopted

It is a theist cliché: you need religion to define morality. The argument doesn't have to be as simplistic as “you need God to impose it”, but at the least it is the belief that your community needs to agree on what is ethical. When a community starts talking about what is ethical, they quickly depart from anything strictly fact-based. As a community, they need to figure out what the morality is (e.g., love your neighbor), construct a narrative using symbols that make sense to everyone (there’s this external entity God, someone like your father, who wants it this way) and enforce it (if you don’t go along, you go to Hell.) This is probably 40% of what religion is.

While the development of a religion is a community effort to some extent (communities choose among competing religions and religions evolve), the main work is done by the priests. The priest is usually an exceptionally good thinker/reasoner/philosopher – maybe 4-7 3-5 standard deviations from the mean. [Correction]

There are a few things that are very confusing to Wednesday when you try to convert her. First of all, she understands on at least a subconscious level that religion is her community’s ethical system. When you say you don’t believe in God, she thinks you’re saying, ‘it’s OK to torture babies’. What’s scary is that she’s somewhat justified here: without an externally applied ethical belief system, individual ethics can vary widely from what she accepts as ethical (and what you accept as ethical).

If morality is something that humanity protects, can we blame Wednesday for that?

Fortunately, Eliezer assures us: Anyone worried that reductionism drains the meaning from existence can stop worrying.

This brings us to the second problem for Wednesday. While I believe Eliezer about rationality not denying morality and meaning, I believe him in the same way Wednesday believes her priest: because he’s been right before and I figure I probably have something to learn.

Rational arguments sound just as good to Wednesday as her Bishop’s theological arguments. What is she to do? Wednesday’s priest has warned her of this with some well-c

...
Generalizing From One Example
by Scott Alexander — 2009-04-28T22:00:50.764Z

Related to: The Psychological Unity of Humankind, Instrumental vs. Epistemic: A Bardic Perspective
"Everyone generalizes from one example. At least, I do."

-- Vlad Taltos (Issola, Steven Brust)
My old professor, David Berman, liked to talk about what he called the "typical mind fallacy", which he illustrated through the following example:
There was a debate, in the late 1800s, about whether "imagination" was simply a turn of phrase or a real phenomenon. That is, can people actually create images in their minds which they see vividly, or do they simply say "I saw it in my mind" as a metaphor for considering what it looked like?
Upon hearing this, my response was "How the stars was this actually a real debate? Of course we have mental imagery. Anyone who doesn't think we have mental imagery is either such a fanatical Behaviorist that she doubts the evidence of her own senses, or simply insane." Unfortunately, the professor was able to parade a long list of famous people who denied mental imagery, including some leading scientists of the era. And this was all before Behaviorism even existed.
The debate was resolved by Francis Galton, a fascinating man who among other achievements invented eugenics, the "wisdom of crowds", and standard deviation. Galton gave people some very detailed surveys, and found that some people did have mental imagery and others didn't. The ones who did had simply assumed everyone did, and the ones who didn't had simply assumed everyone didn't, to the point of coming up with absurd justifications for why they were lying or misunderstanding the question. There was a wide spectrum of imaging ability, from about five percent of people with perfect eidetic imagery1 to three percent of people completely unable to form mental images2.

Dr. Berman dubbed this the Typical Mind Fallacy: the human tendency to believe that one's own mental structure can be generalized to apply to everyone else's.

He kind of took this idea and ran with it. He interpreted certain passages in George Berkeley's biography to mean that Berkeley was an eidetic imager, and that this was why the idea of the universe as sense-perception held such interest to him. He also suggested that experience of consciousness and qualia were as variable as imaging, and that philosophers who deny their existence (Ryle? Dennett? Behaviorists?) were simply people whose mind lacked the a...