trentmkelly/LessWrong-43k huggingface dataset:
Would Your Real Preferences Please Stand Up?
by Scott Alexander — 2009-08-08T22:57:09.266Z

Related to: Cynicism in Ev Psych and Econ

In Finding the Source, a commenter says:

I have begun wondering whether claiming to be victim of 'akrasia' might just be a way of admitting that your real preferences, as revealed in your actions, don't match the preferences you want to signal (believing what you want to signal, even if untrue, makes the signals more effective).

I think I've seen Robin put forth something like this argument [EDIT: Something related, but very different], and TGGP points out that Brian Caplan explicitly believes pretty much the same thing1:

I've previously argued that much - perhaps most - talk about "self-control" problems reflects social desirability bias rather than genuine inner conflict.
Part of the reason why people who spend a lot of time and money on socially disapproved behaviors say they "want to change" is that that's what they're supposed to say.
Think of it this way: A guy loses his wife and kids because he's a drunk. Suppose he sincerely prefers alcohol to his wife and kids. He still probably won't admit it, because people judge a sinner even more harshly if he is unrepentent. The drunk who says "I was such a fool!" gets some pity; the drunk who says "I like Jack Daniels better than my wife and kids" gets horrified looks. And either way, he can keep drinking.

I'll call this the Cynic's Theory of Akrasia, as opposed to the Naive Theory. I used to think it was plausible. Now that I think about it a little more, I find it meaningless. Here's what changed my mind.

What part of the mind, exactly, prefers a socially unacceptable activity (like drinking whiskey or browsing Reddit) to an acceptable activity (like having a wife and kids, or studying)? The conscious mind? As Bill said in his comment, it doesn't seem like it works this way. I've had akrasia myself, and I never consciously think "Wow, I really like browsing Reddit...but I'll trick everyone else into thinking I'd rather be studying so I get more respect. Ha ha! The fools will never see it coming!"
No, my conscious mind fully believes that I would rather be studying2. And this even gets reflected in my actions. I've tried anti-procrastination techniques, both successfully and unsuccessfully, without ever telling them to another living soul. People trying to diet don't take out the cupcakes as soon as no one else is looking (or, if they do, they feel guilty about it).
Thi...
Dreams with Damaged Priors
by Eliezer Yudkowsky — 2009-08-08T22:31:22.994Z

Dreaming is the closest I've gotten to testing myself against the challenge of maintaining rationality under brain damage. So far, my trials have exhibited mixed results.

In one memorable dream a few years ago, I dreamed that the Wall Street Journal had published an article about "Eliezer Yudkowsky", but it wasn't me, it was a different "Eliezer Yudkowsky", and in the dream I wondered if I needed to write a letter to clarify this. Then I realized I was dreaming within the dream... and worried to myself, still dreaming: "But what if the Wall Street Journal really does have an article about an 'Eliezer Yudkowsky' who isn't me?"

But then I thought: "Well, the probability that I would dream about a WSJ article like that, given that a WSJ article like that actually exists in this morning's paper, is the same as the probability that I would have such a dream, given that no such article is in this morning's paper. So by Bayes's Theorem, the dream isn't evidence one way or the other. Thus there's no point in trying to guess the answer now - I'll find out in the morning whether there's an article like that." And, satisfied, my mind went back to ordinary sleep.

I find it fascinating that I was able to explicitly apply Bayes's Theorem in my sleep to correctly compute the 1:1 likelihood ratio, but my dreaming mind didn't notice the damaged prior - didn't notice that the prior probability of such a WSJ article was too low to justify raising the hypothesis to my attention.

At this point even I must concede that there is something to the complaint that, in real-world everyday life, Bayesians dispense too little advice about how to compute priors. With a damaged intuition for the weight of evidence, my dreaming mind was able to explicitly compute a likelihood ratio and correct itself. But with a damaged intuition for the prior probability, my mind didn't successfully check itself, or even notice a problem - didn't get as far as asking "But what is the prior probability?"

On July 20 I had an even more dramatic dream - sparking this essay - when I dreamed that I'd googled my own name and discovered that one of my OBLW articles had been translated into German and published, without permission but with attribution, in a special issue of the Journal of Applied Logic to commemorate the death of Richard Thaler (don't worry, he is in...
A note on hypotheticals
by PhilGoetz — 2009-08-07T18:56:55.392Z

People frequently describe hypothetical situations on LW. Often, other people make responses that suggest they don't understand the purpose of hypotheticals.

When someone puts forth the hypothetical A, it doesn't mean they believe it is true. They may be trying to show not(A).

When someone posits A => B (A implies B), it doesn't mean that they believe A is true. The proposition A => B is commonly used to prove that B is true, or that A is false.

A solution to a hypothetical scenario is useful only if, when you map it back into the original domain, it solves the original problem.

I'll expand on the last point. Sorry for being vague. I'm trying not to name names.

When a hypothetical is put forward to test a theory, ignore aspects of the hypothetical scenario that don't correspond to parts of the theory. Don't get emotionally involved. Don't think of the hypothetical as a narrative. A hypothetical about Omega sounds a lot like a story about a genie from a lamp, but you should approach it in a completely different way. Don't try to outsmart Omega (unless you're making a point about the impossibility of an Omega who can eg decide undecidable problems). When you find a loophole in the way the hypothetical is posed, that doesn't exist in the original domain, point it out only if you are doing so to improve the phrasing of the hypothetical situation.

John Searle's Chinese Room is an example of a hypothetical in which it is important to not get emotionally involved. Searle's conclusion is that the man in the Chinese room doesn't understand Chinese; therefore, a computer doesn't understand Chinese. His model maps the running software onto the complete system of room plus man plus cards; but when he interprets it, he empathizes with the human on each half of the mapping, and so maps the locus of consciousness from the running software onto just the man.1

Sometimes it's difficult to know whether your solution to a hypothetical is exploiting a loophole in the hypothetical, or finding a solution to the original problem. But when the original problem is testing a mathematical model, it's usually obvious. There are a few general situations where it's not obvious.

Consciousness is often a tricky area that makes it hard to tell whether you are looking at a solution to the original pr...
Fighting Akrasia: Finding the Source
by Gordon Seidoh Worley — 2009-08-07T14:49:38.900Z

Followup to: Fighting Akrasia: Incentivising Action

Influenced by: Generalizing From One Example

Previously I looked at how we might fight akrasia by creating incentives for actions. Based on the comments to the previous article and Yvain's now classic post Generalizing From One Example, I want to take a deeper look at the source of akrasia and the techniques used to fight it.

I feel foolish for not looking at this closer first, but let's begin by asking what akrasia is and what causes it. As commonly used, akrasia is the weakness-of-will we feel when we desire to do something but find ourselves doing something else. So why do we experience akrasia? Or, more to the point, why to we feel a desire to take actions contrary the actions we desire most, as indicated by our actions? Or, if it helps, flip that question and ask why are the actions we take not always the ones we feel the greatest desire for?

First, we don't know the fine details of how the human brain makes decisions. We know what it feels like to come to a decision about an action (or anything else), but how the algorithm feels from the inside is not a reliable way to figure out how the decision was actually made. But because most people can relate to a feeling of akrasia, this suggests that there is some disconnect between how the brain decides what actions are most desirable and what actions we believe are most desirable. The hypothesis that I consider most likely is that the ability to form beliefs about desirable actions evolved well after the ability to make decisions about what actions are most desirable, and the decision-making part of the brain only bothers to consult the belief-about-desirability-of-actions part of the brain when there is a reason to do so from evolution's point of view.1 As a result we end up with a brain that only does what we think we really want when evolutionarily prudent, hence we experience akrasia whenever our brain doesn't consider it appropriate to consult what we experience as desirable.

This suggests two main ways of overcoming akrasia assuming my hypothesis (or something close to it) is correct: make the actions we believe to be desirable also desirable to the decision-making part of the brain or make the decision-making part of the brain consult the belief-about-desirability-of-actions par...
Robin Hanson's lists of Overcoming Bias Posts
by AndrewH — 2009-08-06T20:10:39.207Z

I have created a list of Overcoming Bias posts for Robin Hanson available here. Additionally, using the links inside each posts, I have created a set of graphs (available here) such that if post A has a link to post B, then there is an arc from B to A. Enjoy! (There are also ones for Eliezer here).
Exterminating life is rational
by PhilGoetz — 2009-08-06T16:17:43.983Z

Followup to This Failing Earth, Our society lacks good self-preservation mechanisms, Is short term planning in humans due to a short life or due to bias?

I don't mean that deciding to exterminate life is rational. But if, as a society of rational agents, we each maximize our expected utility, this may inevitably lead to our exterminating life, or at least intelligent life.

Ed Regis reports on p 216 of “Great Mambo Chicken and the TransHuman Condition,” (Penguin Books, London, 1992):

Edward Teller had thought about it, the chance that the atomic explosion would light up the surrounding air and that this conflagration would then propagate itself around the world. Some of the bomb makers had even calculated the numerical odds of this actually happening, coming up with the figure of three chances in a million they’d incinerate the Earth. Nevertheless, they went ahead and exploded the bomb.

Was this a bad decision? Well, consider the expected value to the people involved. Without the bomb, there was a much, much greater than 3/1,000,000 chance that either a) they would be killed in the war, or b) they would be ruled by Nazis or the Japanese. The loss to them if they ignited the atmosphere would be another 30 or so years of life. The loss to them if they lost the war and/or were killed by their enemies would also be another 30 or so years of life. The loss in being conquered would also be large. Easy decision, really.

Suppose that, once a century, some party in a conflict chooses to use some technique to help win the conflict that has a p=3/1,000,000 chance of eliminating life as we know it. Then our expected survival time is 100 times the sum from n=1 to infinity of np(1-p)n-1. If I've done my math right, that's ≈ 33,777,000 years.

This supposition seems reasonable to me. There is a balance between offensive and defensive capability that shifts as technology develops. If technology keeps changing, it is inevitable that, much of the time, a technology will provide the ability to destroy all life before the counter-technology to defend against it has been developed. In the near future, biological weapons will be more able to wipe out life than we are able to defend against them. We may then develop the ability to defend against biological attacks; we may then be safe...
LW/OB Rationality Quotes - August 2009
by Rune — 2009-08-06T13:35:40.985Z

I always see that the monthly Rationality Quotes thread has this line: "Do not quote comments/posts on LW/OB - if we do this, there should be a separate thread for it." This is the thread for those quotes.

This is a (possibly) monthly thread for posting any interesting rationality-related quotes you've seen on LW/OB.

Please post all quotes separately (so that they can be voted up/down separately) unless they are strongly related/ordered.

Do not quote yourself.

Do not post quotes that are NOT comments/posts on LW/OB - there is a separate thread for this.

No more than 5 quotes per person per monthly thread, please.
The Objective Bayesian Programme
by cousin_it — 2009-08-06T10:33:40.153Z

Followup to: Bayesian Flame.

This post is a chronicle of my attempts to understand Cyan's #2. (Bayesian Flame was an approximate parse of #1.) Warning: long, some math, lots of links, probably lots of errors. At the very least I want this to serve as a good reference for further reading.

Introduction

To the mathematical eye, many statistical problems share the following minimal structure:

A space of parameters. (Imagine a freeform blob without assuming any metric or measure.)

A space of possible outcomes. (Imagine another, similarly unstructured blob.)

Each point in the parameter space determines a probability measure on the outcome space.

By itself, this kind of input is too sparse to yield solutions to statistical problems. What additional structure on the spaces should we introduce?

The answer that we all know and love

Assuming some "prior" probability measure on the parameter space yields a solution that's unique, consistent and wonderful in all sorts of ways. This has led some people to adopt the "subjectivist" position saying priors are so basic that they ought not be questioned. One of its most prominent defenders was Leonard Jimmie Savage who put forward the following argument:

Suppose, for example, that the person is offered an even-money bet for five dollars - or, to be ultra-rigorous, for five utiles - that internal combustion engines in American automobiles will be obsolete by 1970. If there is any event to which an objectivist would refuse to attach probability, that corresponding to the obsolescence in question is one... Yet, I think I may say without presumption that you would regard the bet against obsolescence as a very sound investment.

This is a fine argument for using priors when you're betting money, but there's a snag: however much you are willing to bet, this doesn't give you grounds to publish papers about the future that you inferred from your intuitive prior! Any apriori information used in science should be justified for scientific objectivity.

(At this point Eliezer raises the suggestion that scientists ought to communicate with likelihood ratios only. That might be a brave new world to live in; too bad we'll have to stop teaching kids that g approximately equals 9.8 m/s2 and give them likelihood profiles instead.)

Rather than dive deeper into the fascinating topic of "uninformative priors", let's go bac...
Why Real Men Wear Pink
by Scott Alexander — 2009-08-06T07:39:04.821Z

"Fashion is a form of ugliness so intolerable we have to alter it every six months."

-- Oscar Wilde

For the past few decades, I and many other men my age have been locked in a battle with the clothing industry. I want simple, good-looking apparel that covers my nakedness and maybe even makes me look attractive. The clothing industry believes someone my age wants either clothing laced with profanity, clothing that objectifies women, clothing that glorifies alcohol or drug use, or clothing that makes them look like a gangster. And judging by the clothing I see people wearing, on the whole they are right.
I've been working my way through Steven Pinker's How The Mind Works, and reached the part where he quotes approvingly Quentin Bell's theory of fashion. The theory provides a good explanation for why so much clothing seems so deliberately outrageous.

Bell starts by offering his own explanation of the "fashion cycle". He claims that the goal of fashion is to signal status. So far, so obvious. But low-status people would like to subvert the signal. Therefore, the goal of lower class people is to look like upper class people, and the goal of upper class people is to not look like lower class people.
One solution is for the upper class to wear clothing so expensive the lower class could not possibly afford it. This worked for medieval lords and ladies, but nowadays after a while mass production will kick in and K-Mart will have a passable rhinestone based imitation available for $49.95. Once the lower class is wearing the once fashionable item, the upper class wouldn't be caught dead in it. They have to choose a new item of clothing to be the status signal, after a short period of grace the lower class copy that too, and the cycle begins again.
For example, maybe in early 2009 a few very high-status people start wearing purple. Everyone who is "in the know" enough to understand that they are trend-setters switches to purple. Soon it becomes obvious that lots of "in the know" people are wearing purple, and anyone who reads fashion magazines starts stocking up on purple clothing. Soon, only the people too out-of-the-loop to know about purple and the people too poor to immediately replace all their clothes are wearing any other color. In mid-2009, some extremely high-status people now go out on a limb and start wearing green; everyone else is too low-status to be comfortabl...
Rationality Quotes - August 2009
by Vladimir_Nesov — 2009-08-06T01:58:49.178Z

A monthly thread for posting any interesting rationality-related quotes you've seen recently on the Internet, or had stored in your quotesfile for ages.

Please post all quotes separately (so that they can be voted up/down separately) unless they are strongly related/ordered.

Do not quote yourself.

Do not quote comments/posts on LW/OB - if we do this, there should be a separate thread for it.

No more than 5 quotes per person per monthly thread, please.
Recommended reading: George Orwell on knowledge from authority
by CronoDAS — 2009-08-05T15:30:46.718Z

This is an excerpt from an article George Orwell wrote in 1946. I will let the text speak for itself.

Somewhere or other — I think it is in the preface to saint Joan — Bernard Shaw remarks that we are more gullible and superstitious today than we were in the Middle Ages, and as an example of modern credulity he cites the widespread belief that the earth is round. The average man, says Shaw, can advance not a single reason for thinking that the earth is round. He merely swallows this theory because there is something about it that appeals to the twentieth-century mentality.

Now, Shaw is exaggerating, but there is something in what he says, and the question is worth following up, for the sake of the light it throws on modern knowledge. Just why do we believe that the earth is round? I am not speaking of the few thousand astronomers, geographers and so forth who could give ocular proof, or have a theoretical knowledge of the proof, but of the ordinary newspaper-reading citizen, such as you or me.

As for the Flat Earth theory, I believe I could refute it. If you stand by the seashore on a clear day, you can see the masts and funnels of invisible ships passing along the horizon. This phenomenon can only be explained by assuming that the earth's surface is curved. But it does not follow that the earth is spherical. Imagine another theory called the Oval Earth theory, which claims that the earth is shaped like an egg. What can I say against it?

Against the Oval Earth man, the first card I can play is the analogy of the sun and moon. The Oval Earth man promptly answers that I don't know, by my own observation, that those bodies are spherical. I only know that they are round, and they may perfectly well be flat discs. I have no answer to that one. Besides, he goes on, what reason have I for thinking that the earth must be the same shape as the sun and moon? I can't answer that one either.

My second card is the earth's shadow: When cast on the moon during eclipses, it appears to be the shadow of a round object. But how do I know, demands the Oval Earth man, that eclipses of the moon are caused by the shadow of the earth? The answer is that I don't know, but have taken this piece of information blindly from newspaper articles and science booklets.

D

...
A Normative Rule for Decision-Changing Metrics
by FrankAdamek — 2009-08-05T05:07:05.158Z

Yesterday I wrote about the difficulties of ethics and potential people. Namely, that whether you bring a person into existence or not changes the moral metric by which your decision is measured. At first all I had was the observation suggesting that the issue was complex, but no answer to the question "Well then, what should we do?" I will write now about an answer that came to me.

All theories regarding potential people start by comparing outcomes to find which is most desirable, then moving towards it. However I believe I have shown that there are two metrics regarding such questions, and those metrics can disagree. What then do we do?

We are always in a particular population ourselves, and so we can ask not which outcome is preferable, but if we should move from one situation to another. This allows us to consider the alternate metrics in series. For an initial name more attractive than "my rule" I will refer to the system as Deontological Consequentialism, or DC. I'm open to other suggestions.

Step 1: Consider your action with the metric of new people not coming to be: that is, only the welfare of the people who will exist regardless of your decision.* I will assume in this discussion there are three possibilities: people receive higher utility, lower utility, or effectively unchanged utility. You might dispense with the third option, the results are similar.

First, if you expect reduced utility for existing people from taking an action: do not take that action. This is regardless of how many new people might otherwise exist or how much utility they might have; if we never bring them into existence, we have wronged no one.

This is the least intuitive aspect of this system, though it is also the most critical for avoiding the paradoxes of which I am aware. I think this unintuitive nature mostly stems from automatically considering future people as if they exist. I'd also note that our intuitions are really not used to dealing with this sort of question, but one more approachable example exists. If a couple strongly expects they will be unhappy having children, will derive little meaning from parenthood and few material benefits from their children and we believe them, if in short they really don't want kids, I suspect few will tell them they really ought to have children anyway as long as the children will be happy. I think few people consider how very h...