trentmkelly/LessWrong-43k huggingface dataset:
Humans don't generally have utility functions
by Manfred — 2010-11-14T09:54:43.821Z

First, the background:

Humans obviously have a variety of heuristics and biases that lead to non-optimal behavior. But can this behavior truly not be described by a function?

Well, the easiest way to show that utility isn't described by a function is to show the existence of cycles. For example, if I prefer A to B, B to C, and C to A, that's a cycle - if all three are available I'll never choose, and if each switch is an increase in utility, my utility blows up to infinity! Well, really, it simply becomes undefined.

Do we have real-world examples of cycles in utility comparisons? Sure. For a cycle of size 2, Eliezer cites the odd behavior of people with regard to money and probabilities of money. However, the money-pumps he cites are rather inefficient. Almost any decision that seems "arbitrary" to us can be translated into a cycle. For example, anchoring means that people assign higher value to a toaster when a more expensive toaster is sitting next to it. But most people, if asked, would certainly assign tiny value to adding/removing options they know they won't buy. So we get the result that they must value the toaster more than they value the toaster. The conjunction fallacy can be made into a cycle by the same reasoning if you ask people to bet on the thing happening together and then ask them to bet on the things happening separately.

So at the very least, not all humans have utility functions, which means that the human brain doesn't automatically give us a utility function to use - if we want one, we have to sculpt it ad-hoc out of intuitions using our general reasoning, and like most human things it probably won't be the best ever.

So, what practical implications does this have, aside from "people are weird?"

Well, I can think of two interesting things. First, there are the implications for utilitarian ethics. If utility functions are arbitrary not just on a person to person basis, but even within a single person, choosing between options using utilitarian ethics requires stronger, more universal moral arguments. The introspective "I feel like X, therefore my utility function must include that" is now a weak argument, even to yourself! The claim that "a utility function of universe-states exists" loses none of its consequences though, like alerting you that som...
Mental focus
by NancyLebovitz — 2010-11-14T09:47:21.262Z

I just noticed how much effort I put into remembering what I was thinking-- I do a lot of "I had an interesting idea about that which I wanted to include-- what was the idea?".

It can take at least 3 or 4 repetitions before the idea finally gets typed out, partly because I might be typing or formulating something else and then want to get back to an earlier idea.

I'd be amazed if I were the only person here with this problem. Any suggestions? Does dual n-back help?
Hi - I'm new here - some questions
by InquilineKea — 2010-11-14T04:11:06.176Z

Hello everyone,

I'm new here, although I've read Less Wrong and Overcoming Bias on and off for the last few years. Anyways, I'm InquilineKea (or Simfish), and I have a website at http://simfishthoughts.wordpress.com/. I think about everything, so I feel that this might be the perfect community for me. I do have some questions though - are we allowed to post anything in this part of the site? (like, could we treat this part like another forum, albeit an intellectually mature forum?) Or do we have to keep things formal? I tend to post a high number of threads, but there don't seem to be many threads here. Are there any terms of service/rules? Or are things just governed by upvotes/downvotes? (much like reddit)

Anyways, I'm an astronomy/physics/math major at the University of Washington (I got in through an early entrance program) and I'm planning on applying to astrophysics grad school fairly soon. However, I'm also intensely interested in complex adaptive systems and data mining, especially as they relate to the social sciences. I'm especially interested in Consilience and in trying to find trends behind every academic field (in fact, I do want to get to a graduate level of education in every natural and social science there is). I'm demographics junkie who literally pours over all the charts and tables of every demographic statistic I can find, although it sometimes ends up hurting my grades. My favorite blogs are Gene Expression, FuturePundit/ParaPundit, and Overcoming Bias. Which I'm sure a lot of people here read.

I always think in terms of maximizing "utility" and maximizing "efficiency". So this leads me to do many untraditional things. For one thing, I have attention deficit disorder, so I realize that I frequently have to take untraditional approaches. The Internet has always been a savior for me because I can always stop and continue later when I feel like I'm about to zone out (in fact, those with ADD have a highly inconsistent learning rate). I also have an Asperger's Syndrome diagnosis, although I've recently tried to stop using it as an excuse for my behavior (in fact, I now only fit the bare minimum of "Aspie" criteria on the DSM IV, but I still think that it strongly influences my interests and behavior). I also consistently think of what's most rational - which means that I have to respect the desires that evolution has given me. Sometimes, peopl...
Spring 1912: A New Heaven And A New Earth 
by Scott Alexander — 2010-11-13T17:11:53.609Z

And so it came to pass that on Christmas Day 1911, the three Great Powers of Europe signed a treaty to divide the continent between them peacefully, ending what future historians would call the Great War.
The sun truly never sets on King Jack's British Empire, which stretches from Spain to Stockholm, from Casablanca to Copenhagen, from the fringes of the Sahara to the coast of the Arctic Ocean. They rule fourteen major world capitals, and innumerable smaller towns and cities, the greatest power of the age and the unquestioned master of Western Europe.
From the steppes of Siberia to the minarets of Istanbul, the Ottoman Empire is no longer the Sick Man of Europe but stands healthy and renewed, a colossus every bit the equal of the Christian powers to its west. Its Sultan calls himself the Caliph, for the entire Islamic world basks in his glory, and his Grand Vizier has been rewarded with a reputation as one of the most brilliant and devious politicians of the age. At his feet grovel representatives of twelve great cities, and even far-flung Tunis has not escaped his sway.
And in between, the Austro-Hungarian Empire straddles the Alps and ancient Italy. Its lack of natural borders presented no difficulty for its wily Emperor, who successfully staved off the surrounding powers and played his enemies off against one another while building alliances that stood the test of time. Eight great cities pay homage to his double-crown, and he is what his predecessors could only dream of being - a true Holy Roman Emperor.
And hidden beneath the tricolor map every student learns in grammar school are echoes of subtler hues. In Germany, people still talk of the mighty Kajser Sotala I, who conquered the ancient French enemy and extended German rule all the way to the Mediterranean, and they still seeth and curse at his dastardly betrayal by his English friends. In Russia, Princess Anastasia claims to be the daughter of Czar Perplexed, and recounts to everyone who will listen the story of her stoic father, who remained brave until the very end; at her side travels a strange bearded man who many say looks like Rasputin, the Czar's long-missing adviser. The French remember President Andreassen, who held off the combined armies of England and Germany for half a decade, and many still go on pilgrimage to Liverpool, the site of their last great victory. And in Italy, Duke Carinthium h...
Requesting some advice on a question
by Carinthium — 2010-11-13T06:07:54.256Z

I'm not sure if this is the right place to put this, but there's a problem I can tell I'm biased on so I'm requesting some advice from people unlikely to be biased on it.

Legally, circa the time of the actual secessions leading to the creation of the C.S.A, was secession legal? As far as I can tell it was (the Supreme Court may have practical control but they aren't infallible, and because of the Tenth Amendment), but I can tell I have biased emotions on the subject so I'm checking with people less likely to.
Diplomacy as a Game Theory Laboratory
by Scott Alexander — 2010-11-12T22:19:50.897Z

Game theory. You've studied the posts, you've laughed at the comics, you've heard the music1. But the best way to make it Truly Part Of You is to play a genuine game, and I have yet to find any more effective than Diplomacy.

Diplomacy is a board game for seven people played on a map of WWI Europe. The goal is to capture as many strategic provinces ("supply centers") as possible; eighteen are needed to win. But each player's country starts off with the same sized army, and there is no luck or opportunity for especially clever tactics. The most common way to defeat an enemy is to form coalitions with other players. But your enemies will also be trying to form coalitions, and the most profitable move is often to be a "double agent", stringing both countries along as long as you can. All game moves are written in secret and revealed at the same time and there are no enforcement mechanisms, so alliances, despite their central importance, aren't always worth the paper they're printed on.

The conditions of Diplomacy - competition for scarce resources, rational self-interested actors, importance of coalitions, lack of external enforcement mechanisms - mirror the conditions of game theoretic situations like the Prisoner's Dilemma (and the conditions of most of human evolution!) and so make a surprisingly powerful laboratory for analyzing concepts like trust, friendship, government, and even religion.

Over the past few months, I've played two online games of Diplomacy. One I won through a particularly interesting method; the other I lost quite badly, but with an unusual consolation. This post is based on notes I took during the games about relevant game theoretic situations. You don't need to know the rules of Diplomacy to understand the post, but if you want a look you can find them here.

Study One: The Prisoner's Dilemma

The Prisoner's Dilemma is a classic case in game theory in which two players must decide whether or not to cooperate for a common goal. If both players cooperate, they both do better than if both defect, but one player can win big by defecting when the other cooperates. This situation is at the heart of Diplomacy.

Germany and France have agreed to ally against Britain. Both countries have demilitarized their mutual border, and are concentrating all of their forces to the north, where they take province after prov...
Rational responses to potential home invasion threat?
by AdeleneDawner — 2010-11-12T13:54:32.020Z

About three hours ago - in the very early morning, pre-dawn - someone knocked on my window to get my attention and made a lewd proposition. Less than two weeks ago, someone - probably the same person; definitely someone with a similar voice and build - woke me up by whispering 'open the door' until I looked out the window. I am, needless to say, not amused.

Since the first incident, I've been leaving my porch light on, and I've had a webcam sitting prominently in the window. The webcam has been commented on by both of the people who know me and would have an opportunity to do so, so I expected that it would be a reasonable deterrent, but apparently this guy is very stupid, very desperate, or both.

I called the police both times, and they responded promptly, but didn't see anyone walking around near my apartment. This leads me to believe that I'm being harassed by a nearby neighbor.

The webcam was not on during the second incident, but it will be on nightly from now on. I also intend to add a light in the window near my bed - I didn't get a good look at the guy, even though he was right there and not making any apparent attempt to hide, because he was between me and the porch light.

I'd appreciate any other practical suggestions that anyone might have, bearing in mind that I'm in an apartment and can't make many changes to the building itself. Also, I was already working on buying a house before the first incident even happened, so suggestions that I move aren't useful - I'm already working on that, thank goodness.

(The chances of me having trouble with this individual in any situation other than a home invasion seem pretty small - I don't leave the house often, and not on a regular schedule at all, plus I don't drive so I generally have a friend in a car watching me to and from the door, so the usually more risky situation of getting in and out of the house isn't an issue for the most part. I will be extra-careful about getting my mail and thoughtful about when I leave to do my laundry.)
Outreach opportunity
by Richard_Kennaway — 2010-11-12T11:07:29.504Z

Ars Technica are holding a competition for people to make a science video up to 3 minutes long "to explain a scientific concept in terms that a high school science class would not only understand, but actually be interested in watching". Prizes in three categories: biology, physics, and mathematics. Deadline is December 25. More details here.

Anyone want to have a go at Bayes' theorem? Cognitive bias? Defeating death? Invisible purple dragons?
Outreach opportunity
by Richard_Kennaway — 2010-11-12T11:04:14.087Z

Ars Technica are holding a competition for people to make a science video up to 3 minutes long "to explain a scientific concept in terms that a high school science class would not only understand, but actually be interested in watching". Deadline is December 25. More details here.

Anyone want to have a go at Bayes' theorem? Cognitive bias? Defeating death? Invisible purple dragons?
Study shows existence of psychic powers.
by bentarm — 2010-11-12T01:46:03.970Z

According to the New Scientist, Daryl Bern has a paper to appear in Journal of Personality and Social Psychology, which claims that the participants in psychological experiments are able to predict the future. A preprint of this paper is available online. Here's a quote from the New Scientist article:

In one experiment, students were shown a list of words and then asked to recall words from it, after which they were told to type words that were randomly selected from the same list. Spookily, the students were better at recalling words that they would later type.

In another study, Bem adapted research on "priming" – the effect of a subliminally presented word on a person's response to an image. For instance, if someone is momentarily flashed the word "ugly", it will take them longer to decide that a picture of a kitten is pleasant than if "beautiful" had been flashed. Running the experiment back-to-front, Bem found that the priming effect seemed to work backwards in time as well as forwards.

Question: even assuming the methodology is sound, given experimenter bias, publication bias and your priors on the existence of psi, what sort of p-values would you need to see in that paper in order to believe with, say, 50% probability that the effect measured is real?
If a tree falls on Sleeping Beauty...
by ata — 2010-11-12T01:14:26.076Z

Several months ago, we had an interesting discussion about the Sleeping Beauty problem, which runs as follows:

Sleeping Beauty volunteers to undergo the following experiment. On Sunday she is given a drug that sends her to sleep. A fair coin is then tossed just once in the course of the experiment to determine which experimental procedure is undertaken. If the coin comes up heads, Beauty is awakened and interviewed on Monday, and then the experiment ends. If the coin comes up tails, she is awakened and interviewed on Monday, given a second dose of the sleeping drug, and awakened and interviewed again on Tuesday. The experiment then ends on Tuesday, without flipping the coin again. The sleeping drug induces a mild amnesia, so that she cannot remember any previous awakenings during the course of the experiment (if any). During the experiment, she has no access to anything that would give a clue as to the day of the week. However, she knows all the details of the experiment.

Each interview consists of one question, “What is your credence now for the proposition that our coin landed heads?”

In the end, the fact that there were so many reasonable-sounding arguments for both sides, and so much disagreement about a simple-sounding problem among above-average rationalists, should have set off major alarm bells. Yet only a few people pointed this out; most commenters, including me, followed the silly strategy of trying to answer the question, and I did so even after I noticed that my intuition could see both answers as being right depending on which way I looked at it, which in retrospect would have been a perfect time to say “I notice that I am confused” and backtrack a bit…

And on reflection, considering my confusion rather than trying to consider the question on its own terms, it seems to me that the problem (as it’s normally stated) is completely a tree-falling-in-the-forest problem: a debate about the normatively “correct” degree of credence which only seemed like an issue because any conclusions about what Sleeping Beauty “should” believe weren’t paying their rent, were disconnected from any expectation of feedback from reality about how right they were.

It may seem either implausible or alarming that as fundamental a concept as probability can be the subject of such debates, but remember that th...
The Aspirin Paradox- replacement for the Smoking Lesion Problem?
by orthonormal — 2010-11-11T23:57:30.174Z

It's been pointed out that the Smoking Lesion problem is a poorly chosen decision theory problem, because in the real world there actually is a direct causal link from smoking to cancer, and people's intuitions are influenced more by that than by the stated parameters of the scenario. In his TDT document, Eliezer concocts a different artificial example (chewing gum and throat abcesses). I recently noticed, though, a potentially good real-world example of the same dynamic: the Aspirin Paradox.

Despite the effectiveness of aspirin in preventing heart attacks, those who regularly take aspirin are at a higher risk of a second heart attack, because those with symptoms of heart disease are more likely than those without symptoms to be taking aspirin regularly. While it turns out this "risk factor" is mostly screened off by other measurable health factors, it's a valid enough correlation for the purposes of decision theory.
SIA and the Two Dimensional Doomsday Argument
by KatjaGrace — 2010-11-11T22:00:17.000Z

This post might be technical. Try reading this if I haven’t explained everything well enough.

When the Self Sampling Assumption (SSA) is applied to the Great Filter it gives something pretty similar to the Doomsday Argument, which is what it gives without any filter. SIA gets around the original Doomsday Argument. So why can’t it get around the Doomsday Argument in the Great Filter?

The Self Sampling Assumption (SSA) says you are more likely to be in possible worlds which contain larger ratios of people you might be vs. people know you are not*.

If you have a silly hat, SSA says you are more likely to be in world 2 - assuming Worlds 1 and 2 are equally likely to exist (i.e. you haven't looked aside at your companions), and your reference class is people.

The Doomsday Argument uses the Self Sampling Assumption. Briefly, it argues that if there are many generations more humans, the ratio of people who might be you (are born at the same time as you) to people you can’t be (everyone else) will be smaller than it would be if there are few future generations of humans. Thus few generations is more likely than previously estimated.

An unusually large ratio of people in your situation can be achieved by a possible world having unusually few people unlike you in it or unusually many people like you, or any combination of these.

Fewer people who can't be me or more people who may be me make a possible world more likely according to SSA.

For instance on the horizontal dimension, you can compare a set of worlds which all have the same number of people like you, and different numbers of people you are not. The world with few people unlike you has the largest increase in probability.

The top row from the previous diagram. The Doomsday Argument uses possible worlds varying in this dimension only.

The Doomsday Argument is an instance of variation in the horizontal dimension only. In every world there is one person with your birth rank, but the numbers of people with future birth ranks differ.

At the other end of the spectrum you could be comparing worlds with the same number of future people and vary the number of current people, as long as you are ignorant of how many current people there are.

The vertical axis. The number of people in your situation changes, while the number of others stays the same. The world with a lot of people l

...
The Strong Occam's Razor
by cousin_it — 2010-11-11T17:28:21.338Z

This post is a summary of the different positions expressed in the comments to my previous post and elsewhere on LW. The central issue turned out to be assigning "probabilities" to individual theories within an equivalence class of theories that yield identical predictions. Presumably we must prefer shorter theories to their longer versions even when they are equivalent. For example, is "physics as we know it" more probable than "Odin created physics as we know it"? Is the Hamiltonian formulation of classical mechanics apriori more probable than the Lagrangian formulation? Is the definition of reals via Dedekind cuts "truer" than the definition via binary expansions? And are these all really the same question in disguise?

One attractive answer, given by shokwave, says that our intuitive concept of "complexity penalty" for theories is really an incomplete formalization of "conjunction penalty". Theories that require additional premises are less likely to be true, according to the eternal laws of probability. Adding premises like "Odin created everything" makes a theory less probable and also happens to make it longer; this is the entire reason why we intuitively agree with Occam's Razor in penalizing longer theories. Unfortunately, this answer seems to be based on a concept of "truth" granted from above - but what do differing degrees of truth actually mean, when two theories make exactly the same predictions?

Another intriguing answer came from JGWeissman. Apparently, as we learn new physics, we tend to discard inconvenient versions of old formalisms. So electromagnetic potentials turn out to be "more true" than electromagnetic fields because they carry over to quantum mechanics much better. I like this answer because it seems to be very well-informed! But what shall we do after we discover all of physics, and still have multiple equivalent formalisms - do we have any reason to believe simplicity will still work as a deciding factor? And the question remains, which definition of real numbers is "correct" after all?

Eliezer, bless him, decided to take a more naive view. He merely pointed out that our intuitive concept of "truth" does seem to distinguish between "physics" and "God created physics", so if our current formalization of "truth" fails to tell them apart, the flaw lies with the formalism rather than with us. I have a lot of sympathy for this ...