trentmkelly/LessWrong-43k huggingface dataset:
Better thinking through experiential games
by Morendil — 2009-10-23T12:59:12.901Z

A few years ago I came across The Logic of Failure by Dietrich Doerner (previously mentioned on LW) which discusses cognitive failures in people dealing with "complex situations".
One section (p.1 28) discusses a little simulation game, where participants are told to "steer" the temperature of a refrigerated storeroom with a defective thermostat, the exact equation governing how the thermostat setting affects actual temperature being unknown. Players control a dial with settings numbered 0 through 100, and can read actual temperature off a thermometer display. The only complications in this task are a) that there is a delay between changing the dial and the effects of the new setting; b) the possibility of "overshoot".
I found the section's title chilling as well as fascinating: "Twenty-eight is a good number." Doerner says this statement is typical of what participants faced with this type of situation tend to say. People don't just make ineffective use of the data they are presented with: they make up magical hypotheses, cling to superstitions or even call into question the very basis of the experiment, that there is a systematic link between thermostat setting and temperature.
Reading about it is one thing, and actually playing the game quite another, so I got a group of colleagues together and we gave it a try. We were all involved in one way or another with managing software projects, which are systems way more complex than the simple thermostat system; our interest was to confirm Doerner's hypothesis that humans are generally inept at even simple management tasks. By the reports of all involved it was one of the most effective learning experiences they'd had. Since then, I have had a particular interest in this type of situation, which I have learned is sometimes called "experiential learning".
As I conceive of it, experiential learning consists of setting up a problematic situation, in such a way that the students ("players") should rely on their own wits to explore the situation, invent ways of dealing with it (sometimes by incorporating conceptual tools provided by an instructor), and test their newfound insights against the original problem - or against real-world situations. My preferred setting for experiential learning is a small-group format, with individual or group interaction with the situation, and group discussion for the debrief.
In experienti...
The continued misuse of the Prisoner's Dilemma
by SilasBarta — 2009-10-23T03:48:08.860Z

Related to: The True Prisoner's Dilemma, Newcomb's Problem and Regret of Rationality

In The True Prisoner's Dilemma, Eliezer Yudkowsky pointed out a critical problem with the way the Prisoner's Dilemma is taught: the distinction between utility and avoided-jail-time is not made clear. The payoff matrix is supposed to represent the former, even as its numerical values happen to coincidentally match the latter. And worse, people don't naturally assign utility as per the standard payoff matrix: their compassion for the friend in the "accomplice" role means they wouldn't feel quite so good about a "successful" backstabbing, nor quite so bad about being backstabbed. ("Hey, at least I didn't rat out a friend.")

For that reason, you rarely encounter a true Prisoner's Dilemma, even an iterated one. The above complications prevent real-world payoff matrices from working out that way.

Which brings us to another unfortunate example of this misunderstanding being taught.

Recently, on the New York Times's "Freakonomics" blog, Professor Daniel Hamermesh gleefully recounts a recent experiment he performed (which he says he does often) on students in his intro economics course, which is basically the same as the Prisoner's Dilemma (henceforth, PD).

Now, before going further, let me make clear that Hamermesh is no small player. Just take a look at all the accolades and accomplishments listed on his Wikipedia page or his university page CV. So, this is a teaching of a professor at the top of his field, so it's only with hesitation that I proceed further to allege that he's Doing It Wrong

Hamermesh's variant of the PD is to pick eight students and auction off a $20 bill to them, with the money split evenly across the winners if there are multiple highest bids. Here, cooperation corresponds to adhering to a conspiracy where everyone agrees to make the same low bid and thus a big profit. Defecting corresponds to breaking the agreement and making a slightly higher bid so you can take everything for yourself. If the others continue to cooperate, their bid is lower and they get nothing.

Here is how Hamermesh describes the result (italics mine, bold in the original):

Today seven of the students stuck to the collusive agreement, and each bid $.01. They figured they would split the $20 eight ways, netting $2.49 each. Ashley, bless her he

...
Generous people cross the street before the beggar
by KatjaGrace — 2009-10-22T17:22:54.000Z

Robert Wiblin points to a study showing that the most generous people are the most keen to avoid situations where they will be generous, even though the people they would have helped will go without.

We conduct an experiment to demonstrate the importance of sorting in the context of social preferences. When individuals are constrained to play a dictator game, 74% of the subjects share. But when subjects are allowed to avoid the situation altogether, less than one third share. This reversal of proportions illustrates that the influence of sorting limits the generalizability of experimental findings that do not allow sorting. Moreover, institutions designed to entice pro-social behavior may induce adverse selection. We find that increased payoffs prevent foremost those subjects from opting out who share the least initially. Thus the impact of social preferences remains much lower than in a mandatory dictator game, even if sharing is subsidized by higher payoffs…

A big example of generosity inducing institutions causing adverse selection is market transactions with poor people.

For some reason we hold those who trade with another party responsible for that party’s welfare. We blame a company for not providing its workers with more, but don’t blame other companies for lack of charity to the same workers. This means that you can avoid responsibility to be generous by not trading with poor people.

Many consumers feel that if they are going to trade with poor people they should buy fair trade or thoroughly research the supplier’s niceness. However they don’t have the money or time for those, so instead just avoid buying from poor people. Only the less ethical remain to contribute to the purses of the poor.

Probably the kindest girl in my high school said to me once that she didn’t want a job where she would get rich because there are so many poor people in the world. I said that she should be rich and give the money to the poor people then. Nobody was wowed by this idea. I suspect something similar happens often with people making business and employment decisions. Those who have qualms about a line of business such as trade with poor people tend not to go into that, but opt for something guilt free already, while the less concerned do the jobs where compassion might help.
Rationality Quotes: October 2009
by Eliezer Yudkowsky — 2009-10-22T16:06:42.037Z

A monthly thread for posting rationality-related quotes you've seen recently (or had stored in your quotesfile for ages).

Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then fine, post them together.)

Do not quote yourself.

Do not quote comments/posts on LW/OB.

No more than 5 quotes per person per monthly thread, please.
Biking Beyond Madness (link)
by hirvinen — 2009-10-22T15:16:39.346Z

‘‘During race, I am going crazy, definitely,’’ he says, smiling in bemused despair. ‘‘I cannot explain why is that, but it is true.’’
The craziness is methodical, however, and Robic and his crew know its pattern by heart. Around Day 2 of a typical weeklong race, his speech goes staccato. By Day 3, he is belligerent and sometimes paranoid. His short-term memory vanishes, and he weeps uncontrollably. The last days are marked by hallucinations: bears, wolves and aliens prowl the roadside; asphalt cracks rearrange themselves into coded messages. Occasionally, Robic leaps from his bike to square off with shadowy figures that turn out to be mailboxes. In a 2004 race, he turned to see himself pursued by a howling band of black-bearded men on horseback.
‘‘Mujahedeen, shooting at me,’’ he explains. ‘‘So I ride faster.’’

This 2006 New York Times story is about Jure Robic, a Slovenian ultra long distance bicycler who goes seriously insane when he pushes himself far enough during the races. At the point he feels like dying out of fatigue he still has a major portion (estimated 50 % by his team) of his strength left. So he hands over control to his team and with their help, pushes himself into the realm of insanity and gives up control to the team:

For Robic, his support crew serves as a second brain, consisting of a well-drilled cadre of a half-dozen fellow Slovene soldiers. It resembles other crews in that it feeds, hydrates, guides and motivates — but with an important distinction. The second brain, not Robic’s, is in charge.

‘‘By the third day, we are Jure’s software,’’ says Lt. Miran Stanovnik, Robic’s crew chief. ‘‘He is the hardware, going down the road.’’

His success isn't because of exceptional physiology or training:

On rare occasions when he permits himself to be tested in a laboratory, his ability to produce power and transport oxygen ranks on a par with those of many other ultra-endurance athletes. He wins for the most fundamental of reasons: he refuses to stop.

The whole thing is an intriguing example of making an extraordinary, desperate effort by knowing that even when his body and brain scream for him to stop, he can go further, and doing so. Also, pushing ...
Trust in the adoration of strangers
by KatjaGrace — 2009-10-21T16:33:04.000Z

Attractive people are more trusting when they think they can be seen:

Here, we tested the effects of cues of observation on trusting behavior in a two-player Trust game and the extent to which these effects are qualified by participants’ own attractiveness. Although explicit cues of being observed (i.e., when participants were informed that the other player would see their face) tended to increase trusting behavior, this effect was qualified by the participants’ other-rated attractiveness (estimated from third-party ratings of face photographs). Participants’ own physical attractiveness was positively correlated with the extent to which they trusted others more when they believed they could be seen than when they believed they could not be seen. This interaction between cues of observation and own attractiveness suggests context dependence of trusting behavior that is sensitive to whether and how others react to one’s physical appearance.

Probably rightly so. It’s interesting that people do not get used to the average level of good treatment expected for their attractiveness, but are sensitive to the difference in treatment when visible and when not. Is it inbuilt that we should expect some difference there, or is it just very noticeable?

I wonder whether widespread beauty enhancement increases overall trust in society, and enhances productivity accordingly, or whether favorable treatment and returned trust both adapt to relative position. Does advertising suggesting that the world is chock full of model material decrease trust between real people?
Why the beliefs/values dichotomy?
by Wei Dai — 2009-10-20T16:35:52.427Z

I'd like to suggest that the fact that human preferences can be decomposed into beliefs and values is one that deserves greater scrutiny and explanation. It seems intuitively obvious to us that rational preferences must decompose like that (even if not exactly into a probability distribution and a utility function), but it’s less obvious why.

The importance of this question comes from our tendency to see beliefs as being more objective than values. We think that beliefs, but not values, can be right or wrong, or at least that the notion of right and wrong applies to a greater degree to beliefs than to values. One dramatic illustration of this is in Eliezer Yudkowsky’s proposal of Coherent Extrapolated Volition, where an AI extrapolates the preferences of an ideal humanity, in part by replacing their "wrong” beliefs with “right” ones. On the other hand, the AI treats their values with much more respect.

Since beliefs and values seem to correspond roughly to the probability distribution and the utility function in expected utility theory, and expected utility theory is convenient to work with due to its mathematical simplicity and the fact that it’s been the subject of extensive studies, it seems useful as a first step to transform the question into “why can human decision making be approximated as expected utility maximization?”

I can see at least two parts to this question:

Why this mathematical structure?

Why this representation of the mathematical structure?

Not knowing how to answer these questions yet, I’ll just write a bit more about why I find them puzzling.

Why this mathematical structure?

It’s well know that expected utility maximization can be derived from a number of different sets of assumptions (the so called axioms of rationality) but they all include the assumption of Independence in some form. Informally, Independence says that what you prefer to happen in one possible world doesn’t depend on what you think happens in other possible worlds. In other words, if you prefer A&C to B&C, then you must prefer A&D to B&

...
Everyone else prefers laws to values
by KatjaGrace — 2009-10-20T11:16:47.000Z

How do you tell what a superhuman AI's values are? ( picture: ittybittiesforyou - see bottom)

Robin Hanson says that it is more important to have laws than shared values. I agree with him when ‘shared values’ means that shared indexical values remain about different people, e.g. If you and I share a high value of orgasms, you value you having orgasms and I value me having orgasms. Unless we are dating it’s all the same to me if you prefer croquet to orgasms. I think the singularitarians aren’t talking about this though. They want to share values in such a way that AI wants them to have orgasms. In principle this would be far better than having different values and trading. Compare gains from trading with the world economy to gains from the world economy’s most heartfelt wish being to please you. However I think that laws will get far more attention than values overall in arranging for an agreeable robot transition, and rightly so. Let me explain, then show you how this is similar to some more familiar situations.

Greater intelligences are unpredictable

If you know exactly what a creature will do in any given situation before it does it, you are at least as smart as it (if we don’t include it’s physical power as intelligence). Greater intelligences are inherently unpredictable. If you know the intelligence is trying to do, then you know what kind of outcome to expect, but guessing how it will get there is harder. This should be less so for lesser intelligences, and more so for more different intelligences. I will have less trouble guessing what a ten year old will do in chess against me than a grand master, though I can guess the outcome in both cases. If I play someone with a significantly different way of thinking about the game they may also be hard to guess.

Unpredictability is dangerous

This unpredictability is a big part of the fear of a superhuman AI. If you don’t know what path an intelligence will take to the goal you set it, you don’t know whether it will affect other things that you care about. This problem is most vividly illustrated by the much discussed case where the AI in question is suddenly very many orders of magnitude smarter than a human. Imagine we initially gave it only a subset of our values, such as our yearning to figure out whether P = NP, and we assume that it won’t influence anything outside its box. It might determine that t...
Shortness is now a treatable condition
by taw — 2009-10-20T01:13:07.428Z

There was some talk here about height taxes, but there's a better solution - redefine shortness as a treatable condition and use HGH to cure it. They even got FDA on board with that, at least for 1.2% shortest people.

Unsatisfactory sexual performance became a treatable condition with Viagra. Depression and hyperactivity became treatable conditions with SSRIs. Being ugly is already almost considered a treatable condition, at least one can get that impression from cosmetic surgery ads. Being overweight is universally considered an illness, even though we don't have too many effective treatment options (surgery is unpopular, and effective drugs like fen-phen and ECA are not officially prescribed any more). If we ever figure out how to increase IQ, you can be certain low IQ will be considered a treatable condition too. Almost everything undesirable gets redefined as an illness as soon as an effective way to fix it is developed.

I welcome these changes. Yes, redefining large parts of normal human variability as illness is a lie, but if that's what society needs to work around its taboos against human enhancement, so be it.
What do trust and sharing do to reputations?
by KatjaGrace — 2009-10-19T17:30:12.000Z

Bryan Caplan asked, ‘when doesn’t reputation work well?‘

He answers,

To me, venereal disease is the most striking response. Unlike other disease, V.D. is simple to prevent: Only have sex with people who credibly show that they aren’t infected. How hard is that? But according to Wikipedia, AIDS alone kills over 2 million people per year.

He suggests this is caused by a demand problem (people are strangely willing to sleep with someone without evidence of their not having VDs) and a supply problem (people who have good reputations can’t take over the whole market), and asks whether there are other areas where reputation fails.

Making good decisions about small risks far in the future while horny is probably a rare skill, but not the only reason for the demand problem I think. Asking someone to credibly show that they aren’t infected credibly shows that you don’t trust them to tell you on their own. Trust is a handy thing to have the appearance of in relationships, but unfortunately requires behaving trustingly. A survey of Texan girls shows 28% of them think they sometimes or never ‘have the right to’ ask their partner if he has been tested for STDs (all the questions in the survey are in terms of ‘rights’ to act certain ways, and I’m not sure what that means, but I guess it implies that asking would detriment their partner unacceptably).

Does this generalize to suggest other areas reputation doesn’t work that well? I think so. Knowing someone’s reputation allows you to trust them more. This means if you want to demonstrate that you trust someone already, something you should not do is visibly seek their reputation. Reputation should work less well then when demonstrating trust is useful and seeking information about reputation is visible.

When else is showing trust useful? Any time in relationships. Sure enough, I could assess a new boyfriend much better if I rung all his exes and got appraisals. But asking for their numbers is awkward. It would make him think I don’t trust his account of himself. Which would usually be entirely sensible of course. Out of earshot we might passionately use gossip and status cues to keep track of reputations, but if you invited your partner to seek reviews of your past behavior from others (as businesses do happily) it would be an implic...
Near and far skills
by cousin_it — 2009-10-19T13:16:44.043Z

Robin Hanson has repeatedly pointed out the difference between near and far modes of thinking, and Alicorn has repeatedly pointed out the difference between procedural and propositional knowledge. Just occurred to me that it's pretty much the same difference. I will now proceed to play some obvious riffs on the theme.

Do you ever feel yourself to be an expert on some topic just from reading about it a lot? Every young enthusiastic programmer just entering the workforce feels like that. (I certainly did.) Every wannabe entrepreneur who hangs out on Hacker News and has read enough Paul Graham essays feels they can take on the world by just applying those valuable insights. (I certainly did.) It's soooo fun to feel knowledgeable, maybe even project it outward by giving Internet advice to newbies. The public relations department of my brain doesn't seem to care that I have no actual experience: reading stuff is quite enough to change my self-image.

The problem is, excessively liking propositional knowledge over procedural is a bias that harms us every day. Though some information is directly useful, most of it is worthless. A couple days ago I had to give advice to a classmate of mine who wants to start his own "thing" but isn't sure. See, he has this theory that one should accumulate propositional knowledge until one reaches critical mass, at which point the successful venture happens by itself. Being the wise and experienced mentor that I am (hah... on my second "thing", without much success), I told him outright that his theory was bullshit. Propositional knowledge doesn't spontaneously turn into procedural.

(Digression: come to think, I'm not even sure why we need the kinds of propositional knowledge that we tend to accumulate. It reeks of a superstimulus. I know more about programming that I'll ever need for work or play, but don't remember the birthdays of all my acquaintances, which would obviously be more useful. Memorizing birthdays just isn't as exciting as reading about comonads or whatever.)

At this point I sincerely wish I had a recipe for overcoming this bias. Like pjeby, he always has a recipe. Well, I don't. Maybe perceiving the bias will turn out to be enough; maybe some kind of social software thing can help cure it on a mass scale, like meetup.com is trying to cure Bowling Alone; or maybe each of us will have to apply force, the oldskool...
Applying Double Standards to ‘‘Divisive’’ Ideas
by PlaidX — 2009-10-19T12:36:30.730Z

This is a commentary by Linda Gottfredson on a paper by Hunt and Carlson about a paper by Richard Nisbett regarding studies done by Arthur Jensen. It's ultimately about race and intelligence, but it seemed meta enough to link to here.

Warning: PDF

Applying Double Standards to ‘‘Divisive’’ Ideas