trentmkelly/LessWrong-43k huggingface dataset:
So You Think You're a Bayesian? The Natural Mode of Probabilistic Reasoning
by Matt_Simpson — 2010-07-14T16:51:40.939Z

Related to: The Conjunction Fallacy, Conjunction Controversy

The heuristics and biases research program in psychology has discovered many different ways that humans fail to reason correctly under uncertainty. In experiment after experiment, they show that we use heuristics to approximate probabilities rather than making the appropriate calculation, and that these heuristics are systematically biased. However, a tweak in the experiment protocols seems to remove the biases altogether and shed doubt on whether we are actually using heuristics. Instead, it appears that the errors are simply an artifact of how our brains internally store information about uncertainty. Theoretical considerations support this view.

EDIT: The view presented here is controversial in the heuristics and biases literature; see Unnamed's comment on this post below.

EDIT 2: The author no longer holds the views presented in this post. See this comment.

A common example of the failure of humans to reason correctly under uncertainty is the conjunction fallacy. Consider the following question:

Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.

What is the probability that Linda is:

(a) a bank teller

(b) a bank teller and active in the feminist movement

In a replication by Gigerenzer, 91% of subjects rank (b) as more probable than (a), saying that it is more likely that Linda is active in the feminist movement AND a bank teller than that Linda is simply a bank teller (1993). The conjunction rule of probability states that the probability of two things being true is less than or equal to the probability of one of those things being true. Formally, P(A & B) ≤ P(A). So this experiment shows that people violate the conjunction rule, and thus fail to reason correctly under uncertainty. The representative heuristic has been proposed as an explanation for this phenomenon. To use this heuristic, you evaluate the probability of a hypothesis by comparing how "alike" it is to the data. Someone using the representative heuristic looks at the Linda question and sees that Linda's characteristics resemble those of a feminist bank teller much more closely than that of just a bank teller, and so they conclude ...
The Instrumental Value of Your Own Time
by Mass_Driver — 2010-07-14T07:57:21.408Z

What is your time worth? Economists generally assume that your time is accurately valued by the market, because time can be converted into money (via wage labor) and back again (by hiring people to, e.g, do your chores for you). In this article, I argue that the economists are wrong, and that, as a result, we have some important questions to ask ourselves about the value of time:

(1) How would a rational person measure the value of an hour of her time?

(2) Can time invested now be meaningfully traded off against time available later?

(3) How much of your time should be spent assessing whether you are spending your time well?

Executive Summary (tl;dr)

The economists are wrong because, on the supply side, it's difficult to supply extra paid labor on a timescale of weeks or months; and, on the demand side, there are practical and aesthetic limits to how much extra time you can buy for yourself. Thus, a rational person will not assume that her time is worth what the market says it is worth; your time is usually more valuable (to you) than it is to the market, because there is a market failure for medium-sized chunks of your time. One way to get a sense of what your time is worth to you is to measure your ability to achieve goals or satisfy desires, but not all time is created equal -- an hour that you spend programming code may be implicitly supported by several hours of sleeping, eating, desk-cleaning, and other homeostatic tasks. Attempting to normalize the value of your time against a hypothetical "maximum productivity" level might yield more accurate information than simply checking your brain's cached value for how satisfying an experience was in hindsight, because memory focuses on events associated with intense emotions and thus often fails to adequately account for the cost of wasted time. People who spend essentially no time computing their ability to achieve their ultimate goals are probably wasting a lot of time, but the cost (in time) of evaluating your own productivity is computationally intractable; ignorance about ignorance is recursive and cannot easily be eliminated from complex chaotic systems like your life.

When Time Isn't Money

Basic models of labor economics tend to treat a person's decision about how many hours to work each week as a scalar variable; Bob could work 39 hours this week, or 40 hours, or 50.3 hours, and would be ...
Room for rent in North Berkeley house
by Kevin — 2010-07-13T20:28:20.155Z

Hi Less Wrong. I am moving into a 5 bedroom house in North Berkeley with Mike Blume and Emil Gilliam. We have an extra bedroom available.

It's located in the Gourmet Ghetto neighborhood (because we can afford to eat at Chez Pannise when we aren't busy saving the world, right? I didn't think so) and is about 1/2 mile from the Downtown Berkeley and North Berkeley BART stations. From Downtown Berkeley to Downtown SF via BART, it is a painless 25 minute commute. The bedroom is unfurnished and available right now. Someone willing to commit to living there for one year is preferred, but willing to consider six month or month to month leases.

I'm open to living with a wide range of people and tend to be extremely tolerant of people's quirks. I am not tolerant to drama, so I am open to living with anyone that will not bring any sort of unneeded conflict to my living space.

~$750/month+utilities. Easy street parking available.

Feel free to ask questions via email (kfischer @# gmail %# com) or in the comments here.

And before any of you pedants downvote me because "Less Wrong is not Craigslist", this is kind of like a year long Less Wrong meetup.
Some Thoughts Are Too Dangerous For Brains to Think
by WrongBot — 2010-07-13T04:44:12.287Z

[EDIT - While I still support the general premise argued for in this post, the examples provided were fairly terrible. I won't delete this post because the comments contain some interesting and valuable discussions, but please bear in mind that this is not even close to the most convincing argument for my point.]

A great deal of the theory involved in improving computer and network security involves the definition and creation of "trusted systems", pieces of hardware or software that can be relied upon because the input they receive is entirely under the control of the user. (In some cases, this may instead be the system administrator, manufacturer, programmer, or any other single entity with an interest in the system.) The only way to protect a system from being compromised by untrusted input is to ensure that no possible input can cause harm, which requires either a robust filtering system or strict limits on what kinds of input are accepted: a blacklist or a whitelist, roughly.

One of the downsides of having a brain designed by a blind idiot is that said idiot hasn’t done a terribly good job with limiting input or anything resembling “robust filtering”. Hence that whole bias thing. A consequence of this is that your brain is not a trusted system, which itself has consequences that go much, much deeper than a bunch of misapplied heuristics. (And those are bad enough on their own!)

In discussions of the AI-Box Experiment I’ve seen, there has been plenty of outrage, dismay, and incredulity directed towards the underlying claim: that a sufficiently intelligent being can hack a human via a text-only channel. But whether or not this is the case (and it seems to be likely), the vulnerability is trivial in the face of a machine that is completely integrated with your consciousness and can manipulate it, at will, towards its own ends and without your awareness.

Your brain cannot be trusted. It is not safe. You must be careful with what you put into it, because it will decide the output, not you. We have been warned, here on Less Wrong, that there is dangerous knowledge; Eliezer has told us that knowing about biases can cause us harm. Nick Bostrom has written a paper describing dozens of ways in which information can hurt us, but he missed (at least) one.

The acquisi

...
MIRI Call for Volunteers
by alyssavance — 2010-07-12T23:49:57.918Z

Are you interested in reducing existential risk? Are you a student who wants to donate to existential risk reduction, but doesn't have any money? Are you a past or present Visiting Fellow applicant? Do you want to apply to the Visiting Fellows program, but can't take time off work or school? If the answer to any one of these questions is yes, you should join the Singularity Institute Volunteer Program. The Singularity Institute is looking for volunteers to do things like:
- Review and proofread MIRI publications.
- Promote MIRI sites like singinst.org and lesswrong.com.
- Contribute content to the MIRI Blog.
- Create online videos or other digital content.
- Volunteer for the 2010 Singularity Summit.
- Organize monthly dinner parties to cultivate new supporters.
- Translate MIRI webpages into other languages, e.g. French, German, Japanese, Mandarin, Spanish, etc.
- Contribute to the collaborative rationality blog Less Wrong.
- Host a Less Wrong meetup, or remind organizers to host them.
Requirements for volunteers are fairly minimal, but you must be able to:
- Read and write English on a basic or higher level
- Complete tasks reliably with minimal supervision
- Stick to deadlines, and let us know if you can't meet them
Additional skills, like programming, ability to write well, foreign languages, math talent, etc. are a definite plus. If you are interested, please shoot us an email with a brief summary of who you are, what your interests and skills are and how you'd like to help.
If you want to contribute, but don't know how you can help, please email MIRI Volunteer Coordinator Louie Helm at louie@intelligence.org.
Apply today!
The President's Council of Advisors on Science and Technology is soliciting ideas
by Kevin — 2010-07-12T23:41:04.746Z

The question that the ideas are supposed to be in response to is:

What are the critical infrastructures that only government can help provide that are needed to enable creation of new biotechnology, nanotechnology, and information technology products and innovations -- a technological congruence that we have been calling the “Golden Triangle" -- that will lead to new jobs and greater GDP?"

Here are links to some proposed ideas that you should vote for, assuming you agree with them. You do have to register to vote, but the email confirmation arrives right away and it shouldn't take much more than two minutes of your time altogether. Why should you do this? The top voted ideas from this request for ideas will be seen by some of the top policy recommendation makers in the USA. They probably won't do anything like immediately convene a presidential panel on AGI, but we are letting them know that these things are really important.

Research the primary cause of degenerative diseases: aging / biological senescence

Explore proposals for sustaining the economy despite ubiquitous automation

Establish a Permanent Panel or Program to Address Global Catastrophic Risks, Including AGI

Does anyone have any other ideas? Feel free to submit them directly to ideascale, but it may be a better idea to first post them in the comments of this post for discussion.
The red paperclip theory of status
by Morendil — 2010-07-12T23:08:57.088Z

Followup to: The Many Faces of Status (This post co-authored by Morendil and Kaj Sotala - see note at end of post.)

In brief: status is a measure of general purpose optimization power in complex social domains, mediated by "power conversions" or "status conversions".

What is status?

Kaj previously proposed a definition of status as "the ability to control (or influence) the group", but several people pointed out shortcomings in that. One can influence a group without having status, or have status without having influence. As a glaring counterexample, planting a bomb is definitely a way of influencing a group's behavior, but few would consider it to be a sign of status.

But the argument of status as optimization power can be made to work with a couple of additional assumptions. By "optimization power", recall that we mean "the ability to steer the future in a preferred direction". In general, we recognize optimization power after the fact by looking at outcomes. Improbable outcomes that rank high in an agent's preferences attest to that agent's power. For the purposes of this post, we can in fact use "status" and "power" interchangeably.

In the most general sense, status is the general purpose ability to influence a group. An analogy to intelligence is useful here. A chess computer is very skilled at the domain of chess, but has no skill in any other domain. Intuitively, we feel like a chess computer is not intelligent, because it has no cross-domain intelligence. Likewise, while planting bombs is a very effective way of causing certain kinds of behavior in groups, intuitively it doesn't feel like status because it can only be effectively applied to a very narrow set of goals. In contrast, someone with high status in a social group can push the group towards a variety of different goals. We call a certain type of general purpose optimization power "intelligence", and another type of general purpose optimization power "status". Yet the ability to make excellent chess moves is still a form of intelligence, but only a very narrow one.

A power conversion framework

The framework that (provisionally) makes the most sense to us is the following, based on the idea of power conversions.

There are a great many forms of status, and a great many forms of power. (Earlier discussion hinted to the "godshatteriness" of status.) Dominance is one, but so are wealth,

...
A Rational Identity
by Kaj_Sotala — 2010-07-12T22:59:12.814Z

How facts backfire (previous discussion) discusses the phenomenon where correcting people's mistaken beliefs about political issues doesn't actually make them change their minds. In fact, telling them the truth about things might even reinforce their opinions and entrench them even firmer in their previous views. "The general idea is that it’s absolutely threatening to admit you’re wrong", says one of the researchers quoted in the article.

This should come as no surprise to the people here. But the interesting bit is that the article suggests a way to make people evaluate information in a less biased manner. They mention that one's willingness to accept contrary information is related to one's self-esteem: Nyhan worked on one study in which he showed that people who were given a self-affirmation exercise were more likely to consider new information than people who had not. In other words, if you feel good about yourself, you’ll listen — and if you feel insecure or threatened, you won’t.

I suspect that the beliefs that are the hardest to change, even if the person had generally good self-esteem, are those which are central to their identity. If someone's identity is built around capitalism being evil, or socialism being evil, then any arguments about the benefits of the opposite economical system are going to fall on deaf ears. Not only will that color their view of the world, but it's likely that they're deriving a large part of their self-esteem from that identity. Say something that challenges the assumptions built into their identity, and you're attacking their self-esteem.

Keith Stanovich tells us that simply being intelligent isn't enough to avoid bias. Intelligent people might be better at correcting for bias, but there's no strong correlation between intelligence and the disposition to actually correct for your own biases. Building on his theory, we can assume that threatening opinions will push even non-analytical people into thinking critically, but non-threatening ones won't. Stanovich believes that spreading awareness of biases might be enough to help a lot of people, and to some degree it might. But we also know about the tendency to only use your awareness of bias to attack arguments you don't like. In the same way that telling people facts about politics sometimes only polarizes opinions, telling people about biases m...
What Intelligence Tests Miss: The psychology of rational thought
by Kaj_Sotala — 2010-07-11T23:01:55.080Z

This is the fourth and final part in a mini-sequence presenting Keith E. Stanovich's excellent book What Intelligence Tests Miss: The psychology of rational thought.

If you want to give people a single book to introduce people to the themes and ideas discussed on Less Wrong, What Intelligence Tests Miss is probably the best currenty existing book for doing so. It does have a somewhat different view on the study of bias than we on LW: while Eliezer concentrated on the idea of the map and the territory and aspiring to the ideal of a perfect decision-maker, Stanovich's perspective is more akin to bias as a thing that prevents people from taking full advantage of their intelligence. Regardless, for someone less easily persuaded by LW's somewhat abstract ideals, reading Stanovich's concrete examples first and then proceeding to the Sequences is likely to make the content presented in the sequences much more interesting. Even some of our terminology such as "carving reality at the joints" and the instrumental/epistemic rationality distinction will be more familiar to somebody who was first read What Intelligence Tests Miss.

Below is a chapter-by-chapter summary of the book.

Inside George W. Bush's Mind: Hints at What IQ Tests Miss is a brief introductory chapter. It starts with the example of president George W. Bush, mentioning that the president's opponents frequently argued against his intelligence, and even his supporters implicitly conceded the point by arguing that even though he didn't have "school smarts" he did have "street smarts". Both groups were purportedly surprised when it was revealed that the president's IQ was around 120, roughly the same as his 2004 presidential candidate opponent John Kerry. Stanovich then goes on to say that this should not be surprising, for IQ tests do not tap into the tendency to actually think in an analytical manner, and that IQ had been overvalued as a concept. For instance, university admissions frequently depend on tests such as the SAT, which are pretty much pure IQ tests. The chapter ends by a disclaimer that the book is not an attempt to say that IQ tests measure nothing important, or that there would be many kinds of intelligence. IQ does measure something real and important, but that doesn't change the fact that people overvalue it and are ge...
How Irrationality Can Win: The Power of Group Cohesion
by alyssavance — 2010-07-09T06:15:23.708Z

(This post is the first in a planned sequence, discussing the ways in which irrationality can sometimes win. I write this with the hope that rationalists can see why rationality sometimes loses, and adopt "irrationalist" techniques if the techniques are actually effective.)

I'd like to begin this sequence by introducing one of the most powerful groups of people in the world- more powerful than such legendary organizations as Skull and Bones and the Freemasons. The members of this organization control wealth, power, and influence almost beyond imagining. Among their current and former members are six U.S. Presidents, four Vice Presidents, three Speakers of the House, three Supreme Court justices, dozens of Cabinet members, and well over a hundred Congressmen. This group also has vast economic power, founding and leading dozens of billion-dollar companies. No fewer than 86 members have been university presidents, of universities including Berkeley, U. Chicago, Dartmouth, MIT, and Yale. There are members of this organization in virtually every Fortune 500 company. And yet, the entire group is no bigger in size than a smallish US city. This organization definitely knows how to win. What is it? Is it the Bilderbergers? The Council on Foreign Relations? The Gnomes of Zurich?

No. It's the Delta Kappa Epsilon (DKE) college fraternity, known for branding its members with red-hot coat hangers during initiation.

College fraternity brothers are, on the whole, not the most rational of people. (I say this as a current Yale student.) Rationalists are supposed to win. DKE is clearly winning; the members of DKE are collectively far more powerful than the entire atheist movement, despite having far fewer members. What is going on here?

To help answer this question, consider a community of ninety people, living, perhaps, during the Bronze Age. For simplicity, let's assume that all the members are purely selfish. The ninety people are governed by a village council of nine, elected annually. At the end of every year, every one of the ninety people gives a campaign speech, and whoever can persuade the most people to vote for them wins.

Now, suppose that, out of this tribe of ninety, a smaller cabal of ten people forms, and the members of this cabal agree to help each other. Why the cabal was formed doesn't matter. The cabal could be about branding people with hot irons...
A Taxonomy of Bias: Mindware Problems
by Kaj_Sotala — 2010-07-07T21:53:01.966Z

This is the third part in a mini-sequence presenting content from Keith E. Stanovich's excellent book What Intelligence Tests Miss: The psychology of rational thought. It will culminate in a review of the book itself.

Noting that there are many different kinds of bias, Keith Stanovich proposes a classification scheme for bias that has two primary categories: the Cognitive Miser, and Mindware Gaps. Last time, I discussed the Cognitive Miser category. Today, I will discuss Mindware Problems, which has the subcategories of Mindware Gaps and Corrupted Mindware.

Mindware Problems

Stanovich defines "mindware" as "a generic label for the rules, knowledge, procedures, and strategies that a person can retrieve from memory in order to aid decision making and problem solving".

Mindware Gaps

Previously, I mentioned two tragic cases. In one, a pediatrician incorrectly testified the odds of a two children in the same family suffering infant death syndrome to be 73 million to 1. In the other, people bought into a story of "facilitated communication" helping previously non-verbal children to communicate, without looking at it in a critical manner. Stanovich uses these two as examples of a mindware gap. The people involved were lacking critical mindware: in one case, that of probabilistic reasoning, in the other, that of scientific thinking. One of the reasons why so many intelligent people can act in an irrational manner is that they're simply missing the mindware necessary for rational decision-making.

Much of the useful mindware is a matter of knowledge: knowledge of Bayes' theorem, taking into account alternative hypotheses and falsifiability, awareness of the conjunction fallacy, and so on. Stanovich also mentions something he calls strategic mindware, which refers to the disposition towards engaging the reflective mind in problem solving. These were previously mentioned as thinking dispositions, and some of them can be measured by performance-based tasks. For instance, in the Matching Familiar Figures Test (MFFT), participants are presented with a picture of an object, and told to find the correct match from an array of six other similar pictures. Reflective people have long response times and few errors, while impulsive people have short response times and numerous errors. These types of mindware are closer to strategies, tendenc...