trentmkelly/LessWrong-43k huggingface dataset:
Perfect principles are for bargaining
by KatjaGrace — 2010-01-02T04:38:21.000Z

When people commit to principles, they often consider one transgression ruinous to the whole agenda. Eating a sausage by drunken accident can end years of vegetarianism.

As a child I thought this crazy. Couldn’t vegetarians just eat meat when it was cheap under their rationale? Scrumptious leftovers at our restaurant, otherwise to be thrown away, couldn’t tempt vegetarian kids I knew. It would break their vegetarianism. Break it? Why did the integrity of the whole string of meals matter? Any given sausage was such a tiny effect.

I eventually found two explanations. First, it’s easier to thwart temptation if you stake the whole deal on every choice. This is similar to betting a thousand dollars that you won’t eat chocolate this month. Second, commitment without gaps makes you seem a nicer, more reliable person to deal with. Viewers can’t necessarily judge the worthiness of each transgression, so they suspect the selectively committed of hypocrisy. Plus everyone can better rely on and trust a person who honors his commitments with less regard to consequence.

There’s another good reason though, which is related to the first. For almost any commitment there are constantly other people saying things like ‘What?! You want me to cook a separate meal because you have some fuzzy notion that there will be slightly less carbon emitted somewhere if you don’t eat this steak?’ Maintaining an ideal requires constantly negotiating with other parties who must suffer for it. Placing a lot of value on unmarred principles gives you a big advantage in these negotiations.

In negotiating generally, it is often useful to arrange visible costs to yourself for relinquishing too much ground. This is to persuade the other party that if they insist on the agreement being in that region, you will truly not be able to make a deal. So they are forced to agree to a position more favorable to you. This is the idea behind arranging for your parents to viciously punish you for smoking with your friends if you don’t want to smoke much. Similarly, attaching a visible large cost – the symbolic sacrifice of your principles – to relieving a friend of cooking tofu persuades your friend that you just can’t eat with them unless they concede. So that whole conversation is avoided, determined in your favor from the outset.

I used to be a vegetarian, and it was much less embarrassing to ask for vegetarian fo...
Are wireheads happy?
by Scott Alexander — 2010-01-01T16:41:31.102Z

Related to: Utilons vs. Hedons, Would Your Real Preferences Please Stand Up

And I don't mean that question in the semantic "but what is happiness?" sense, or in the deep philosophical "but can anyone not facing struggle and adversity truly be happy?" sense. I mean it in the totally literal sense. Are wireheads having fun?
They look like they are. People and animals connected to wireheading devices get upset when the wireheading is taken away and will do anything to get it back. And it's electricity shot directly into the reward center of the brain. What's not to like?
Only now neuroscientists are starting to recognize a difference between "reward" and "pleasure", or call it "wanting" and "liking". The two are usually closely correlated. You want something, you get it, then you feel happy. The simple principle behind our entire consumer culture. But do neuroscience and our own experience really support that?

It would be too easy to point out times when people want things, get them, and then later realize they weren't so great. That could be a simple case of misunderstanding the object's true utility. What about wanting something, getting it, realizing it's not so great, and then wanting it just as much the next day? Or what about not wanting something, getting it, realizing it makes you very happy, and then continuing not to want it?

The first category, "things you do even though you don't like them very much" sounds like many drug addictions. Smokers may enjoy smoking, and they may want to avoid the physiological signs of withdrawl, but neither of those is enough to explain their reluctance to quit smoking. I don't smoke, but I made the mistake of starting a can of Pringles yesterday. If you asked me my favorite food, there are dozens of things I would say before "Pringles". Right now, and for the vast majority of my life, I feel no desire to go and get Pringles. But once I've had that first chip, my motivation for a second chip goes through the roof, without my subjective assessment of how tasty Pringles are changing one bit.

Think of the second category as "things you procrastinate even though you like them." I used to think procrastination applied only to things you disliked but did anyway. Then I tried to write a novel. I loved writing. Every second I was writing, I was thinking "This is so much fun". And I never got past the second chapter, because I just cou...
New Year’s and New Decade’s Predictions
by MichaelVassar — 2009-12-30T21:37:45.042Z

<!-- /* Font Definitions */ @font-face	{font-family:Cambria;	panose-1:2 4 5 3 5 4 6 3 2 4;	mso-font-charset:0;	mso-generic-font-family:auto;	mso-font-pitch:variable;	mso-font-signature:3 0 0 0 1 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal	{mso-style-parent:"";	margin-top:0in;	margin-right:0in;	margin-bottom:10.0pt;	margin-left:0in;	mso-pagination:widow-orphan;	font-size:12.0pt;	font-family:"Times New Roman";	mso-ascii-font-family:Cambria;	mso-ascii-theme-font:minor-latin;	mso-fareast-font-family:Cambria;	mso-fareast-theme-font:minor-latin;	mso-hansi-font-family:Cambria;	mso-hansi-theme-font:minor-latin;	mso-bidi-font-family:"Times New Roman";	mso-bidi-theme-font:minor-bidi;} @page Section1	{size:8.5in 11.0in;	margin:1.0in 1.25in 1.0in 1.25in;	mso-header-margin:.5in;	mso-footer-margin:.5in;	mso-paper-source:0;} div.Section1	{page:Section1;} --> I would like to propose this as a thread for people to write in their predictions for the next year and the next decade, when practical with probabilities attached. I'll make some in the comments as they come to me.
New Year’s Resolutions Thread
by MichaelVassar — 2009-12-30T21:36:20.869Z

I would like to propose this as a thread for people to write in their New Year’s Resolutions (goals and sub-goals) as instrumental rationalists.

Here are mine.

Goals

I resolve to try to enable all pareto-optimizing trades between my sub-agents can be made between them. For instance, I have agents which would like to maximize my success via improving my cognition and energy levels via eating more healthily via eating less. Other agents would like enjoyment from food. I note that these agents aren't cooperating even though they both benefit from the same changes in behavior, largely because it hasn't been pointed out to them that they are on the same side. If I make a serious effort to eat the hedonic utility maximizing amount, this will probably involve eating less than I default to. After all, food is better when one is hungry. Most of my eating is probably driven by simple non-reflective systems that tell me to eat. These systems are probably promoted by hedonistic systems which are failing to understand the consequences of doing so. In practice, this resolutoin means paying attention to the experience of eating anything that wasn’t chosen for social or nutritional purposes, rarely clearing my plate, and rarely eating more than one would get served in a European restaurant, but above all, it means paying attention (and thus sending this information to many of my sub-agents) to the pleasure of eating when one is actually hungry.

I resolve to find a new home, get fully moved in, get a car, touch base with all interested Bay Area supporters, and get started on the 2010 Summit by the end of February despite this involving many boring activities.

I resolve to stop trying to keep up with a significant part of the blogosphere. My web-browsing will be limited to a) actually seeking specific information, b) checking email no more than 5-times-per-day and c) the keeping up, via google reader, with fewer than ten sites.

Also, in February I will try to use dual-n-back every day and in March I will try to publish a Less-Wrong piece every day.
Boksops -- Ancient Superintelligence?
by MBlume — 2009-12-30T11:12:15.041Z

[...] before long the skull came to the attention of S. H. Haughton, one of the country’s few formally trained paleontologists. He reported his findings at a 1915 meeting of the Royal Society of South Africa. “The cranial capacity must have been very large,” he said, and “calculation by the method of Broca gives a minimum figure of 1,832 cc [cubic centimeters].” The Boskop skull, it would seem, housed a brain perhaps 25 percent or more larger than our own.

The idea that giant-brained people were not so long ago walking the dusty plains of South Africa was sufficiently shocking to draw in the luminaries back in England. Two of the most prominent anatomists of the day, both experts in the reconstruction of skulls, weighed in with opinions generally supportive of Haughton’s conclusions.

The Scottish scientist Robert Broom reported that “we get for the corrected cranial capacity of the Boskop skull the very remarkable figure of 1,980 cc.” Remarkable indeed: These measures say that the distance from Boskop to humans is greater than the distance between humans and their Homo erectus predecessors.

What Happened to the Hominids who were Smarter than Us?

I'm strongly inclined to defy the data -- true superintelligence should have just dominated our ancestors -- but given the expense of large skull size (primarily in difficult birthing) it also seems profoundly unlikely that a lineage would see expansion like this that wasn't buying them something mentally.
Singularity Institute $100K Challenge Grant / 2009 Donations Reminder
by Eliezer Yudkowsky — 2009-12-30T00:36:19.627Z

In case you missed the notice at the SIAI Blog, the Singularity Institute's next Challenge Grant is now running - $100,000 of matching funds for all donations until February 28th, 2010. If you want your donation to be tax-deductible in the U.S. for 2009, donations postmarked by December 31st will be counted for this tax year.
That other kind of status
by Scott Alexander — 2009-12-29T02:45:34.179Z

"Human nature 101. Once they've staked their identity on being part of the defiant elect who know the Hidden Truth, there's no way it'll occur to them that they're our catspaws." - Mysterious Conspirator A

This sentence sums up a very large category of human experience and motivation. Informally we talk about this all the time; formally it usually gets ignored in favor of a simple ladder model of status.
In the ladder model, status is a one-dimensional line from low to high. Every person occupies a certain rung on the ladder determined by other people's respect. When people take status-seeking actions, their goal is to to change other people's opinions of themselves and move up the ladder.
But many, maybe most human actions are counterproductive at moving up the status ladder. 9-11 Conspiracy Theories are a case in point. They're a quick and easy way to have most of society think you're stupid and crazy. So is serious interest in the paranormal or any extremist political or religious belief. So why do these stay popular?

Could these just be the conclusions reached by honest (but presumably mistaken) truth-seekers unmotivated by status? It's possible, but many people not only hold these beliefs, but flaunt them out of proportion to any good they could do. And there are also cases of people pursuing low-status roles where there is no "fact of the matter". People take great efforts to identify themselves as Goths or Juggalos or whatever even when it's a quick status hit.
Classically people in these subcultures are low status in normal society. Since subcultures are smaller and use different criteria for high status, maybe they just want to be a big fish in a small pond, or rule in Hell rather than serve in Heaven, or be first in a village instead of second in Rome. The sheer number of idioms for the idea in the English language suggests that somebody somewhere must have thought along those lines.
But sometimes it's a subculture of one. That Time Cube guy, for example. He's not in it to gain cred with all the other Time Cube guys. And there are 9-11 Truthers who don't know any other Truthers in real life and may not even correspond with others online besides reading a few websites.
Which brings us back to Eliezer's explanation: the Truthers have "staked their identity on being part of the defiant elect who know the Hidden Truth". But what does that mean?
A biologi...
A Master-Slave Model of Human Preferences
by Wei Dai — 2009-12-29T01:02:23.694Z

[This post is an expansion of my previous open thread comment, and largely inspired by Robin Hanson's writings.]

In this post, I'll describe a simple agent, a toy model, whose preferences have some human-like features, as a test for those who propose to "extract" or "extrapolate" our preferences into a well-defined and rational form. What would the output of their extraction/extrapolation algorithms look like, after running on this toy model? Do the results agree with our intuitions about how this agent's preferences should be formalized? Or alternatively, since we haven't gotten that far along yet, we can use the model as one basis for a discussion about how we want to design those algorithms, or how we might want to make our own preferences more rational. This model is also intended to offer some insights into certain features of human preference, even though it doesn't capture all of them (it completely ignores akrasia for example).

I'll call it the master-slave model. The agent is composed of two sub-agents, the master and the slave, each having their own goals. (The master is meant to represent unconscious parts of a human mind, and the slave corresponds to the conscious parts.) The master's terminal values are: health, sex, status, and power (representable by some relatively simple utility function). It controls the slave in two ways: direct reinforcement via pain and pleasure, and the ability to perform surgery on the slave's terminal values. It can, for example, reward the slave with pleasure when it finds something tasty to eat, or cause the slave to become obsessed with number theory as a way to gain status as a mathematician. However it has no direct way to control the agent's actions, which is left up to the slave.

The slave's terminal values are to maximize pleasure, minimize pain, plus additional terminal values assigned by the master. Normally it's not aware of what the master does, so pain and pleasure just seem to occur after certain events, and it learns to anticipate them. And its other interests change from time to time for no apparent reason (but actually they change because the master has responded to changing circumstances by changing the slave's values). For example, the number theorist might one day have a sudden revelation that abstract mathematics is a waste of time and it should go into politics and philanthropy instead, all the whil...
Scaling Evidence and Faith
by AndrewKemendo — 2009-12-27T12:30:45.561Z

Often when talking with people of faith about the issue of atheism, a common response I get is:

“It takes as much faith to believe in evolution as it does to believe in god.”

Any atheist who has ever talked to a theist has likely encountered this line of reasoning. There is a distinction to be made between the glorifying of faith as evidence of truth, as theists do, and the desire to become less faithful in our understanding, as the rational truth seeking person does.

As has been discussed previously, realistically P = 1 is not attainable, at least for the foreseeable future. Nor is P=1 necessary to be comfortable in the universe. Richard Feynman and Carl Sagan among others, feel the same way. We know that heuristics “fill the gaps” in knowledge of recognizable scenarios such that we can comfortably demand less evidence and still come to a reasonable conclusion about our surroundings in most cases. We also know the myriad of ways in which those heuristics fail. Much has been written about uncertainty in the decision making process and I would imagine it will be the focus of much of the future research into logic. So, I have no problems concluding that there is some level of faith in all decision making. How much faith in a claim am I comfortable with? The way in which I have been thinking about the different levels of "faith" recently is as a sliding bar:

On the left side, natural evidence is indicated in blue which is comprised of data points which indicate to the observer the natural proof of the argument. On the right, faith is indicated in red which is comprised of either counter evidence for the claim or a lack of information. The bar’s units can be anything (probability, individual records) so long as both sides are measured in equal parts.

A few examples:

Claim: Natural and sexual selection as described by Charles Darwin accurately describes the processes which develop biological traits in separate species.

I agree with this claim with 99% natural evidence in the form of observed speciation and fossil and geologic discoveries and accept 1% faith because, well, anything can happen:

Claim: The price of Human full DNA sequencing will fall below $1000.00 before 2012.

I agree with this at 80% natural evidence based on the projection that the price wil

...
Playing the Meta-game
by Technologos — 2009-12-25T10:06:46.559Z

In honor of today's Schelling-pointmas, a true Schelling-inspired story from a class I was in at a law school I did not attend:

As always, the class was dead silent as we walked to the front of the room. The professor only described the game after the participants had volunteered and been chosen; as a result, we rarely were familiar with the games we were playing, which the professor preferred because his money was on the line.

Both of us were assigned different groups of seven partners in the class. I was given seven slips of paper and my opponent was given six. Our goal was to make deals with our partners about how to divide a dollar, one per partner, and then write the deal down on a slip of paper. Whoever had a greater total take from the deals won $20. All negotiations were public.

The professor left the room, giving us three minutes to negotiate. The class exploded.

And then I hit a wall. Everybody with whom I was negotiating knew the rules, and they knew that I cared a hell of a lot more about the results of the negotiation than they did. I was getting offers on the order of $.20 and less--results straight from the theory of the ultimatum game--and no amount of begging or threatening was changing that.

Three minutes pass quickly under pressure. When the professor returned, I had written a total of $1.45 in deals: most people eventually accepted my meta-argument that they really didn't want to carry small coins around with them, so they should give me a quarter and take three for themselves, but two people waited until the last second and took 90 cents each. Even then, I only got ten cents from those two by threatening not to accept one- or five-cent deals.

My opponent, on the other hand, had amassed a relative fortune: over five dollars. It turned out that he had been using the fact that he could make fewer deals than he had partners to auction off the chance to make a deal. His partners kept naming lower and lower demands, and he ended up getting the majority of each dollar with little effort.

I made a mock-anguished face, as the professor explained that the game was set up to demonstrate the effect of scarcity on the balance between merchants and customers. Yeah, yeah, monopolies are bad. Econ 101 stuff.

Then he turned to me and asked why I lost, when the odds were stacked in my f...
Positive-affect-day-Schelling-point-mas Meetup
by Marcello — 2009-12-23T19:41:02.761Z

There will be a LessWrong Meetup on the Friday December 25th (day after tomorrow.) We're meeting at 6:00 PM at Pan Tao Restaurant at 1686 South Wolfe Road, Sunnyvale, CA the SIAI House in Santa Clara, CA for pizza or whatever else we can figure out how to cook. Consider it an available refuge if you haven't other plans.

Please comment if you plan to show up!

(Edit - See poll below on whether we'd rather stay in and eat something simple vs. going out to a restaurant - it's possible that everyone was assuming everyone else would prefer the latter while actually preferring the former themselves. - EY)
Are these cognitive biases, biases?
by Kaj_Sotala — 2009-12-23T17:27:09.137Z

Continuing my special report on people who don't think human reasoning is all that bad, I'll now briefly present some studies which claim that phenomena other researchers have considered signs of faulty reasoning aren't actually that. I found these from Gigerenzer (2004), which I in turn found when I went looking for further work done on the Take the Best algorithm.

Before we get to the list - what is Gigerenzer's exact claim when he lists these previous studies? Well, he's saying that minds aren't actually biased, but may make judgments that seem biased in certain environments.

Table 4.1 Twelve examples of phenomena that were first interpreted as "cognitive illusions" but later revalued as reasonable judgments given the environmental structure. [...]

The general argument is that an unbiased mind plus environmental structure (such as unsystematic error, unequal sample sizes, skewed distributions) is sufficient to produce the phenomenon. Note that other factors can also contribute to some of the phenomena. The moral is not that people would never err, but that in order to understand good and bad judgments, one needs to analyze the structure of the problem or of the natural environment.

On to the actual examples. Of the twelve examples referenced, I've included three for now.

The False Consensus Effect

Bias description: People tend to imagine that everyone responds the way they do. They tend to see their own behavior as typical. The tendency to exaggerate how common one’s opinions and behavior are is called the false consensus effect. For example, in one study, subjects were asked to walk around on campus for 30 minutes, wearing a sign board that said "Repent!". Those who agreed to wear the sign estimated that on average 63.5% of their fellow students would also agree, while those who disagreed estimated 23.3% on average.

Counterclaim (Dawes & Mulford, 1996): The correctness of reasoning is not estimated on the basis of whether or not one arrives at the correct result. Instead, we look at whether reach reasonable conclusions given the data they have. Suppose we ask people to estimate whether an urn contains more blue balls or red balls, after allowing them to draw one ball. If one person first draws a red ball, and another person draws a blue ball, then we should expect them to give different estimates. In the absence of other data, you ...