trentmkelly/LessWrong-43k huggingface dataset:
Is Google Paperclipping the Web? The Perils of Optimization by Proxy in Social Systems
by Alexandros — 2010-05-10T13:25:41.567Z

Related to: The Importance of Goodhart's Law, Lucas Critique, Campbell's Law

tl;dr version: The article introduces the pattern of Optimization by Proxy (OBP), which can be found in many large scale distributed systems, including human societies. The pattern occurs when a computationally limited algorithm uses a proxy property as a shortcut indicator for the presence of a hard to measure target quality. When intelligent actors with different motivations control part of the data, the existence of the algorithm reifies the proxy into a separate attribute to be manipulated with the goal of altering the algorithm's results. This concept is then applied to Google and the many ways it interacts with the various groups of actors on the web. The second part of this article contains examination of how OBP contributes towards the degrading of the content of the web, and how this relates to the Friendly Artificial Intelligence concept of 'paperclipping'.

Introducing OBP

The first thing a newly-hatched herring gull does after breaking out of its shell is to peck on its mother’s beak, which causes her to give it its first feeding. Puzzled by this apparent automatic recognition of its mother, Dutch ethologist and ornithologist Nikolaas Tinbergen conducted a sequence of experiments designed to determine what precisely it was that the newborn herring gull was attracted to. After experimenting with facsimiles of adult female herring gulls, he realized that the beak alone, without the bird, would elicit the response. Through multiple further iterations he found that the characteristics that the newborns were attracted to were thinness, elongation, redness and an area with high contrast. Thus, the birds would react much more intensely to a long red stick-like beak with painted stripes on the tip than they would to a real female herring gull. It turns out that the chicks don't have an ingrained definition of 'motherness' but rather determine their initial actions by obeying very simple rules, and are liable to radically miss the mark in the presence of objects that are explicitly designed to the specification of these rules. Objects of this class, able to dominate the attention of an animal away from the intended target were later called ‘supernormal stimuli’ (or superstimuli) and have been commonly observed in nature and our own human env...
Why not fake height?
by KatjaGrace — 2010-05-09T11:59:20.000Z

Barking up the wrong tree:

to match the dating success of a man one inch taller, a 5’9″ man would have to make $30,000 a year more…

So why don’t men wear high heels? Obviously the immediate reason is to avoid looking like women, since women wear them. But there are all sorts of things that both men and women do without men becoming sullied by girliness (for instance wearing high heels at other times in history). And why didn’t men get in first and claim high heels for manliness, if they should benefit from it so much? We would be puzzled if in another culture men were the only ones with push up bras, because push up bras were too manly for women to wear.

Even with the danger of looking like a girl in proper high heels, isn’t there a temptation to get men’s shoes with a tiny bit higher heel than usual? Maybe just $10,00 worth of income’s heel? Presumably there is some heel increase that wouldn’t stand out as effeminate. And when that’s commonplace, wouldn’t it be tempting to add a tiny bit more? If height is such an advantage to men, and the danger of girliness shouldn’t stop a gradual increase, what’s the barrier?

Note: I removed the possibility of trackbacks to this post because it was receiving more filter evading spam than I could be bothered looking at.
The Psychological Diversity of Mankind
by Kaj_Sotala — 2010-05-09T05:53:54.487Z

The dominant belief on this site seems to be in the "psychological unity of mankind". In other words, all of humanity shares the same underlying psychological machinery. Furthermore, that machinery has not had the time to significantly change in the 50,000 or so years that have passed after we started moving out of our ancestral environment.

In The 10,000 Year Explosion, Gregory Cochran and Henry Harpending dispute part of this claim. While they freely admit that we have probably not had enough time to develop new complex adaptations, they emphasize the speed at which minor adaptations can spread throughout populations and have powerful effects. Their basic thesis is that the notion of a psychological unity is most likely false. Different human populations are likely for biological reasons to have slightly different minds, shaped by selection pressures in the specific regions the populations happened to live in. They build support for their claim by:

Discussing known cases where selection has led to rapid physiological and psychological changes among animals

Discussing known cases where selection has led to physiological changes among humans in the last few thousand years, as well as presenting some less certain hypotheses of this.

Postulating selection pressures that would have led to some cognitive abilities to be favored among humans.

In what follows, I will present their case by briefly summarizing the contents of the book. Do note that I've picked the points that I found the most interesting, leaving a lot out.

They first chapter begins by discussing a number of interesting examples:

Dogs were domesticated from wolves around 15,000 years ago: by now, there exists a huge variety of different dog breeds. Dogs are good at reading human voice and gestures, while wolves can't understand us at all. Male wolves pair-bond with females and put a lot of effort into helping raise their pups, but male dogs generally do not. Most of the dog breeds we know today are no more than a couple of centuries old. There is considerable psychological variance between dog breeds: in 1982-2006, there were 1,110 dog attacks in the US that were attributable to pit bull terriers, but only one attributable to Border collies. Border collies, on average, learn a new command after 5 repetitions and respond correctly 95 percent of the time, while a basset hound needs 80-100 repetitions

...
What is bunk?
by  — 2010-05-08T18:06:08.435Z

Related: http://lesswrong.com/lw/1kh/the_correct_contrarian_cluster/, http://lesswrong.com/lw/1mh/that_magical_click/, http://lesswrong.com/lw/18b/reason_as_memetic_immune_disorder/

Given a claim, and assuming that its truth or falsehood would be important to you, how do you decide if it's worth investigating? How do you identify "bunk" or "crackpot" ideas?

Here are some examples to give an idea.

"Here's a perpetual motion machine": bunk. "I've found an elementary proof of Fermat's Last Theorem": bunk. "9-11 was an inside job": bunk.

"Humans did not cause global warming": possibly bunk, but I'm not sure. "The Singularity will come within 100 years": possibly bunk, but I'm not sure. "The economic system is close to collapse": possibly bunk, but I'm not sure.

"There is a genetic difference in IQ between races": I think it's probably false, but not quite bunk. "Geoengineering would be effective in mitigating global warming": I think it's probably false, but not quite bunk.

(These are my own examples. They're meant to be illustrative, not definitive. I imagine that some people here will think "But that's obviously not bunk!" Sure, but you probably can think of some claim that *you* consider bunk.)

A few notes of clarification: I'm only examining factual, not normative, claims. I also am not looking at well established claims (say, special relativity) which are obviously not bunk. Neither am I looking at claims where it's easy to pull data that obviously refutes them. (For example, "There are 10 people in the US population.") I'm concerned with claims that look unlikely, but not impossible. Also, "Is this bunk?" is not the same question as "Is this true?" A hypothesis can turn out to be false without being bunk (for example, the claim that geological formations were created by gradual processes. That was a respectable position for 19th century geologists to take, and a claim worth investigating, even if subsequent evidence did show it to be false.) The question "Is this bunk?" arises when someone makes an unlikely-sounding claim, but I don't actually have the knowledge right now to effectively refute it, and I want to know if the claim is a legitimate subject of inquiry or the work of a conspiracy theory/hoax/cult/crackpot. In other words, is it a ...
Moving marginal mothers
by KatjaGrace — 2010-05-07T14:40:14.000Z

Julian Savulescu suggests extending the idea of paying drug addicts not to have children to everyone. At first the purpose is to avoid the eugenics feel of discouraging only one set of people from procreating, but then he reasons:

“The benefit of a policy of offering inducements to sterilisation is that it would select those who do not value, do not understand, do not want the role of parent. And it is precisely these people who are likely to be the worst parents.

Being a parent is, at best, a difficult job. Why not excuse those with the least motivation and determination? There are plenty of others willing to take their place. And the earth can only sustain a finite number of people.”

It’s of course true that if you penalize an activity, those to whom it is most expensive already will be the ones to quit. However:

The existing costs of parenting already induce those who dislike parenting most not to parent. Adding another cost to parenting would just move the line where it becomes worthwhile to parent, not implement such selection. Justifying this requires an argument that the level of value at which people find parenting worthwhile is too low, not just a desire to encourage better parents to do a greater proportion of parenting in general.

“Excuse those with the least motivation and determination”? We aren’t exactly pushing them to do it. Why presume they don’t excuse themselves at the appropriate point? This goes with the above point; the line where parenting seems worthwhile could be in the wrong place if parents were pushed for some reason to have too many children, but why think they misjudge?

Why would there be plenty of others willing to take their places? Presumably those wanting to bear children will do so already or at least would not start at a 1:1 ratio on the news that others are not. Few factors influencing conception depend on the ambient birthrate.

If others really were willing to ‘take their place’, the exit of poor parents from parenting wouldn’t be relevant to the total population and whether the planet can sustain it.

Presumably the issue is how big a finite number of people the earth’s resources can support, and more importantly why and to what extent parents should be expected to misjudge.

Smaller populations are not automatically better if you value human life at all. That parents are unlikely to ac

...
Beauty quips, "I'd shut up and multiply!"
by neq1 — 2010-05-07T14:34:27.204Z

When it comes to probability, you should trust probability laws over your intuition. Many people got the Monty Hall problem wrong because their intuition was bad. You can get the solution to that problem using probability laws that you learned in Stats 101 -- it's not a hard problem. Similarly, there has been a lot of debate about the Sleeping Beauty problem. Again, though, that's because people are starting with their intuition instead of letting probability laws lead them to understanding.

The Sleeping Beauty Problem

On Sunday she is given a drug that sends her to sleep. A fair coin is then tossed just once in the course of the experiment to determine which experimental procedure is undertaken. If the coin comes up heads, Beauty is awakened and interviewed on Monday, and then the experiment ends. If the coin comes up tails, she is awakened and interviewed on Monday, given a second dose of the sleeping drug, and awakened and interviewed again on Tuesday. The experiment then ends on Tuesday, without flipping the coin again. The sleeping drug induces a mild amnesia, so that she cannot remember any previous awakenings during the course of the experiment (if any). During the experiment, she has no access to anything that would give a clue as to the day of the week. However, she knows all the details of the experiment.

Each interview consists of one question, "What is your credence now for the proposition that our coin landed heads?"

Two popular solutions have been proposed: 1/3 and 1/2

The 1/3 solution

From wikipedia:

Suppose this experiment were repeated 1,000 times. We would expect to get 500 heads and 500 tails. So Beauty would be awoken 500 times after heads on Monday, 500 times after tails on Monday, and 500 times after tails on Tuesday. In other words, only in a third of the cases would heads precede her awakening. So the right answer for her to give is 1/3.

Yes, it's true that only in a third of cases would heads precede her awakening.

Radford Neal (a statistician!) argues that 1/3 is the correct solution.

This [the 1/3] view can be reinforced by supposing that on each awakening Beauty is offered a bet in which she wins 2 dollars if the coin lands Tails and loses 3 dollars if it lands Heads. (We suppose that Beauty knows such a bet will always be offered.) Beauty would not accept this bet if she assigns probability 1/2 to Heads. If she...
Antagonizing Opioid Receptors for (Prevention of) Fun and Profit
by Scott Alexander — 2010-05-05T14:40:12.797Z

Related to: Ugh Fields, Are Wireheads Happy?

In his post Ugh Fields, Roko discussed "temporal difference learning", the process by which the brain propagates positive or negative feedback to the closest cause it can find for the feedback. For example, if he forgets to pay his bills and gets in trouble, the trouble (negative feedback) propagates back to thoughts about bills. Next time he gets a bill, he might paradoxically have even more trouble paying it, because it's become associated with trouble and negative emotions, and his brain tends to unconsciously flinch away from it.
He links to the associated Wikipedia article:

The TD algorithm has also received attention in the field of neuroscience. Researchers discovered that the firing rate of dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) appear to mimic the error function in the algorithm. The error function reports back the difference between the estimated reward at any given state or time step and the actual reward received. The larger the error function, the larger the difference between the expected and actual reward. When this is paired with a stimulus that accurately reflects a future reward, the error can be used to associate the stimulus with the future reward.
Dopamine cells appear to behave in a similar manner. In one experiment measurements of dopamine cells were made while training a monkey to associate a stimulus with the reward of juice. Initially the dopamine cells increased firing rates when exposed to the juice, indicating a difference in expected and actual rewards. Over time this increase in firing back propagated to the earliest reliable stimulus for the reward. Once the monkey was fully trained, there was no increase in firing rate upon presentation of the predicted reward. This mimics closely how the error function in TD is used for reinforcement learning.

So if I understand this right, the monkey hears a bell and is unimpressed, having no expectation of reward. Then the monkey gets some juice that tastes really good and activates (opioid dependent?) reward pathways. The dopamine system is pretty surprised, and broadcasts that surprise back to all the neurons that have been especially active recently, most notably the neurons that activated upon hearing the bell. These neurons are now more heavily associated with the dopamine system. So the next time the...
The Cameron Todd Willingham test
by Kevin — 2010-05-05T00:11:47.162Z

In 2004, The United States government executed Cameron Todd Willingham via lethal injection for the crime of murdering his young children by setting fire to his house.

In 2009, David Grann wrote an extended examination of the evidence in the Willingham case for The New Yorker, which has called into question Willingham's guilt. One of the prosecutors in the Willingham case, John Jackson, wrote a response summarizing the evidence from his current perspective. I am not summarizing the evidence here so as to not give the impression of selectively choosing the evidence.

A prior probability estimate for Willingham's guilt (certainly not a close to optimal prior probability) is the probability that a fire resulting in the fatalities of children was intentionally set. The US Fire Administration puts this probability at 13%. The prior probability could be made more accurate by breaking down that 13% of intentionally set fires into different demographic sets, or looking at correlations with other things such as life insurance data.

My question for Less Wrong: Just how innocent is Cameron Todd Willingham? Intuitively, it seems to me that the evidence for Willingham's innocence is of higher magnitude than the evidence for Amanda Knox's innocence. But the prior probability of Willingham being guilty given his children died in a fire in his home is higher than the probability that Amanda Knox committed murder given that a murder occurred in Knox's house.

Challenge question: What does an idealized form of Bayesian Justice look like? I suspect as a start that it would result in a smaller percentage of defendants being found guilty at trial. This article has some examples of the failures to apply Bayesian statistics in existing justice systems.
Experiences are friends
by KatjaGrace — 2010-05-04T21:03:03.000Z

Products tend to be less satisfying than experiences. This is old news. Psyblog elaborates on six reasons from a set of (hard to access) recent experiments, which add up to this:

We compare products more than experiences, and since products are doomed to not be the best we could ever have got, we are sad. When we don’t compare, we are happy.

This requires one of two things:

that when we can’t compare something, we assume it is better than average

that we find knowing how something compares displeasing in itself unless the thing is the best.

Either of these seem like puzzling behaviour. Why would we do one of them?

The first one reminds me of the way people usually like the children they have more than the hypothetical children of any other combinations of genes they could have had. Similarly but to a lesser extent, people are uncomfortable comparing their friends and partners with others they might have had instead, and in the absence of comparison most people think those they love are pretty good. You rarely hear ‘there are likely about half a billion wives I would like more than you out there, but you are the one I’m arbitrarily in love with’.

This all makes evolutionary sense; blind loyalty is better than ongoing evaluation from an ally, at least towards you. So you evaluate people accurately for a bit, then commit to the good ones. Notice that here the motivation for not comparing appears to come from the benefits of committing to people without regret, rather than the difficulty of figuring out what a nice bottom is worth next to a good career.

I wonder if our not comparing experiences, and rating them well regardless is related. Experiences we buy are often parts of our relationships with other people, while objects usually aren’t. So to compare your experiences and evaluate them as accurately as you can comes dangerously close to comparing bits of your relationships and evaluating them accuracely.

For instance, if I see there was a cheaper airfare than one I took, to entertain the thought that it would have been better to travel a week later is to admit I would give up all the moments you and I spent together on that trip for some other set of experiences and fifty dollars, which feels uncomfortably like calculating and judging our time together as average and replacable.

If this explanation were true there would be less need for the other explanation for ...
But Somebody Would Have Noticed
by Alicorn — 2010-05-04T18:56:34.802Z

When you hear a hypothesis that is completely new to you, and seems important enough that you want to dismiss it with "but somebody would have noticed!", beware this temptation. If you're hearing it, somebody noticed.

Disclaimer: I do not believe in anything I would expect anyone here to call a "conspiracy theory" or similar. I am not trying to "soften you up" for a future surprise with this post.

1. Wednesday

Suppose: Wednesday gets to be about eighteen, and goes on a trip to visit her Auntie Alicorn, who has hitherto refrained from bringing up religion around her out of respect for her parents1. During the visit, Sunday rolls around, and Wednesday observes that Alicorn is (a) wearing pants, not a skirt or a dress - unsuitable church attire! and (b) does not appear to be making any move to go to church at all, while (c) not being sick or otherwise having a very good excuse to skip church. Wednesday inquires as to why this is so, fearing she'll find that beloved Auntie has been excommunicated or something (gasp! horror!).

Auntie Alicorn says, "Well, I never told you this because your parents asked me not to when you were a child, but I suppose now it's time you knew. I'm an atheist, and I don't believe God exists, so I don't generally go to church."

And Wednesday says, "Don't be silly. If God didn't exist, don't you think somebody would have noticed?"

2. Ignoring Soothsayers

Wednesday's environment reinforces the idea that God exists relentlessly. Everyone she commonly associates with believes it; people who don't, and insist on telling her, are quickly shepherded out of her life. Because Wednesday is not the protagonist of a fantasy novel, people who are laughed out of public discourse for shouting unpopular, outlandish, silly ideas rarely turn out to have plot significance later: it simply doesn't matter what that weirdo was yelling, because it was wrong and everybody knows it. It was only one person. More than one person would have noticed if something that weird were true. Or maybe it was only six or twelve people. At any rate, it wasn't enough. How many would be enough? Well, uh, more than that.

But even if you airdropped Wednesday into an entire convention center full of atheists, you would find that you cannot outnumber her home team. We have lots of mechanisms for...