trentmkelly/LessWrong-43k huggingface dataset:
The Popularization Bias
by Wei Dai — 2009-07-17T15:43:30.338Z

I noticed that most recommendations in the recent recommended readings thread consist of either fiction or popularizations of specific scientific disciplines. This introduces a potential bias: aspiring rationalists may never learn about some fields or ideas that are important for the art of rationality, just because they've never been popularized.

In my recent post on the fair division of black-hole negentropy, I tried to introduce two such ideas/fields (which may be one too many for a single post :). One is that black holes have entropy quadratic in mass, and therefore are ideal entropy dumps (or equivalently, negentropy mines). This is a well-known result in thermodynamics, plus an obvious application of it. Some have complained that the idea is too sci-fi, but actually the opposite is true. Unlike other perhaps equally obvious futuristic ideas such as cryonics, AI and the Singularity, I've never read or watched a piece of science fiction that explorered this one. (BTW, in case it's not clear why black-hole negentropy is important for rationality, it implies that value probably scales superlinearly with material and that huge gains from cooperation can be directly derived from the fundamental laws of physics.)

Similarly, there are many popularizations of topics such as the Prisoner's Dilemma and the Nash Equilibrium in non-cooperative game theory (and even a blockbuster movie about John Nash!), but I'm not aware of any for cooperative game theory.

Much of Less Wrong, and Overcoming Bias before it, can be seen as an attempt to correct this bias. Eliezer's posts have provided fictional treatments or popular accounts of probability theory, decision theory, MWI, algorithmic information theory, Bayesian networks, and various ethical theories, to name a few, and others have continued the tradition to some extent. But since popularization and writing fiction are hard, and not many people have both the skills and the motivation to do them, I wonder if there are still other important ideas/fields that most of us don't know about yet.

So here's my request: if you know of such a field or idea, just name it in a comment and give a reference for it, and maybe say a few words about why it's important, if that's not obvious. Some of us may be motivated to learn about it for whatever reason, even from a textbook or academic article, and may eventually produce a popular accoun...
Causes of disagreements
by JustinShovelain — 2009-07-16T21:51:57.422Z

You have a disagreement before you. How do you handle it?
Causes of fake disagreements:
Is the disagreement real? The trivial case is an apparent disagreement occuring over a noisy or low information channel. Internet chat is especially liable to fail this way because of the lack of tone, body language, and relative location cues. People can also disagree through the use of differing definitions with corresponding denotations and connotations. Fortunately, when recognized this cause of disagreement rarely produces problems; the topic at issue rarely is the definitions themselves. If there is a game theoretic reason the agents may also give the appearance of disagreement even though they might well agree in private. The agents could also disagree if they are victims of a Man-in-the-middle attack where someone is intercepting and altering the messages passed between the two parties. Finally, the agents could disagree simply because they are in different contexts. Is the sun yellow I ask? Yes, say you. No, say the aliens at Eta Carinae.
Causes of disagreements about predictions:
Evidence
Assuming the disagreement is real what does that give us? Most commonly the disagreement is about the facts that predicate our actions. To handle these we must first consider our relationship to the other person and how they think (a la superrationality); observations made by others may not be given the same weight we would give those observations if we had made them ourselves. After considering this we must then merge their evidence with our own in a controlled way. With people this gets a bit tricky. Rarely do people give us information we can handle in a cleanly Bayesian way (a la Aumann's agreement theorem). Instead we must merge our explicit evidence sets along with vague abstracted probabilistic intuitions that are half speculation and half partially forgotten memories.
Priors
If we still have a disagreement after considering the evidence what now? The agents could have "started" at different locations in prior or induction space. While it is true that a persons "starting" point and what evidence they've seen can be conflated, it is also possible that they really did start at different locations.
Resource limitations
The disagreement could also be caused by resource limitations and implementation details. Cognition could have sensitive dependence on initial...
Absolute denial for atheists
by taw — 2009-07-16T15:41:02.412Z

This article is a deliberate meta-troll. To be successful I need your trolling cooperation. Now hear me out.

In The Strangest Thing An AI Could Tell You Eliezer talks about asognostics, who have one of their arm paralyzed, and what's most interesting are in absolute denial of this - in spite of overwhelming evidence that their arm is paralyzed they will just come with new and new rationalizations proving it's not.

Doesn't it sound like someone else we know? Yes, religious people! In spite of heaps of empirical evidence against existence of their particular flavour of the supernatural, internal inconsistency of their beliefs, and perfectly plausible alternative explanations being well known, something between 90% and 98% of humans believe in the supernatural world, and is in a state of absolute denial not too dissimilar to one of asognostics. Perhaps as many as billions of people in history have even been willing to die for their absurd beliefs.

We are mostly atheists here - we happen not to share this particular delusion. But please consider an outside view for a moment - how likely is it that unlike almost everyone else we don't have any other such delusions, for which we're in absolute denial of truth in spite of mounting heaps of evidence?

If the delusion is of the kind that all of us share it, we won't be able to find it without building an AI. We might have some of those - it's not too unlikely as we're a small and self-selected group.

What I want you to do is try to trigger absolute denial macro in your fellow rationalists! Is there anything that you consider proven beyond any possibility of doubt by both empirical evidence and pure logic, and yet saying it triggers automatic stream of rationalizations in other people? Yes, I pretty much ask you to troll, but it's a good kind of trolling, and I cannot think of any other way to find our delusions.
Fair Division of Black-Hole Negentropy: an Introduction to Cooperative Game Theory
by Wei Dai — 2009-07-16T04:17:44.081Z

Non-cooperative game theory, as exemplified by the Prisoner’s Dilemma and commonly referred to by just "game theory", is well known in this community. But cooperative game theory seems to be much less well known. Personally, I had barely heard of it until a few weeks ago. Here’s my attempt to give a taste of what cooperative game theory is about, so you can decide whether it might be worth your while to learn more about it.

The example I’ll use is the fair division of black-hole negentropy. It seems likely that for an advanced civilization, the main constraining resource in the universe is negentropy. Every useful activity increases entropy, and since entropy of the universe as a whole never decreases, the excess entropy produced by civilization has to be dumped somewhere. A black hole is the only physical system we know whose entropy grows quadratically with its mass, which makes it ideal as an entropy dump. (See http://weidai.com/black-holes.txt where I go into a bit more detail about this idea.)

Let’s say there is a civilization consisting of a number of individuals, each the owner of some matter with mass mi. They know that their civilization can’t produce more than (∑ mi)2 bits of total entropy over its entire history, and the only way to reach that maximum is for every individual to cooperate and eventually contribute his or her matter into a common black hole. A natural question arises: what is a fair division of the (∑ mi)2 bits of negentropy among the individual matter owners?

Fortunately, Cooperative Game Theory provides a solution, known as the Shapley Value. There are other proposed solutions, but the Shapley Value is well accepted due to its desirable properties such as “symmetry” and “additivity”. Instead of going into the theory, I’ll just show you how it works. The idea is, we take a sequence of players, and consider the marginal contribution of each player to the total value as he or she joins the coalition in that sequence. Each player is given an allocation equal to his or her average marginal contribution over all possible

...
The Dirt on Depression
by pjeby — 2009-07-15T17:58:44.128Z

(From the "humans are crazy" and "truth is stranger than fiction" departments...)

Want to be happy? Try eating dirt... or at least dirty plants.

Seriously.

From an article in Discover magazine, "Is Dirt The New Prozac?":

The results so far suggest that simply inhaling M. vaccae—you get a dose just by taking a walk in the wild or rooting around in the garden—could help elicit a jolly state of mind. “You can also ingest mycobacteria either through water sources or through eating plants—lettuce that you pick from the garden, or carrots,” Lowry says.

Graham Rook, an immunologist at University College London and a coauthor of the paper, adds that depression itself may be in part an inflammatory disorder. By triggering the production of immune cells that curb the inflammatory reaction typical of allergies, M. vaccae may ease that inflammation and hence depression. Therapy with M. vaccae—or with drugs based on the bacterium’s molecular components—might someday be used to treat depression. “It’s not clear to me whether the way ahead will be drugs that circumvent the use of these bugs,” Rook says, “or whether it will be easier to say, ‘The hell with it, let’s use the bugs.’”

Given the way the industry works, we'll probably either see drugs, or somebody will patent the bacteria. But that's sort of secondary. The real point is that to the extent our current environment doesn't match our ancestral one, there are likely to be "bugs", no pun intended.

(The original study: “Identification of an Immune-Responsive Mesolimbocortical Serotonergic System: Potential Role in Regulation of Emotional Behavior,” by Christopher Lowry et al., published online on March 28 in Neuroscience.)
"Sex Is Always Well Worth Its Two-Fold Cost"
by cousin_it — 2009-07-15T09:52:57.503Z

This might be of interest to the evo bio and game theory wannabes here: "Sex Is Always Well Worth Its Two-Fold Cost" by Alexander Feigel, Avraham Englander and Assaf Engel.

Abstract:

Sex is considered as an evolutionary paradox, since its positive contribution to Darwinian fitness remains unverified for some species. Defenses against unpredictable threats (parasites, fluctuating environment and deleterious mutations) are indeed significantly improved by wider genetic variability and by positive epistasis gained by sexual reproduction. The corresponding evolutionary advantages, however, do not overcome universally the barrier of the two-fold cost for sharing half of one's offspring genome with another member of the population. Here we show that sexual reproduction emerges and is maintained even when its Darwinian fitness is twice as low as the fitness of asexuals. We also show that more than two sexes (inheritance of genetic material from three or even more parents) are always evolutionary unstable. Our approach generalizes the evolutionary game theory to analyze species whose members are able to sense the sexual state of their conspecifics and to adapt their own sex consequently, either by switching or by taxis towards the highest concentration of the complementary sex. The widespread emergence and maintenance of sex follows therefore from its co-evolution with the even more widespread environmental sensing abilities.

I'm currently trying to parse the article, and on first reading could only see a disguised form of the old familiar argument about the stability of sex ratios. It still doesn't seem to answer why females don't switch to parthenogenesis and block all male advances. But maybe you can detect something I missed?
The Strangest Thing An AI Could Tell You
by Eliezer Yudkowsky — 2009-07-15T02:27:37.651Z

Human beings are all crazy. And if you tap on our brains just a little, we get so crazy that even other humans notice. Anosognosics are one of my favorite examples of this; people with right-hemisphere damage whose left arms become paralyzed, and who deny that their left arms are paralyzed, coming up with excuses whenever they're asked why they can't move their arms.

A truly wonderful form of brain damage - it disables your ability to notice or accept the brain damage. If you're told outright that your arm is paralyzed, you'll deny it. All the marvelous excuse-generating rationalization faculties of the brain will be mobilized to mask the damage from your own sight. As Yvain summarized:

After a right-hemisphere stroke, she lost movement in her left arm but continuously denied it. When the doctor asked her to move her arm, and she observed it not moving, she claimed that it wasn't actually her arm, it was her daughter's. Why was her daughter's arm attached to her shoulder? The patient claimed her daughter had been there in the bed with her all week. Why was her wedding ring on her daughter's hand? The patient said her daughter had borrowed it. Where was the patient's arm? The patient "turned her head and searched in a bemused way over her left shoulder".

I find it disturbing that the brain has such a simple macro for absolute denial that it can be invoked as a side effect of paralysis. That a single whack on the brain can both disable a left-side motor function, and disable our ability to recognize or accept the disability. Other forms of brain damage also seem to both cause insanity and disallow recognition of that insanity - for example, when people insist that their friends have been replaced by exact duplicates after damage to face-recognizing areas.

And it really makes you wonder...

...what if we all have some form of brain damage in common, so that none of us notice some simple and obvious fact? As blatant, perhaps, as our left arms being paralyzed? Every time this fact intrudes into our universe, we come up with some ridiculous excuse to dismiss it - as ridiculous as "It's my daughter's arm" - only there's no sane doctor watching to pursue the argument any further. (Would we all come up with the same excuse?)

If the "absolute denial macro" is that simple, and invoked that easily...

Now, suppose y...
How likely is a failure of nuclear deterrence?
by Roko — 2009-07-15T00:01:28.640Z

Last month I asked Robert McNamara, the secretary of defense during the Kennedy and Johnson administrations, what he believed back in the 1960s was the status of technical locks on the Minuteman intercontinental missiles. ... McNamara replied ... that he personally saw to it that these [PAL's] ... were installed on the Minuteman force, and that he regarded them as essential to strict central control and preventing unauthorized launch. ... What I then told McNamara about his vitally important locks elicited this response: “I am shocked, absolutely shocked and outraged. Who the hell authorized that?” What he had just learned from me was that the locks had been installed, but everyone knew the combination. The Strategic Air Command (SAC) in Omaha quietly decided to set the “locks” to all zeros in order to circumvent this safeguard. During the early to mid-1970s, during my stint as a Minuteman launch officer, they still had not been changed. Our launch checklist in fact instructed us, the firing crew, to double-check the locking panel in our underground launch bunker to ensure that no digits other than zero had been inadvertently dialed into the panel. SAC remained far less concerned about unauthorized launches than about the potential of these safeguards to interfere with the implementation of wartime launch orders. And so the “secret unlock code” during the height of the nuclear crises of the Cold War remained constant at 00000000.

Training exercises can be mistaken for the real thing. In 1979, a test tape, simulating a Russian attack was mistakenly fed into a NORAD computer connected to the operational missile alert system, resulting in an alert and the launching of American aircraft [Borning 1988].

Read the rest here.
Good Quality Heuristics
by CannibalSmith — 2009-07-14T09:53:11.871Z

We use heuristics when we don't have the time to think more, which is almost all the time. So why don't we compile a big list of good quality heuristics that we can trust? (Insert eloquent analogy with mathematical theorems and proofs.) Here are some heuristics to kick things off:

Make important decisions in a quiet, featureless room. [1]

Apply deodorant before going to bed rather than any other time. [1]

Avoid counterfactuals and thought experiments in when talking to other people. [Because they don't happen in real life. Not in mine at least (anecdotal evidence). For example with the trolley, I would not push the fat man because I'd be frozen in horror. But what if you wouldn't be? But I would! And all too often the teller of a counterfactual abuses it by crafting it so that the other person has to give either an inconsistent or unsavory answer. (This proof is a stub. You can improve it by commenting.)]

If presented with a Monty Hall problem, switch. [1]

Sign up for cryonics. [There are so many. Which ones to link? Wait, didn't Eliezer promise us some cryonics articles here in LW?]

In chit-chat, ask questions and avoid assertions. [How to Win Friends and Influence People by Dale Carnegie]

When in doubt, think what your past and future selves would say. [1, also there was an LW article with the prince with multiple personality disorder chaining himself to his throne that I can't find. Also, I'm not sure if I should include this because it's almost Think More.]

I urge you to comment my heuristics and add your own. One heuristic per comment. Hopefully this takes off and turns into a series if wiki pages. Edit: We should concentrate on heuristics that save time, effort, and thought.
Our society lacks good self-preservation mechanisms
by Roko — 2009-07-12T09:26:23.365Z

The prospect of a dangerous collection of existential risks and risks of major civilizational-level catastrophes in the 21st century, combined with a distinct lack of agencies whose job it is to mitigate against such risks probably indicates that the world might be in something of an emergency at the moment. Firstly, what do we mean by risks? Well, Bostrom has a paper on existential risks, and he lists the following risks as being "most likely":

Deliberate misuse of nanotechnology,

Nuclear holocaust,

Badly programmed superintelligence,

Genetically engineered biological agent,

Accidental misuse of nanotechnology (“gray goo”),

Physics disasters,

Naturally occurring disease,

Asteroid or comet impact,

Runaway global warming,

Resource depletion or ecological destruction,

Misguided world government or another static social equilibrium stops technological progress,

“Dysgenic” pressures (We might evolve into a less brainy but more fertile species, homo philoprogenitus “lover of many offspring”)

Our potential or even our core values are eroded by evolutionary development,

Technological arrest,

Take-over by a transcending upload,

Flawed superintelligence,

[Stable] Repressive totalitarian global regime,

Hanson's cosmic locusts scenario [Added by author]

To which I would add various possibilities for major civilization-level disasters that aren't existential risks, such as milder versions of all of the above, or the following:

convergence of computer viruses and cults/religions,

advanced personal weapons or surveillance devices such as nanotech, micro-UAV bugs (cyberpunk dystopia),

erosion of privacy and freedom through massively oppressive government,

highly effective meta-religions such as Scientology or a much more virulent version of modern evangelical Christianity

This collection is daunting, especially given that the human race doesn't have any official agency dedicated to mitigating risks to its own medium-long term survival. We face a long list of challenges, and we aren't even formally trying to mitigate many of them in advance, and in many past cases, mitigation of risks occurred on a last-minute, ad-hoc basis, such as individuals in the cold war making the decision not to initiate a nulcear exchange, particularly in the Cuban missile crisis.

So, a small group of people have realized that the likely outco...
Jul 12 Bay Area meetup - Hanson, Vassar, Yudkowsky
by Eliezer Yudkowsky — 2009-07-11T21:13:19.421Z

Just a reminder that the July 12th Overcoming Bias / Less Wrong meetup in Santa Clara, CA @7pm will feature Robin Hanson, Michael Vassar, and the Singularity Institute summer interns. This meetup will also take place in a larger house - there should be plenty of room to mingle.
Debate: Is short term planning in humans due to a short life or due to bias?
by  — 2009-07-11T16:34:36.230Z

One of the proposed benefits of life extension is that it will help us long term plan as we will be around in the future, so we will be more likely to care about the long term future of the world if we live longer.

So is this true? Are we rational in this respect or will the mind recoil from thinking in time scales longer than 40-60 years even when we are living hundreds of years, due to biases intrinsic to the mammalian brain.

I don't have time to research this question right now, so I thought I would experiment by throwing out this question to lesswrong and see how people treat it.