trentmkelly/LessWrong-43k huggingface dataset:
Conflicts Between Mental Subagents: Expanding Wei Dai's Master-Slave Model
by Scott Alexander — 2010-08-04T09:16:35.219Z

Related to: Alien Parasite Technical Guy, A Master-Slave Model of Human Preferences

In Alien Parasite Technical Guy, Phil Goetz argues that mental conflicts can be explained as a conscious mind (the "alien parasite”) trying to take over from an unsuspecting unconscious.
Last year, Wei Dai presented a model (the master-slave model) with some major points of departure from Phil's: in particular, the conscious mind was a special-purpose subroutine and the unconscious had a pretty good idea what it was doing1. But Wei said at the beginning that his model ignored akrasia.
I want to propose an expansion and slight amendment of Wei's model so it includes akrasia and some other features of human behavior. Starting with the signaling theory implicit in Wei's writing, I'll move on to show why optimizing for signaling ability would produce behaviors like self-signaling and akrasia, speculate on why the same model would also promote some of the cognitive biases discussed here, and finish with even more speculative links between a wide range of conscious-unconscious conflicts.
The Signaling Theory of Consciousness
This model begins with the signaling theory of consciousness. In the signaling theory, the conscious mind is the psychological equivalent of a public relations agency. The mind-at-large (hereafter called U for “unconscious” and similar to Wei's “master”) has socially unacceptable primate drives you would expect of a fitness-maximizing agent like sex, status, and survival. These are unsuitable for polite society, where only socially admirable values like true love, compassion, and honor are likely to win you friends and supporters. U could lie and claim to support the admirable values, but most people are terrible liars and society would probably notice.

So you wall off a little area of your mind (hereafter called C for “conscious” and similar to Wei's “slave”) and convince it that it has only admirable goals. C is allowed access to the speech centers. Now if anyone asks you what you value, C answers "Only admirable things like compassion and honor, of course!" and no one detects a lie because the part of the mind that's moving your mouth isn't lying.
This is a useful model because it replicates three observed features of the real world: people say they have admirable goals, they honestly believe on introspection th...
The Threat of Cryonics
by lsparrish — 2010-08-03T19:57:59.883Z

It is obvious that many people find cryonics threatening. Most of the arguments encountered in debates on the topic are not calculated to persuade on objective grounds, but function as curiosity-stoppers. Here are some common examples:

Elevated burden of proof. As if cryonics demands more than a small amount of evidence to be worth trying.

Elevated cost expectation. Thinking that cryonics is (and could only ever be) affordable only for the very rich.

Unresearched suspicions regarding the ethics and business practices of cryonics organizations.

Sudden certainty that earth-shattering catastrophes are just around the corner.

Assuming the worst about the moral attitudes of humanity's descendants towards cryonics patients.

Associations with prescientific mummification, or sci-fi that handwaves the technical difficulties.

The question is what causes this sensation that cryonics is a threat? What does it specifically threaten?

It doesn't threaten the notion that we will all die eventually. Accident, homicide, and war will remain possibilities unless we can defeat them, and suicide will always remain an option. It doesn't threaten the state, the environment, anyone's health, or any particular religion. It doesn't cost much on a large scale, doesn't generate radioactive waste or pollution, has a low carbon footprint, and is both religiously neutral and life affirming.

Rather, it seems to threaten something else, less conspicuous and more universal. This something I have termed the "Historical Death Meme", and is something which we can see influencing all human cultures throughout history. It is something we probably aren't too comfortable about leaving behind, and perhaps in fact shouldn't be. And yet, it is at least as prescientific and as hazardous as creationism.

According to the HDM, if you die the cosmetic state of your body matters. If it is grotesque, you are dishonored; if not, you are respected. Society has taken advantage of this in historical times by flaying, beheading, or hanging corpses. The idea is that despite the fact that these things have zero direct impact on a living individual who can feel them, the disgust and revulsion felt in looking at them from the outside is symbolic in some way. Likewise, when a person dies and is embalmed or cremated, the features are either restored to normal or obliviated entirely.

A second aspect of the HDM is that your...
Rationality quotes: August 2010
by Cyan — 2010-08-03T00:16:45.738Z

This is our monthly thread for collecting these little gems and pearls of wisdom, rationality-related quotes you've seen recently, or had stored in your quotesfile for ages, and which might be handy to link to in one of our discussions.

Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote comments/posts on LW/OB.

No more than 5 quotes per person per monthly thread, please.
Harry Potter and the Methods of Rationality discussion thread, part 2
by dclayh — 2010-08-01T22:58:39.413Z

ETA: There is now a third thread, so send new comments there.

Since the first thread has exceeded 500 comments, it seems time for a new one, with Eliezer's just-posted Chapter 33 & 34 to kick things off.

From previous post:

Spoiler Warning: this thread contains unrot13'd spoilers for Harry Potter and the Methods of Rationality up to the current chapter and for the original Harry Potter series. Please continue to use rot13 for spoilers to other works of fiction, or if you have insider knowledge of future chapters of Harry Potter and the Methods of Rationality.
A suggestion: mention at the top of your comment which chapter you're commenting on, or what chapter you're up to, so that people can understand the context of your comment even after more chapters have been posted. This can also help people avoid reading spoilers for a new chapter before they realize that there is a new chapter.
AI cooperation in practice
by cousin_it — 2010-07-30T16:21:50.871Z

You know that automated proof verifiers exist, right? And also that programs can know their own source code? Well, here's a puzzle for you:

Consider a program A that knows its own source code. The algorithm of A is as follows: generate and check all possible proofs up to some huge size (3^^^^3). If A finds a proof that A returns 1, it returns 1. If the search runs to the end and fails, it returns 0. What will this program actually return?

Wait, that was the easy version. Here's a harder puzzle:

Consider programs A and B that both know their own, and each other's, source code. The algorithm of A is as follows: generate and check all proofs up to size 3^^^^3. If A finds a proof that A returns the same value as B, it returns 1. If the search fails, it returns 0. The algorithm of B is similar, but possibly with a different proof system and different length limit. What will A and B return?

This second puzzle is a formalization of a Prisoner's Dilemma strategy proposed by Eliezer: "I cooperate if and only I expect you to cooperate if and only if I cooperate". So far we only knew how to make this strategy work by "reasoning from symmetry", also known as quining. But programs A and B can be very different - a human-created AI versus an extraterrestrial crystalloid AI. Will they cooperate?

I may have a tentative proof that the answer to the first problem is 1, and that in the second problem they will cooperate. But: a) it requires you to understand some logic (the diagonal lemma and Löb's Theorem), b) I'm not sure it's correct because I've only studied the subject for the last four days, c) this margin is too small to contain it. So I put it up here. I submit this post with the hope that, even though the proof is probably wrong or incomplete, the ideas may still be useful to the community, and someone else will discover the correct answers.

Edit: by request from Vladimir Nesov, I reposted the proofs to our wiki under my user page. Many thanks to all those who took the time to parse and check them.
Forager Anthropology
by WrongBot — 2010-07-28T05:48:13.761Z

(This is the second post in a short sequence discussing evidence and arguments presented by Christopher Ryan and Cacilda Jethá's Sex at Dawn, inspired by the spirit of Kaj_Sotala's recent discussion of What Intelligence Tests Miss. It covers Part II: Lust in Paradise and Part III: The Way We Weren't.)

Forager anthropology is a discipline that is easy to abuse. It relies on unreliable first-hand observations of easily misunderstood cultures that are frequently influenced by the presence of modern observers. These cultures are often exterminated or assimilated within decades of their discovery, making it difficult to confirm controversial claims and discoveries. But modern-day foraging societies are the most direct source of evidence we have about our pre-agricultural ancestors; in many ways, they are agriculture's control group, living in conditions substantially similar to the ones under which our species evolved. The standard narrative of human sexual evolution ignores or manipulates the findings of forager anthropology to support its claims, and this is no doubt responsible for much of its confused support.

Steven Pinker is one of the most prominent and well-respected advocates of the standard narrative, both on Less Wrong and elsewhere. Eliezer has referenced him as an authority on evolutionary psychology. One commenter on the first post in this series claimed that Pinker is "the only mainstream academic I'm aware of who visibly demonstrates the full suite of traditional rationalist virtues in essentially all of his writing." Another cited Pinker's claim that 20-60% of hunter-gatherer males were victims of lethal human violence ("murdered") as justification for a Malthusian view of human nature.

That 20-60% number comes from a claim about war casualties in a 2007 TED talk Pinker gave on "the myth of violence", for which he drew upon several important findings in forager anthropology. (The talk is based on an argument presented in the third chapter of The Blank Slate; there is a text version of the talk available, but it omits the material on forager anthropology that Ryan and Jethá critique.)

At 2:45 in the video Pinker displays a slide which reads

Until 10,000 years ago, humans lived as hunter-gatherers, without permanent se

...
Alien parasite technical guy
by PhilGoetz — 2010-07-27T16:51:20.688Z

Custers & Aarts have a paper in the July 2 Science called "The Unconscious Will: How the pursuit of goals operates outside of conscious awareness". It reviews work indicating that people's brains make decisions and set goals without the brains' "owners" ever being consciously aware of them.

A famous early study is Libet et al. 1983, which claimed to find signals being sent to the fingers before people were aware of deciding to move them. This is a dubious study; it assumes that our perception of time is accurate, whereas in fact our brains shuffle our percept timeline around in our heads before presenting it to us, in order to provide us with a sequence of events that is useful to us (see Dennett's Consciousness Explained). Also, Trevina & Miller repeated the test, and also looked at cases where people did not move their fingers; and found that the signal measured by Libet et al. could not predict whether the fingers would move.

Fortunately, the flaws of Libet et al. were not discovered before it spawned many studies showing that unconscious priming of concepts related to goals causes people to spend more effort pursuing those goals; and those are what Custers & Aarts review. In brief: If you expose someone, even using subliminal messages, to pictures, words, etc., closely-connected to some goals and not to others, people will work harder towards those goals without being aware of it.

This was no surprise to me. I spent the middle part of the 1990s designing and implementing a control structure for an artificial intelligence (influenced by Anderson's ACT* architecture), and it closely resembled the design that Custers & Aarts propose to explain goal priming. I had an agent with a semantic network representing all its knowledge, goals, plans, and perceptions. Whenever it perceived a change in the environment, the node representing that change got a jolt of activation, which spread to the connected concepts. Whenever it perceived an internal need (hunger, boredom), the node representing that need got a jolt of activation. Whenever it decided to pursue a subgoal, the node representing the desired goal got a jolt of activation. And when this flowing activation passed through a node representing an action that was possible at the moment, it carried out that action, modulo some magic to...
Chicago Meetup: Sunday, August 1 at 2:00 pm
by Airedale — 2010-07-27T15:10:31.611Z

We’re holding the Chicago meetup discussed here on Sunday, August 1, 2010 at 2:00 pm. The tentative location is the Corner Bakery at the corner of State and Cedar (1121 N. State St.), but we’re also happy to move the meetup further up to the North side as has been previously discussed, if anyone has a suggestion for a good venue.

We will post any updates here as well as to our Chicago LW meetup Google group. Please comment here if you plan to attend. We'll have a table-top sign to help you identify us.

We’re looking forward to a second successful Chicago meetup and hope to see some old and new faces!
Metaphilosophical Mysteries
by Wei Dai — 2010-07-27T00:55:59.222Z

Creating Friendly AI seems to require us humans to either solve most of the outstanding problems in philosophy, or to solve meta-philosophy (i.e., what is the nature of philosophy, how do we practice it, and how should we program an AI to do it?), and to do that in an amount of time measured in decades. I'm not optimistic about our chances of success, but out of these two approaches, the latter seems slightly easier, or at least less effort has already been spent on it. This post tries to take a small step in that direction, by asking a few questions that I think are worth investigating or keeping in the back of our minds, and generally raising awareness and interest in the topic.

The Unreasonable Effectiveness of Philosophy

It seems like human philosophy is more effective than it has any right to be. Why?

First I'll try to establish that there is a mystery to be solved. It might be surprising so see the words "effective" and "philosophy" together in the same sentence, but I claim that human beings have indeed made a non-negligible amount of philosophical progress. To cite one field that I'm especially familiar with, consider probability and decision theory, where we went from having no concept of probability, to studies involving gambles and expected value, to subjective probability, Bayesian updating, expected utility maximization, and the Turing-machine-based universal prior, to the recent realizations that EU maximization with Bayesian updating and the universal prior are both likely to be wrong or incomplete.

We might have expected that given we are products of evolution, the amount of our philosophical progress would be closer to zero. The reason for low expectations is that evolution is lazy and shortsighted. It couldn't possibly have "known" that we'd eventually need philosophical abilities to solve FAI. What kind of survival or reproductive advantage could these abilities have offered our foraging or farming ancestors?

From the example of utility maximizers, we also know that there are minds in the design space of minds that could be considered highly intelligent, but are incapable of doing philosophy. For example, a Bayesian expected utility maximizer programmed with a TM-based universal prior would not be able to realize that the prior is wrong. Nor would it be able to see that Bayesian updating is the wrong thing to do in some situations.

Why aren't ...
Madison meetup: Wednesday, July 28th, 6PM
by Kevin — 2010-07-26T02:11:38.809Z

We are holding a Less Wrong meetup at Indie Coffee this Wednesday the 28th at 6PM. Wednesday is waffle day at Indie.

Confirmed attendees include me, Will_Newsome, fiddlemath, and orthonormal. Expect a casual, friendly conversation. All are welcome. Really, everyone is welcome, please don't be intimidated because you don't have enough Less Wrong karma. I'll be on the road until I get to Madison and may not be checking Less Wrong regularly, so feel free to give me a call/text: 412-480-4060. Cheers.
Bay Area Events Roundup
by Eliezer Yudkowsky — 2010-07-24T06:04:42.815Z

This Saturday (i.e., the 24th, that is, tomorrow) is the peak of the Floating Festival.

Michael Vassar says: "I'm going to be speaking tomorrow (= Saturday July 24th) at 4PM at Bay Area Mensa, in Mountain View, on the scientific method, the history of science, and how to think rationally about most scientific controversies including the Singularity. Less Wrongers are invited to attend. Interested people should email David Verdirame."

The Open Science Summit is July 29-31, in Berkeley.

And as ever, the Singularity Summit approaches on August 14-15 in San Francisco. Now featuring James Randi, Irene Pepperberg, and John Tooby.
Contrived infinite-torture scenarios: July 2010
by PlaidX — 2010-07-23T23:54:46.595Z

This is our monthly thread for collecting arbitrarily contrived scenarios in which somebody gets tortured for 3^^^^^3 years, or an infinite number of people experience an infinite amount of sorrow, or a baby gets eaten by a shark, etc. and which might be handy to link to in one of our discussions. As everyone knows, this is the most rational and non-obnoxious way to think about incentives and disincentives.

Please post all infinite-torture scenarios separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

No more than 5 infinite-torture scenarios per person per monthly thread, please.
Against the standard narrative of human sexual evolution
by WrongBot — 2010-07-23T05:28:40.817Z

(This post is the beginning of a short sequence discussing evidence and arguments presented by Christopher Ryan and Cacilda Jethá's Sex at Dawn, inspired by the spirit of Kaj_Sotala's recent discussion of What Intelligence Tests Miss. It covers Part I: On the Origin of the Specious.)

Sex at Dawn: The Prehistoric Origins of Modern Sexuality was first brought to my attention by a rhapsodic mention in Dan Savage's advice column, and while it seemed quite relevant to my interests I am generally very skeptical of claims based on evolutionary psychology. I did eventually decide to pick up the book, primarily so that I could raid its bibliography for material for an upcoming post on jealousy management, and secondarily to test my vulnerability to confirmation bias. I succeeded in the first and failed in the second: Sex at Dawn is by leaps and bounds the best evolutionary psychology book I've read, largely because it provides copious evidence for its claims.1 I mention the strength of my opinion as a disclaimer of sorts, so that careful readers may take the appropriate precautions.

The book's first section focuses on the current generally accepted explanation for human sexual evolution, which the authors call "the standard narrative." It's an explanation that should be quite familiar to regular LessWrong readers: men are attracted to fertile-appearing women and try to prevent them from having sex with other men so as to confirm the paternity of their offspring; women are attracted to men who seem like they will be good providers for their children and try to prevent them from forming intimate bonds with other women so as to maintain access to their resources.

This narrative is remarkable for several reasons. In Chapter 2, Ryan and Jethá point out that it fits in neatly with much of Darwin's work, which famously drew upon Malthus and, to a lesser extent, Hobbes. The problem here is, of course, that Malthus's theory of population growth was wrong (see Michael Vassar's criticism and my reply). Like Hobbes, he looked at his society's current condition and assumed that prehistorical man lived in a similar state; the book calls this unfortunate tendency "Flintstonization" after the famously modern stone-age cartoon family. Those familiar with the heuristics and biases program may recognize this as an example of the availability heuristic.

T...
Public Choice and the Altruist's Burden
by Roko — 2010-07-22T21:34:52.740Z

The reason that we live in good times is that markets give people a selfish incentive to seek to perform actions that maximize total utility across all humans in the relevant economy: namely, they get paid for their efforts. Without this incentive, people would gravitate to choosing actions that maximized their own individual utility, finding local optima that are not globally optimal. Capitalism makes us all into efficient little utilitarians, which we all benefit enormously from.

The problem with charity, and especially efficient charity, is that the incentives for people to contribute to it are all messed up, because we don't have something analogous to the financial system for charities to channel incentives for efficient production of utility back to the producer. One effect of giving away lots of your money and effort to seriously efficient charity is that you create the counterpoint public choice problem to the special interests problem in politics. You harm a concentrated interest (friends, potential partners, children) in order to reward a diffuse interest (helping each of billions of people by a tiny amount).

The concentrated interest then retaliates, because by standard public choice theory it has an incentive to do so, but the diffuse interest just ignores you. Concretely, your friends think that you're weird and potential partners may, in the interest of their own future children, refrain from involvement with you. People in general may perceive you as being of lower status, both because of your reduced ability to signal status via conspicuous consumption if you give a lot of money away, and because of the weirdness associated with the most efficient charities.

Anyone involved in futurism, singularitarianism etc, has probably been on the sharp end of this public choice problem. Presumably, anyone in the west who donated a socially optimal amount of money to charity (i.e. almost everything) would also be on the sharp end (though I know of no cases of someone donating 99.5% of their disposable income to any charity, so we have no examples). This is the Altruist's Burden.

Evidence

Do people around you really punish you for being an altruist? This claim requires some justification.

First off, I have personal experience in this area. Not me, but someone vitally important in the existential risks movement has...