trentmkelly/LessWrong-43k huggingface dataset:
Is your subconscious communist?
by KatjaGrace — 2009-08-22T15:39:46.000Z

People can be hard to tell apart, even to themselves (picture: Giustino)

Humans make mental models of other humans automatically, and appear to get somewhat confused about who is who at times. This happens with knowledge, actions, attention and feelings:

Just having another person visible hinders your ability to say what you can see from where you stand, though considering a non-human perspective does not:

[The] participants were also significantly slower in verifying their own perspective when the avatar’s perspective was incongruent. In Experiment 2, we found that the avatar’s perspective intrusion effect persisted even when participants had to repeatedly verify their own perspective within the same block. In Experiment 3, we replaced the avatar by a bicolor stick …[and then] the congruency of the local space did not influence participants’ response time when they verified the number of circles presented in the global space.

Believing you see a person moving can impede you in moving differently, similar to rubbing your tummy while patting your head, but if you believe the same visual stimulus is not caused by a person, there is no interference:

[A] dot display followed either a biologically plausible or implausible velocity profile. Interference effects due to dot observation were present for both biological and nonbiological velocity profiles when the participants were informed that they were observing prerecorded human movement and were absent when the dot motion was described as computer generated…

Doing a task where the cues to act may be incongruent with the actions (a red pointer signals that you should press the left button, whether the pointer points left or right, and a green pointer signals right), the incongruent signals take longer to respond to than the congruent ones. This stops when you only have to look after one of the buttons. But if someone else picks up the other button, it becomes harder once again to do incongruent actions:

The identical task was performed alone and alongside another participant. There was a spatial compatibility effect in the group setting only. It was similar to the effect obtained when one person took care of both responses. This result suggests that one’s own actions and others’ actions are represented in a functionally equivalent way.

You can learn to subconsciously fear a stimulus by seeing the stim...
ESR's New Take on Qualia
by billswift — 2009-08-21T09:26:24.196Z

http://esr.ibiblio.org/?p=1192#more-1192

ADDED: Even if you disagree with ESR's take, and many will, this is the clearest definition I have seen on what qualia is. So it should present a useful starting point, even for those who strongly disagree, to argue from.
Timeless Decision Theory and Meta-Circular Decision Theory
by Eliezer Yudkowsky — 2009-08-20T22:07:46.662Z

(This started as a reply to Gary Drescher's comment here in which he proposes a Metacircular Decision Theory (MCDT); but it got way too long so I turned it into an article, which also contains some amplifications on TDT which may be of general interest.)

Part 1: How timeless decision theory does under the sort of problems that Metacircular Decision Theory talks about.

Say we have an agent embodied in the universe. The agent knows some facts about the universe (including itself), has an inference system of some sort for expanding on those facts, and has a preference scheme that assigns a value to the set of facts, and is wired to select an action--specifically, the/an action that implies (using its inference system) the/a most-preferred set of facts.

But without further constraint, this process often leads to a contradiction. Suppose the agent's repertoire of actions is A1, ...An, and the value of action Ai is simply i. Say the agent starts by considering the action A7, and dutifully evaluates it as 7. Next, it contemplates the action A6, and reasons as follows: "Suppose I choose A6. I know I'm a utility-maximizing agent, and I already know there's another choice that has value 7. Therefore, if follows from my (hypothetical) choice of A6 that A6 has a value of at least 7." But that inference, while sound, contradicts the fact that A6's value is 6.

This is why timeless decision theory is a causality-based decision theory. I don't recall if you've indicated that you've studied Pearl's synthesis of Bayesian networks and causal graphs(?) (though if not you should be able to come up to speed on them pretty quickly).

So in the (standard) formalism of causality - just causality, never mind decision theory as yet - causal graphs give us a way to formally compute counterfactuals: We set the value of a particular node surgically. This means we delete the structural equations that would ordinarily give us the value at the node N_i as a function of the parent values P_i and the background uncertainty U_i at that node (which U_i must be uncorrelated to all other U, or the causal graph has not been fully factored). We delete this structural equation for N_i and make N_i parentless, so we don't send any likelihood messages up to the former parents when we update our knowledge of the value at N_i. However, we do send prior-messages from N_i...
How inevitable was modern human civilization - data
by taw — 2009-08-20T21:42:41.869Z

We have a sample of one modern human civilization, but there are some hints on how likely it was to happen.
Major types of hints are:

Time - if something happened extremely quickly; or extremely late, it suggests how likely it was.

Independent invention - something that was invented independently multiple times is likelier; something invented only once in spite of plenty of time, isolation, and prerequisites is less likely.

Data for:

Life seems to have developed extremely quickly after creation of Earth. [Origin of life]

Multicellularity seems to have evolved multiple times independently, at least in animals, fungi, and plants. [Evolution of multicellularity]

Similar process also happened multiple time on higher level - eusociality developed in aphids, thrips, mole rats, termites, and at least 11 times in Hymenoptera (ants, bees, and wasps). [Eusociality]

Life did not die out on Earth, or on any particular environment where it previously thrived, in spite of major changes in temperature, composition of atmosphere, and multiple large scale disasters. This suggests life is very resilient. Every time life is wiped out in some part of Earth, it is quickly recolonized.

Many different lineages of animals developed societies. [Social animal]

Many different lineages of animals developed communication. [Animal communication]

All transitions from Middle Paleolithic onwards happened relatively fast to extremely fast on evolutionary scale. [Paleolithic]

Invention of Mesolithic and Neolithic culture including agriculture, bow, boats, animal husbandry, pottery were all invented multiple times independently, in Afroeurasia, and Americas. [Stone Age]

Likewise many of latter inventions including metallurgy, writing, money, and state were developed multiple times independently.

Data against:

Universe is not filled with technical civilizations. Some (dubious due to zero empirical evidence) models suggest once such civilization develops anywhere in the galaxy, it is very likely to colonize the entire galaxy in relatively short period of time. As it didn't happen, it's a strong evidence that there are very few, perhaps no, advanced technical civilizations in our galaxy; or anywhere else in the universe if our galaxy is a good representative. [Fermi paradox]

Life can survive in a very wide range of circumstances, so there are plenty of places where we might expect to find li

...
Evolved Bayesians will be biased
by taw — 2009-08-20T14:54:18.626Z

I have a small theory which strongly implies that getting less biased is likely to make "winning" more difficult.

Imagine some sort of evolving agents that follow vaguely Bayesianish logic. They don't have infinite resources, so they use a lot of heuristics, not direct Bayes rule with priors based on Kolmogorov complexity. Still, they employ a procedure A to estimate what the world is like based on data available, and a procedure D to make decisions based on their estimations, both of vaguely Bayesian kind.

Let's be kind to our agents and grant that for every possible data and every possible decision they might have encountered in their ancestral environment, they make exactly the same decision as an ideal Bayesian agent would. A and D have been fine-tuned to work perfectly together.

That doesn't mean that either A or D are perfect even within this limited domain. Evolution wouldn't care about that at all. Perhaps different biases within A cancel each other. For example an agent might overestimate snakes' dangerousness and also overestimate his snake-dodging skills - resulting in exactly the right amount of fear of snakes.

Or perhaps a bias in A cancels another bias in D. For example an agent might overestimate his chance of success at influencing tribal policy, what neatly cancels his unreasonably high threshold for trying to do so.

And then our agents left their ancestral environment, and found out that for some of the new situations their decisions aren't that great. They thought about it a lot, noticed how biased they are, and started a website on which they teach each other how to make their A more like perfect Bayesian's A. They even got quite good at it.

Unfortunately they have no way of changing their D. So biases in their decisions which used to neatly counteract biases in their estimation of the world now make them commit a lot of mistakes even in situations where naive agents do perfectly well.

The problem is that for virtually every A and D pair that could have possibly evolved, no matter how good the pair is together, neither A nor D would be perfect in isolation. In all likelihood both A and D are ridiculously wrong, just in a special way that never hurts. Improving one without improving the other, or improving just part of either A or D, will lead to much worse decisions, even if your idea of what the world is like gets better.

I think humans might be...
You have just been Counterfactually Mugged!
by CronoDAS — 2009-08-19T22:24:38.805Z

I'm going to test just how much the people here are committed to paying a Counterfactual Mugger, by playing Omega.

I'm going to roll a die. If it doesn't come up 5 or 6, I'm going to ask Eliezer Yudkowsky to reply to this article with the comment "I am a poopy head." If I roll a 5 or 6, I'm going to donate $20 to SIAI if I predict that Eliezer Yudkowsky will post the above comment.

Because Eliezer has indicated that he would pay up when counterfactually mugged, I do predict that, if I roll a 5 or 6, he'll respond.

::rolls die::

Darn it! It's a 5. Well, I'm a man of my word, so...

::donates::

Um, let's try that again. (At least I've proven my honesty!)

::rolls die::

Okay, this time it's a 1.

So, Eliezer, will you post a comment admitting that you're a poopy head?
Ingredients of Timeless Decision Theory
by Eliezer Yudkowsky — 2009-08-19T01:10:11.862Z

Followup to: Newcomb's Problem and Regret of Rationality, Towards a New Decision Theory

Wei Dai asked:

"Why didn't you mention earlier that your timeless decision theory mainly had to do with logical uncertainty? It would have saved people a lot of time trying to guess what you were talking about."

...

All right, fine, here's a fast summary of the most important ingredients that go into my "timeless decision theory". This isn't so much an explanation of TDT, as a list of starting ideas that you could use to recreate TDT given sufficient background knowledge. It seems to me that this sort of thing really takes a mini-book, but perhaps I shall be proven wrong.

The one-sentence version is: Choose as though controlling the logical output of the abstract computation you implement, including the output of all other instantiations and simulations of that computation.

The three-sentence version is: Factor your uncertainty over (impossible) possible worlds into a causal graph that includes nodes corresponding to the unknown outputs of known computations; condition on the known initial conditions of your decision computation to screen off factors influencing the decision-setup; compute the counterfactuals in your expected utility formula by surgery on the node representing the logical output of that computation.

To obtain the background knowledge if you don't already have it, the two main things you'd need to study are the classical debates over Newcomblike problems, and the Judea Pearl synthesis of causality. Canonical sources would be "Paradoxes of Rationality and Cooperation" for Newcomblike problems and "Causality" for causality.

For those of you who don't condescend to buy physical books, Marion Ledwig's thesis on Newcomb's Problem is a good summary of the existing attempts at decision theories, evidential decision theory and causal decision theory. You need to know that causal decision theories two-box on Newcomb's Problem (which loses) and that evidential decision theories refrain from smoking on the smoking lesion problem (which is even crazier). You need to know that the expected utility formula is actually over a counterfactual on our actions, rather than an ordinary probability update on our actions.

I'm not sure what you'd use for online reading on causality. Mainly you need to know:

That a causal graph fac

...
Scott Aaronson's "On Self-Delusion and Bounded Rationality"
by cousin_it — 2009-08-18T19:17:07.019Z

Poignant short story about truth-seeking that I just found. Quote:

"No," interjected an internal voice. "You need to prove that your dad will appear by a direct argument from the length of your nails, one that does not invoke your subsisting in a dream state as an intermediate step."

"Nonsense," retorted another voice. "That we find ourselves in a dream state was never assumed; rather, it follows so straightforwardly from the long-nail counterfactual that the derivation could be done, I think, even in an extremely weak system of inference."

The full thing reads like a flash tour of OB/LW, except it was written in 2001.
Singularity Summit 2009 (quick post)
by Eliezer Yudkowsky — 2009-08-16T23:29:40.282Z

Someone else may do a more formal announcement later, but since early registration expires on August 20th, I'm doing a quick heads-up to Less Wrong readers:

The Singularity Summit 2009 is in New York on Oct 3-4.

There are discounts for students, blog mentions, referrals, and registration before August 20th.

Speakers of note to rationalists will include Robin Hanson, Gary Drescher (author of Good and Real, one of the few master-level works of reductionism out there), and David Chalmers. Also speaking will be Marcus Hutter and Juergen Schmidhuber, as well as some of the usual suspects: Aubrey de Grey, Peter Thiel, Ben Goertzel, and Ray Kurzweil.

They're really trying to raise the intellectual level this year.

Singularity Summit 2009 home page, program, and registration.
Friendlier AI through politics
by Jonathan_Graehl — 2009-08-16T21:29:56.353Z

David Brin suggests that some kind of political system populated with humans and diverse but imperfectly rational and friendly AIs would evolve in a satisfactory direction for humans.

I don't know whether creating an imperfectly rational general AI is any easier, except that limited perceptual and computational resources obviously imply less than optimal outcomes; still, why shouldn't we hope for optimal given those constraints? I imagine the question will become more settled before anyone nears unleashing a self-improving superhuman AI.

An imperfectly friendly AI, perfectly rational or not, is a very likely scenario. Is it sufficient to create diverse singleton value-systems (demographically representative of humans' values) rather than a consensus (over all humans' values) monolithic Friendly?

What kind of competitive or political system would make fragmented squabbling AIs safer than an attempt to get the monolithic approach right? Brin seems to have some hope of improving politics regardless of AI participation, but I'm not sure exactly what his dream is or how to get there - perhaps his "disputation arenas" would work if the participants were rational and altruistically honest).
Experiential Pica
by Alicorn — 2009-08-16T21:23:10.050Z

tl;dr version: Akrasia might be like an eating disorder!

When I was a teenager, I ate ice. Lots of ice. Cups and cups and cups of ice, constantly, all day long, when it was freely available. This went on for years, during which time I ignored the fact that others found it peculiar. ("Oh," I would joke to curious people at the school cafeteria, ignoring the opportunity to detect the strangeness of my behavior, "it's for my pet penguin.") I had my cache of excuses: it keeps my mouth occupied. It's so nice and cool in the summer. I don't drink enough water anyway, it keeps me hydrated. Yay, zero-calorie snack!

Then I turned seventeen and attempted to donate blood, and was basically told, when they did the finger-stick test, "Either this machine is broken or you should be in a dead faint." I got some more tests done, confirmed that extremely scary things were wrong with my blood, and started taking iron supplements. I stopped eating ice. I stopped having any interest in eating ice at all.

Pica is an impulse to eat things that are not actually food. Compared to some of the things that people with pica eat, I got off very easy: ice did not do me any harm on its own, and was merely a symptom. But here's the kicker: What I needed was iron. If I'd been consciously aware of that need, I'd have responded to it with the supplements far earlier, or with steak1 and spinach and cereals fortified with 22 essential vitamins & minerals. Ice does not contain iron. And yet when what I needed was iron, what I wanted was ice.

What if akrasia is experiential pica? What if, when you want to play Tetris or watch TV or tat doilies instead of doing your Serious Business, that means that you aren't going to art museums enough, or that you should get some exercise, or that what your brain really craves is the chance to write a symphony?

The existence - indeed, prevalence - of pica is a perfect example of how the brain is very bad at communicating certain needs to the systems that can get those needs met. Even when the same mechanism - that of instilling the desire to eat something, in the case of pica - could be used to meet the need, the brain misfires2. It didn't make me crave liver and shellfish and molasses, it made me crave water in frozen form. A substance which did not...
Happiness is a Heuristic
by  — 2009-08-16T21:12:39.466Z

Whenever the topic of happiness is mentioned, it's always discussed like it's the most important thing in the world. People talk about it like they would a hidden treasure or a rare beast - you have to seek it, hunt it, ensnare it and hold it tight, or it'll slip through your fingers. Perhaps it's just the contrarian in me, but this seems misguided - happiness shouldn't be searched for like the holy grail. Not that I don't want to be happy, but is that really the purpose of my life - to have my neurons stimulated in a way that feels good, and try to keep that up until I die? Why don't I just slip myself into a Soma-coma then? Of course, anything I do boils down to a particular stimulation of neurons, but that doesn't mean there's not something better to aspire to. To pursue happiness as an end itself I think, is backwards. It wasn't built into our brains because evolution was being nice - it's there because it increases our fitness. Happiness is designed to get us somewhere, not to be a destination in itself.

Fortunately, we're not obligated to follow evolution's whims. But this confusion might be the reason that, in general, we're crappy at predicting what will make us happy. A few data points:

-In an oft-quoted study, lottery winners are less happy than they predicted a year afterwards, and parapalegics are more happy.[1]

-People tend to enjoy potato chips the same amount despite predicting (after being primed) that they'll either love them or hate them.[2]

-Both assistant professors who do and do not receive tenure over-emphasize the impact the result will have on their happiness.

Our wanting system is seperate from our liking system[3], and, contrary to intuition, we don't seem to be wired to pursue things that will make us happy. Rather, we seem to be wired to pursue things, and then get a fixed amount of happiness from whatever we end up with. We try to get what we like, but we tend to end up liking what we get. Dan Gilbert refers to this as 'synthetic' happiness, and posits it as the reason people who miss big opportunities tend to say that "it worked out for the best." It's tempting to assume this is just a rationalization technique, but there's evidence it happens on a subconscious level. When amnesiacs are given a painting as a gift, later they tend to pick it as their favorite out of a group of paintings, despite not being able to remember o

...
My God! It's full of Nash equilibria!
by Cyan — 2009-08-16T19:59:33.959Z

Speaking of Scott Aaronson, his latest post at Shtetl-Optimized seems worthy of some linky love.

Why do native speakers of the language you’re studying talk too fast for you to understand them? Because otherwise, they could talk faster and still understand each other.

...

Again and again, I’ve undergone the humbling experience of first lamenting how badly something sucks, then only much later having the crucial insight that its not sucking wouldn’t have been a Nash equilibrium. Clearly, then, I haven’t yet gotten good enough at Malthusianizing my daily life—have you?
Bloggingheads: Yudkowsky and Aaronson talk about AI and Many-worlds
by Vladimir_Nesov — 2009-08-16T16:06:18.646Z

Eliezer Yudkowsky and Scott Aaronson - Percontations: Artificial Intelligence and Quantum Mechanics

Sections of the diavlog:

When will we build the first superintelligence?

Why quantum computing isn’t a recipe for robot apocalypse

How to guilt-trip a machine

The evolutionary psychology of artificial intelligence

Eliezer contends many-worlds is obviously correct

Scott contends many-worlds is ridiculous (but might still be true)