trentmkelly/LessWrong-43k huggingface dataset:
Eluding Attention Hijacks
by ABranco — 2010-04-17T03:23:46.520Z

Do my taxes? Oh, no! It’s not going to be that easy. It’s going to be different this year, I’m sure. I saw the forms—they look different. There are probably new rules I’m going to have to figure out. I might need to read all that damn material. Long form, short form, medium form? File together, file separate? We’ll probably want to claim deductions, but if we do we’ll have to back them up, and that means we’ll need all the receipts. Oh, my God—I don’t know if we really have all the receipts we’d need, and what if we didn’t have all the receipts and claimed the deductions anyway and got audited? Audited? Oh, no—the IRS—JAIL!!

And so a lot of people put themselves in jail, just glancing at their 1040 tax forms. Because they are so smart, sensitive, and creative.
—David Allen, Getting Things Done

Intro

Very recently, Roko wrote about ugh fields, “an unconscious flinch we have from even thinking about a serious personal problem. The ugh field forms a self-shadowing blind spot covering an area desperately in need of optimization, imposing huge costs.” Suggested antidotes included PJ Eby’s technique to engage with the ugh field, locate its center, and access information—thereupon dissolving the negative emotions.

I want to explore here something else that prevents us from doing what we want. Consider these situations:
Situation 1
You attack a problem that is at least slightly complex (distasteful or not), but are unable to systematically tackle it step by step because your mind keeps diverging wildly within the problem. Your brain starts running simulations and gets stuck. To make things worse, you are biased towards thinking of the worst possible scenarios. Having visualized 30 steps ahead, you panic and do nothing. David Allen's quote in the introduction of this post illustrates that.
Situation 2
You attack a problem of any complexity—anything you need to get done—and your mind keeps diverging to different directions outside the problem. Examples:
a. You decide you need to quickly send an important email before an appointment. You log in. Thirty minutes later, you find yourself watching some motivational Powerpoint presentation your uncle sent you. You stare at the inbox and can't remember what you were doing there in the first place. Yo...
Attention Lurkers: Please say hi
by Kevin — 2010-04-16T20:46:38.533Z

Some research says that lurkers make up over 90% of online groups. I suspect that Less Wrong has an even higher percentage of lurkers than other online communities.

Please post a comment in this thread saying "Hi." You can say more if you want, but just posting "Hi" is good for a guaranteed free point of karma.

Also see the introduction thread.
When is forced organ selfishness good for you?
by KatjaGrace — 2010-04-16T13:43:02.000Z

Simon Rippon claims having a market for organs might harm society:

It might first be thought that it can never be a good thing for you to have fewer rather than more options. But I believe that this attitude is mistaken on a number of grounds. For one, consider that others hold you accountable for not making the choices that are necessary in order to fulfil your obligations. As things stand, even if you had no possessions to sell and could not find a job, nobody could criticize you for failing to sell an organ to meet your rent. If a free market in body parts were permitted and became widespread, they would become economic resources like any other, in the context of the market. Selling your organs would become something that is simply expected of you when the financial need arises. A new “option” can thus easily be transformed into an obligation, and it can drastically change the attitudes that it is appropriate for others to adopt towards you in a particular context.

He’s right that at that moment where you would normally throw your hands in the air and move on, you are worse off if an organ market gives you the option of paying more debts before declaring your bankruptcy. But this is true for anything you can sell. Do we happen to have just the right number of salable possessions? By Simon’s argument people should benefit from bans on selling all sorts of things. For instance labor. People (the poor especially) are constantly forced to sell their time – such an integral part of their selves – to pay rent, and other debts they had no choice but to induce. If only they were protected from this huge obligation that we laughably call an ‘option’. Such a ban might be costly for the landlord, but it would be good for the poor people, right? No! The landlords would react and not rent to them.

So why shouldn’t we expect the opposite effect if people are allowed to sell more of their possessions? People who currently don’t have the assets or secure income to be trusted with loans or ongoing rental payments might be legitimately offered such things if they had another asset to sell. Think of all the people who would benefit from being able to mortgage their kidney to buy a car instead of riding to some closer job while they gradually save up.

In general when negotiating, it’s best to not have options that are worse for you. When the time comes to carry out your side o...
The Concepts Problem
by Kaj_Sotala — 2010-04-16T06:21:40.567Z

I'm not sure how obvious the following is to people, and it probably is obvious to most of the people thinking about FAI. But just thought I'd throw out a summary of it here anyway, since this is the one topic that makes me the most pessimistic about the notion of Friendly AI being possible. At least one based heavily on theory and not plenty of experimentation.

A mind can only represent a complex concept X by embedding it into a tightly intervowen network of other concepts that combine to give X its meaning. For instance, a "cat" is playful, four-legged, feline, a predator, has a tail, and so forth. These are the concepts that define what it means to be a cat; by itself, "cat" is nothing but a complex set of links defining how it relates to these other concepts. (As well as a set of links to memories about cats.) But then, none of those concepts means anything in isolation, either. A "predator" is a specific biological and behavioral class, the members of which hunt other animals for food. Of that definition, "biological" pertains to "biology", which is a "natural science concerned with the study of life and living organisms, including their structure, function, growth, origin, evolution, distribution, and taxonomy". "Behavior", on the other hand, "refers to the actions of an organism, usually in relation to the environment". Of those words... and so on.

It does not seem likely that humans could preprogram an AI with a ready-made network of concepts. There have been attempts to build knowledge ontologies by hand, but any such attempt is both hopelessly slow and lacking in much of the essential content. Even given a lifetime during which to work and countless of assistants, could you ever hope to code everything you knew into a format from which it was possible to employ that knowledge usefully? Even a worse problem is that the information would need to be in a format compatible with the AI's own learning algorithms, so that any new information the AI learnt would fit seamlessly to the previously-entered database. It does not seem likely that we can come up with an efficient language of thought that can be easily translated into a format that is intuitive for humans to work with.

Indeed, there are existing plans for AI systems which make the explicit assumption that the AI's network of knowledge will develop independently as the sy

...
Self-indication assumption is wrong for interesting reasons
by neq1 — 2010-04-16T04:51:23.166Z

The self-indication assumption (SIA) states that

Given the fact that you exist, you should (other things equal) favor hypotheses according to which many observers exist over hypotheses on which few observers exist.

The reason this is a bad assumption might not be obvious at first. In fact, I think it's very easy to miss.

Argument for SIA posted on Less Wrong

First, let's take a look at a argument for SIA that appeared at Less Wrong (link). Two situations are considered.

1. we imagine that there are 99 people in rooms that have a blue door on the outside (1 person per room). One person is in a room with a red door on the outside. It was argued that you are in a blue door room with probability 0.99.

2. Same situation as above, but first a coin is flipped. If heads, the red door person is never created. If tails, the blue door people are never created. You wake up in a room and know these facts. It was argued that you are in a blue door room with probability 0.99.

So why is 1. correct and 2. incorrect? The first thing we have to be careful about is not treating yourself as special. The fact that you woke up just tells you that at least one conscious observer exists.

In scenario 1 we basically just need to know what proportion of conscious observers are in a blue door room. The answer is 0.99.

In scenario 2 you never would have woken up in a room if you hadn't been created. Thus, the fact that you exist is something we have to take into account. We don't want to estimate P(randomly selected person, regardless of if they exist or not, is in a blue door room). That would be ignoring the fact that you exist. Instead, the fact that you exist tells us that at least one conscious observer exists. Again, we want to know what proportion of conscious observers are in blue door rooms. Well, there is a 50% chance (if heads landed) that all conscious observers are in blue door rooms, and a 50% chance that all conscious observers are in red door rooms. Thus, the marginal probability of a conscious observer being in a blue door room is 0.5.

The flaw in the more detailed Less Wrong proof (see the post) is when they go from step C to step D. The *you* being referred to in step A might not exist to be asked the question in step D. You have to take th...
The many faces of status
by Morendil — 2010-04-15T15:31:03.163Z

The term "status" gets used on LessWrong a lot. Google finds 316 instances; the aggregate total for the phrases "low status" and "high status" (which suggest more precision than "status" by itself) is 170. By way of comparison, "many worlds", an important topic here, yields 164 instances.
We find the term used as an explanation, for instance, "to give offense is to imply that a person or group has or should have low status". In this community I would expect that a term used often, with authoritative connotations, and offered as an explanation could be tabooed readily, for instance when someone confused by this or that use asks for clarification: previous discussions of "high status" or "low status" behaviours seemed to flounder in the particular way that definitional arguments often do.
Somewhat to my surprise, there turned out not to be a commonly understood way of tabooing "status". Lacking a satisfactory unpacking of the "status" terms and how they should control anticipation, I decided to explore the topic on my own, and my intention here is to report back and provide a basis for further discussion.
The "Status" chapter of Keith Johnstone's 1979 book "Impro", previously discussed here and on OB, is often cited as a reference on the topic (follow this link for an excerpt); I'll refer to it throughout as simply "Johnstone". Also, I plan to entirely avoid the related but distinct concept of "signaling" in this post, reserving it for later examination.

Dominance hierarchies

My initial impression was that "status" had some relation to the theory of dominance hierarchies. Section 3 of Johnstone starts with:

Social animals have inbuilt rules which prevent them killing each other for food, mates, and so on. Such animals confront each other, and often fight, until a hierarchy is established, after which there is no fighting unless an attempt is made to change the ‘pecking order’. This system is found in animals as diverse as humans, chickens, and woodlice.

This reinforced an impression I had previously acquired: that the term "alpha male", often used in certain circles synonymously with "high status male", indicated an explicit link between the theoretical underpinnings of the term "status" and some sort of dominance theory.

However, substantiating this link turned out a more frustrating task than I had expected. For instance, I looked for primar...
A LessWrong poster for the Humanity+ conference next Saturday
by Paul Crowley — 2010-04-14T21:38:46.831Z

An email from David Wood, organiser of the Humanity+ UK 2010 conference in London on Saturday 2010-04-24:

One of the rooms in Conway Hall on Sat 24th April will be set aside for posters and general socialising.

The posters are opportunities for people to publicise various activities or ideas. We've received half a dozen applications for posters so far, and we have room for one or two more.

We expect that many of the attendees will mingle in this room at lunchtime, during the afternoon break, and (for early birds) before the formal start of activities in the main hall at 9.45am.

Would one of you be interested in creating and displaying a poster about Less Wrong / Overcoming Bias?

Many of the attendees to the H+UK event will have little prior knowledge about Less Wrong, so it's a good chance to reach out to potential new supporters.

Posters can be a number of sheets of paper, stuck onto the wall with bluetack or sellotape. Maximise size in total allowed per poster is A0. Several of the posters will be A1, made up of 4 A3 sheets.

If you are interested in this, please let me know, since we have to control overall numbers of posters.

To be clear, there's no charge for this - consider it as an opportunity for free advertising :-) You're also welcome to bring small pieces of printed literature for interested people to take away.

I plan to make such a poster, and I'd like the advice of people here. Trying to represent what Less Wrong is about in the space of a poster could be challenging. I have maximum space equivalent to sixteen A4 pieces of paper. In the spirit of not proposing a solution until you've had a chance to think about the problem, I'll put my current plans into a comment.

Update: looks like it will be Roko rather than me making the poster, but the same applies, your ideas could doubtless be very useful!
How does information affect hookups?
by KatjaGrace — 2010-04-14T14:00:56.000Z

With social networking sites enabling the romantically inclined to find out more about a potential lover before the first superficial chat than they previously would have in the first month of dating, this is an important question for the future of romance.

Lets assume that in looking for partners, people care somewhat about rank and and somewhat about match. That is, they want someone ‘good enough’ for them who also has interests and personality that they like.

First look at the rank component alone. Assume for a moment that people are happy to date anyone they believe is equal to or better than them in desirability. Then if everyone has a unique rank and perfect information, there will never be any dating at all. The less information they have the more errors in comparing, so the more chance that A will think B is above her while B thinks A is above him. Even if people are willing to date people somewhat less desirable than they, the same holds – by making more errors you trade wanting more desirable people for wanting less desirable people, who are more likely to want you back , even if they are making their own errors. So to the extent that people care about rank, more information means fewer hookups.

How about match then? Here it matters exactly what people want in a match. If they mostly care about their beloved having certain characteristics, more information will let everyone hear about more people who meet their requirements. On the other hand if we mainly want to avoid people with certain characteristics, more information will strike more people off the list. We might also care about an overall average desirability of characteristics – then more information is as likely to help or harm assuming the average person is averagely desirable. Or perhaps we want some minimal level of commonality, in which case more information is always a good thing – it wouldn’t matter if you find out she is a cannibalistic alcoholic prostitute, as long as eventually you discover those board games you both like. There are more possibilities.

You may argue that you will get all the information you want in the end, the question is only speed – the hookups prevented by everyone knowing more initially are those that would have failed later anyway. However flaws that dissuade you from approaching one person with a barge pole are often ‘endeari...
Preference utilitarian measure of historical welfare
by taw — 2010-04-14T13:32:21.158Z

GDP measures essentially how good we are at making widgets - and while widgets are useful, it is a very weak and indirect measure of welfare. For example UK GDP per capita doubled between 1975 and 2007 - and people's quality of life indeed improved - but it would be extremely difficult to argue that this improvement was "doubling", and that the gap between 2007's and 1975's quality of life is greater than between 1975's and hunter-gatherer times.
It's not essential to this post, but my very quick theory is that we overestimate GDP thanks to economic equivalent of Amdahl's Law - if someone's optimal consumption mix consisted of 9 units of widgets and 1 unit of personalized services - and their purchasing power increased so now they can acquire 100x as many widgets, but still the same number of services as before - amount of the mix they can purchase increased only 9x, not 90x you'd get by weighted average of original consumption levels (and they spend 92% of their purchasing power on services now). The least scalable factor - whichever it is - will be the bottleneck.
If we're unhappy with GDP there are alternative measures like HDI, but they're highly artificial. It would be very easy to construct completely different measures which would "feel" about as right.
Fortunately there exists a very natural measure of welfare, which I haven't seen used before in this context - preference utilitarian lotteries. Would you rather live in 1700, or take a 50% chance of living in 2010 or 700? Make a list of such bets, assign numbers coherent with bet values (with 100 for highest and 0 for your lowest value) and you're done! By averaging many people's estimates we can hopefully reduce the noise, and get some pretty reasonable welfare estimates.

And now disclaimer time. This approach has countless problems, here are just a few but I'm sure you can think about more.

Probabilities are difficult - People are really bad at intuiting about a difference between 1% chance of something vs 3% of something, even though it will count for three times as much in results. We can mostly work around this problem by not comparing extremes, but instead sorting situations by desirability, and only comparing nth situation, with p chance of n+1st vs (1-p) chance of (n-1)st. Such probabilities will usually be in comfortable medium range.

Risk aversion - you prefer certainty of moderate outcome to

...
Ugh fields
by Roko — 2010-04-12T17:06:18.510Z

Tl;Dr version: Pavlovian conditioning can cause humans to unconsciously flinch from even thinking about a serious personal problem they have, we call it an "Ugh Field"1. The Ugh Field forms a self-shadowing blind spot covering an area desperately in need of optimization, imposing huge costs.

A problem with the human mind — your human mind — is that it's a horrific kludge that will fail when you most need it not to. The Ugh Field failure mode is one of those really annoying failures. The idea is simple: if a person receives constant negative conditioning via unhappy thoughts whenever their mind goes into a certain zone of thought, they will begin to develop a psychological flinch mechanism around the thought. The "Unhappy Thing" — the source of negative thoughts — is typically some part of your model of the world that relates to bad things being likely to happen to you.

A key part of the Ugh Field phenomenon is that, to start with, there is no flinch, only negative real consequences resulting from real physical actions in the problem area. Then, gradually, you begin to feel the emotional hit when you are planning to take physical actions in the problem area. Then eventually, the emotional hit comes when you even begin to think about the problem. The reason for this may be that your brain operates a temporal difference learning (TDL) algorithm. Your brain propagates the psychological pain "back to the earliest reliable stimulus for the punishment". If you fail or are punished sufficiently many times in some problem area, and acting in that area is always preceeded by thinking about it, your brain will propagate the psychological pain right back to the moment you first begin to entertain a thought about the problem, and hence cut your conscious optimizing ability right out of the loop. Related to this is engaging in a displacement activity: this is some activity that usually involves comfort, done instead of confronting the problem. Perhaps (though this is speculative) the comforting displacement activity is there to counterbalance the psychological pain that you experienced just because you thought about the problem.

For example, suppose that you started off in life with a wandering mind and were punished a few times for failing to respond to official letters. Your TDL algorithm began to propagate the pain back to the moment you looked at...
Self-modification is the correct justification for updateless decision theory
by Benya — 2010-04-11T16:39:33.270Z

Reply to: Late great filter is not bad news

Suppose that you build an AI, and Omega appears to it and says:

Here's a button. A million years ago I calculated the umpteenth digit of pi. If it is even, I calculated whether you would press this button (in such a way that your human creator was never simulated as a conscious being). If I predicted that you wouldn't press the button, I destroyed Earth right then and there.* If it is odd, I created a doomsday device that will destroy the solar system if you press this button.

[* ETA: Assume that if the digit is even and the AI is predicted to press the button, Omega does not destroy Earth, but does turn Alpha Centauri purple (say). The point is for this to be a scenario that you, the AI creator, know not to have come to pass.]

Suppose you're the kind of AI creator whose AI is time consistent in a certain sense from the beginning of time and presses the button. Then you have an AI that satisfies a certain kind of philosopher, wins big in a certain logically impossible world, and destroys humanity.

Suppose, on the other hand, that you're a very similar kind of AI creator, only you program your AI not to take into account impossible possible worlds that had already turned out to be impossible (when you created the AI | when you first became convinced that timeless decision theory is right). Then you've got an AI that most of the time acts the same way, but does worse in worlds we know to be logically impossible, and destroys humanity less often in worlds we do not know to be logically impossible.

Wei Dai's great filter post seems to suggest that under UDT, you should be the first kind of AI creator. I don't think that's true, actually; I think that in UDT, you should probably not start with a "prior" probability distribution that gives significant weight to logical propositions you know to be false: do you think the AI should press the button if it was the first digit of pi that Omega calculated?

But obviously, you don't want tomorrow's you to pick the prior that way just after Omega has appeared to it in a couterfactual mugging (because according to your best reasoning today, there's a 50% chance this loses you a million dollars).

The most convincing argument I know for timeless flavors of decision theory is that if you could modify your own source code, the course of action that maximizes your expected utility is to modif...