trentmkelly/LessWrong-43k huggingface dataset:
Why is medical advice all caution and no info?
by KatjaGrace — 2010-08-24T12:48:37.000Z

Wordpress now offers me free images of whatever I'm talking about. This is presumably an illustration of how the world doesn't necessarily become a better place if you tell everyone they urgently need to seek medical help for something you won't tell them anything more about. Image by uncultured via Flickr

I had a couple of bad looking medical test results in a row, so I was sent to a specialist, with advice along the lines of ‘well, we can’t say it’s not cancer… probably get checked out as soon as possible’. When I eventually got to the specialist he immediately told me a bunch of relevant conditional probabilities: of any problem at all given such test results, of it being a bad problem given it’s some kind of problem, the probability per year of cancer given each kind of problem. These were not scary numbers at all. Given that it can take months to extract your data from one doctor and get an appointment with a specialist, it would have been very nice to have been told these numbers by the original doctor, instead of just knowing for a few months that such results are some unknown degree of evidence in favor of cancer. Is this just an oversight by a particular useless doctor?

It seems not. I’ve noticed another two examples of the same problem in medicine recently. If you look up symptoms online, you will often be told to seek emergency medical assistance immediately. It often doesn’t even tell you what the potential problem is, and certainly not what the odds are of it occurring, so it’s pretty hard to evaluate the suggestion. If you actually go to a doctor about one of these symptoms, the doctor often tells you not to worry about it without more than knowledge of your age. Often the website also knows your age, or at least asked it, and it would be simple for them to mention that said symptom is only a concern if you are over fifty, or even just the basic information about how common such a thing is given that symptom. Similarly my region has a free health phone line where you can ask a nurse whether symptoms are worth bothering to go to a doctor about. That seems like a decent idea, since apparently people overestimate when its worth going to the doctor. However in my small amount of experimentation it seems that anything I say prompts the suggestion that I see a doctor ‘within four hours’. I mean, I have tried telling them my bottom hu...
The Smoking Lesion: A problem for evidential decision theory
by  — 2010-08-23T09:01:21.260Z

This is part of a sequence titled "An introduction to decision theory". The previous post was Newcomb's Problem: A problem for Causal Decision Theories

For various reasons I've decided to finish this sequence on a seperate blog. This is principally because there were a large number of people who seemed to feel that this sequence either wasn't up to the Less Wrong standard or felt that it was simply covering ground that had already been covered on Less Wrong.

The decision to post it on another blog rather than simply discontinuing it came down to the fact that other people seemed to feel that the sequence had value. Those people can continue reading it at "The Smoking Lesion: A problem for evidential decision theory".

Alternatively, there is a sequence index available: Less Wrong and decision theory: sequence index
Who are we?
by KatjaGrace — 2010-08-23T00:35:55.000Z

I wonder if part of the reason for persistent disagreement on political positions is that people mean quite different things by ‘We’ in sentences like ‘We should do x’. Here are three:

We = ‘the government’

As in, ‘we should control markets to avoid the dangers of their extremes’, ‘we should have discretion in the treatment of prisoners to allow for the complexities of the situation’, ‘we should ban smoking even though people want to smoke, because they won’t when they stop being addicted’, and ‘we should censor especially harmful writing’.

This is interesting because if ‘we’ should do these things, naturally ‘we’ should be given the power to do them. However in practice since you aren’t actually in the government, what you think ‘we’ should do is not very relevant once you have allowed such power. This is especially the case with issues where you can’t easily check that ‘we’ are doing what ‘we’ should, or do anything about it. For instance issues which prohibit your knowing what’s going on (e.g. censorship), or where good and bad actions would be hard to tell apart from the outside (for instance well justified paternalism and interest-driven paternalism), issues which involve no simple standards to check behavior against (where much discretion is allowed it is hard to claim particular decisions were wrong, or to show this to others), and issues where you are expected to disagree (for instance paternalistic laws). By calling the government ‘we’ it’s easy to forget the difference in effort required to do something and to check a large powerful organization elsewhere is doing something.

We = ‘everyone’

As in, ‘if we just cut our meat consumption in half we would cut carbon emissions by –%’

This one is interesting for similar reasons to the above; it makes it extremely easy to overlook the fact that you don’t make decisions for everyone else, and don’t know what they are doing mostly. ‘Just cutting our meat consumption in half’ requires somehow persuading perhaps billions of others to reduce their meat consumption, despite their other priorities, disagreement with the claim, lack of sympathy to the cause, inability to hear you o...
Justifying Induction
by Unknowns — 2010-08-22T11:13:29.656Z

Related to: Where Recursive Justification Hits Bottom, Priors as Mathematical Objects, Probability is Subjectively Objective

Follow up to: A Proof of Occam's Razor

In my post on Occam’s Razor, I showed that a certain weak form of the Razor follows necessarily from standard mathematics and probability theory. Naturally, the Razor as used in practice is stronger and more concrete, and cannot be proven to be necessarily true. So rather than attempting to give a necessary proof, I pointed out that we learn by induction what concrete form the Razor should take.

But what justifies induction? Like the Razor, some aspects of it follow necessarily from standard probability theory, while other aspects do not.

Suppose we consider the statement S, “The sun will rise every day for the next 10,000 days,” assigning it a probability p, between 0 and 1. Then suppose we are given evidence E, namely that the sun rises tomorrow. What is our updated probability for S? According to Bayes’ theorem, our new probability will be:

P(S|E) = P(E|S)P(S)/P(E) = p/P(E), because given that the sun will rise every day for the next 10,000 days, it will certainly rise tomorrow. So our new probability is greater than p. So this seems to justify induction, showing it to work of necessity. But does it? In the same way we could argue that the probability that “every human being is less than 10 feet tall” must increase every time we see another human being less than 10 feet tall, since the probability of this evidence (“the next human being I see will be less than 10 feet tall”), given the hypothesis, is also 1. On the other hand, if we come upon a human being 9 feet 11 inches tall, our subjective probability that there is a 10 foot tall human being will increase, not decrease. So is there something wrong with the math here? Or with our intuitions?

In fact, the problem is neither with the math nor with the intuition. Given that every human being is less than 10 feet tall, the probability that “the next human being I see will be less than 10 feet tall” is indeed 1, but the probability that &ldq

...
Consciousness of simulations & uploads: a reductio
by simplicio — 2010-08-21T20:02:20.067Z

Related articles: Nonperson predicates, Zombies! Zombies?, & many more.

ETA: This argument appears to be a rehash of the Chinese room, which I had previously thought had nothing to do with consciousness, only intelligence. I nonetheless find this one instructive in that it makes certain things explicit which the Chinese room seems to gloss over.

ETA2: I think I may have made a mistake in this post. That mistake was in realizing what ontology functionalism would imply, and thinking that ontology too weird to be true. An argument from incredulity, essentially. Double oops.

Consciousness belongs to a class of topics I think of as my 'sore teeth.' I find myself thinking about them all the time: in the middle of bathing, running, cooking. I keep thinking about consciousness because no matter how much I read on the subject, I find I am still confused.

Now, to the heart of the matter. A major claim on which the desirability of uploading (among other things) depends, is that the upload would be conscious (as distinct from intelligent). I think I found a reductio of this claim at about 4:00 last night while staring up at my bedroom ceiling.

Simulating a person

The thought experiment that is supposed to show us that the upload is conscious goes as follows. (You can see an applied version in Eliezer's bloggingheads debate with Massimo Pigliucci, here. I also made a similar argument to Massimo here.)

Let us take an unfortunate member of the public, call her Simone, and simulate her brain (plus inputs and outputs along the nervous system) on an arbitrarily powerful philosophical supercomputer (this also works if you simulate her whole body plus surroundings). This simulation can be at any level of complexity you like, but it's probably best if we stick to an atom-by-atom (or complex amplitudes) approach, since that leaves less room for doubt.

Since Simone is a lawful entity within physics, there ought to be nothing in principle stopping us from doing so, and we should get behavioural isomorphism between the simulation and the biological Simone.

Now, we can also simulate inputs and outputs to and from the visual, auditory and language regions of her brain. It follows that with the right expertise, we can ask her questions - questions like "Are you experiencing the subjective feeling of consciousness you had when you were in a biological body?" - and get answers.

I'm almo...
Rationality Lessons in the Game of Go
by GreenRoot — 2010-08-21T14:33:03.391Z

There are many reasons I enjoy playing go: complex gameplay arises out of simple rules, single mistakes rarely decide games, games between between people of different skill can be handicapped without changing the dynamics of the game too much, there are no draws, and I just like the way it looks. The purpose of this article is to illustrate something else I like about playing go: the ways that it provides practice in basic habits of rationality, that is, the ways in which playing go helps me be less wrong.

I've tried to write this so that you don't need to know the game to follow it, but reading a quick introduction would probably help. (ETA: A commenter below has helpfully pointed to more go info online.) The main aspect to understand for this article is that go is a game of territory. The two sides vie to occupy space and surround one another. If a group of stones is surrounded without sufficient internal space to support itself, it is killed and removed from the board.

Lesson 1: Having accurate beliefs matters.

Here are three examples of a group of white stones being surrounded by black stones. The important distinction between them is whether the white stones will eventually be captured, i.e. whether they are "dead" or "alive".

Fig 1: The white stones are dead.

Fig 2: The white stones are alive

Fig 3: The white stones have ambiguous status.

In Figure 1, the white stones are being smothered by the black stones. They are overwhelmed and cannot escape eventual capture. They are "dead".

In Figure 2, the white stones are surrounded but have sufficient structure to protect their internal space and thus will not eventually be capture by the black stones. They are "alive".

In Figure 3, the issue is unsettled. Depending on how play proceeds, the white stones may eventually live or may eventually die.

Accurately determining whether the white stones in situations like Figure 3 are doomed or not is essential to winning go. There are go training books that consist of nothing but hundreds of exercises in correctly assessing the status of stones and learning the subtleties of how their arrangement affects that status. Correct assessment of life and death is one of things besides a large branching factor that makes computer go so hard.

I once read something on LessWrong which was later labeled The Fundam...
Transparency and Accountability
by multifoliaterose — 2010-08-21T13:01:24.750Z

[Added 02/24/14: After writing this post, I discovered that I had miscommunicated owing to not spelling out my thinking in sufficient detail, and also realized that it carried unnecessary negative connotations (despite conscious effort on my part to avoid them). See Reflections on a Personal Public Relations Failure: A Lesson in Communication. SIAI (now MIRI) has evolved substantially since 2010 when I wrote this post, and the criticisms made in the post don't apply to MIRI as presently constituted.]

Follow-up to: Existential Risk and Public Relations, Other Existential Risks, The Importance of Self-Doubt

Over the last few days I've made a string of posts levying strong criticisms against SIAI. This activity is not one that comes naturally to me. In The Trouble With Physics Lee Smolin writes

...it took me a long time to decide to write this book. I personally dislike conflict and confrontation [...] I kept hoping someone in the center of string-theory research would write an objective and detailed critique of exactly what has and has not been acheived by the theory. That hasn't happened.

My feelings about and criticisms of SIAI are very much analogous to Smolin's feelings about and criticisms of string theory. Criticism hurts feelings and I feel squeamish about hurting feelings. I've found the process of presenting my criticisms of SIAI emotionally taxing and exhausting. I fear that if I persist for too long I'll move into the region of negative returns. For this reason I've decided to cut my planned sequence of posts short and explain what my goal has been in posting in the way that I have.

Edit: Removed irrelevant references to VillageReach and StopTB, modifying post accordingly.

As Robin Hanson never ceases to emphasize, there's a disconnect between what humans say that what they're trying to do and what their revealed goals are. Yvain has written about this topic recently under his posting Conflicts Between Mental Subagents: Expanding Wei Dai's Master-Slave Model. This problem becomes especially acute in the domain of philanthropy. Three quotes on this point:

(1) In Public Choice and the Altruist's Burden Roko says:

The reason that we live in good times is that markets give people a selfish incentive to seek to perform actions that maximize total utility across all humans in the relevant economy: namely, they get paid

...
The Importance of Self-Doubt
by multifoliaterose — 2010-08-19T22:47:15.149Z

[Added 02/24/14: After I got feedback on this post, I realized that it carried unnecessary negative connotations (despite conscious effort on my part to avoid them), and if I were to write it again, I would have framed things differently. See Reflections on a Personal Public Relations Failure: A Lesson in Communication for more information. SIAI (now MIRI) has evolved substantially since 2010 when I wrote this post, and the criticisms made in the post don't apply to MIRI as presently constituted.]

Follow-up to: Other Existential Risks, Existential Risk and Public Relations

Related to: Tsuyoku Naritai! (I Want To Become Stronger), Affective Death Spirals, The Proper Use of Doubt, Resist the Happy Death Spiral, The Sin of Underconfidence

In Other Existential Risks I began my critical analysis of what I understand to be SIAI's most basic claims. In particular I evaluated part of the claim

(1) At the margin, the best way for an organization with SIAI's resources to prevent global existential catastrophe is to promote research on friendly Artificial Intelligence, work against unsafe Artificial Intelligence, and encourage rational thought.

It's become clear to me that before I evaluate the claim

(2) Donating to SIAI is the most cost-effective way for charitable donors to reduce existential risk.

I should (a) articulate my reasons for believing in the importance of self-doubt and (b) give the SIAI staff an opportunity to respond to the points which I raise in the present post as well as my two posts titled Existential Risk and Public Relations and Other Existential Risks.

Yesterday SarahC described to me how she had found Eliezer's post Tsuyoku Naritai! (I Want To Become Stronger) really moving. She explained:

I thought it was good: the notion that you can and must improve yourself, and that you can get farther than you think.

I'm used to the other direction: "humility is the best virtue."

I mean, this is a big fuck-you to the book of Job, and it appeals to me.

I was happy to learn that SarahC had been positively affected by Eliezer's post. Self-actualization is a wonderful thing and it appears as though Eliezer's posting has helped her self-actualize. On the other hand, rereading the post prompted me to notice that there's something about it which I find very problematic. The last few paragraphs of the post read:

Take no pride in y

...
Positioning oneself to make a difference
by Mitchell_Porter — 2010-08-18T23:54:38.901Z

Last weekend, while this year's Singularity Summit took place in San Francisco, I was turning 40 in my Australian obscurity. 40 is old enough to be thinking that I should just pick a SENS research theme and work on it, and also move to wherever in the world is most likely to have the best future biomedicine (that might be Boston). But at least since the late 1990s, when Eliezer first showed up, I have perceived that superintelligence trumps life extension as a futurist issue. And since 2006, when I first grasped how something like CEV could be an answer to the problem of superintelligence, I've had it before me as a model of how the future could and should play out. I have "contrarian" ideas about how consciousness works, but they do not contradict any of the essential notions of seed AI and friendly AI; they only imply that those notions would need to be adjusted and fitted to the true ontology, whatever that may be.

So I think this is what I should be working on - not just the ontological subproblem, but all aspects of the problem. The question is, how to go about this. At the moment, I'm working on a lengthy statement of how I think a Friendly Singularity could be achieved - a much better version of my top-level posts here, along with new material. But the main "methodological" problem is economic and perhaps social - what can I live on while I do this, and where in the world and in society should I situate myself for maximum insight and productivity. That's really what this post is about.

The obvious answer is, apply to SIAI. I'm not averse to the idea, and on occasion I raise the possibility with them, but I have two reasons for hesitation.

The first is the problem of consciousness. I often talk about this in terms of vaguely specified ideas about quantum entanglement in the brain, but the really important part is the radical disjunction between the physical ontology of the natural sciences and the manifest nature of consciousness. I cannot emphasize enough that this is a huge gaping hole in the scientific understanding of the world, the equal of any gap in the scientific worldview that came before it, and that the standard "scientific" way of thinking about it is a form of property dualism, even if people won't admit this to themselves. All the quantum stuff you hear from me is just an idea about how to restore a type of monism. I actually think it's a co...
Transhumanism and the denotation-connotation gap
by PhilGoetz — 2010-08-18T15:33:20.529Z

A word's denotation is our conscious definition of it. You can think of this as the set of things in the world with membership in the category defined by that word; or as a set of rules defining such a set. (Logicians call the former the category's extension into the world.)

A word's connotation can mean the emotional coloring of the word. AI geeks may think of it as a set of pairs, of other concepts that get activated or inhibited by that word, and the changes to the odds of recalling each of those concepts.

When we think analytically about a word - for instance, when writing legislation - we use its denotation. But when we are in values/judgement mode - for instance, when deciding what to legislate about, or when voting - we use its denotation less and its connotation more.

This denotative-connotative gap can cause people to behave less rationally when they become more rational. People who think and act emotionally are at least consistent. Train them to think analytically, and they will choose goals using connotation but pursue them using denotation. That's like hiring a Russian speaker to manage your affairs because he's smarter than you, but you have to give him instructions via Google translate. Not always a win.

Consider the word "human". It has wonderful connotations, to humans. Human nature, humane treatment, the human condition, what it means to be human. Often the connotations are normative rather than descriptive; behaviors we call "inhumane" are done only by humans. The denotation is bare by comparison: Featherless biped. Homo sapiens, as defined by 3 billion base pairs of DNA.

Some objections to transhumanism are actually objections to transhumanism. But some are caused by the denotative-connotative gap. A person's analytic reasoner says, "What about this transhumanism thing, then?", and their connotative reasoner replies, "Human good! Ergo, not-human bad! QED."

I don't mean that we can get around this by renaming "transhumanism" as "humanism with sprinkles!" This confusion over denotation and connotation happens inside another person's head, and you can't control it with labels. If you propose making a germline genetic modification, this will trigger thoughts about the definition of "human" in someone else's head. When that pe...