trentmkelly/LessWrong-43k huggingface dataset:
More stuff
by KatjaGrace — 2010-01-14T13:30:39.000Z

A few more bits I liked in The Stuff of Thought:

Hypernyms elevate

Labeling someone with a small aspect of what they are – a trait or part – undignifies them. Calling someone a cripple, the blonde, a suit, isn’t nice. The opposite works too often – things sound more dignified if you label them with a larger category than usual. Driving machines and dental cleaning systems sound more pretentious than cars and toothbrushes.

Lots of our phrases rest on the same conceptual metaphors

Though we don’t have a specific saying that ‘up is like good and down is like bad’, it’s easy to see that we equate these things from our endless sayings that spring from this metaphor. Feeling high, spirits soaring, hitting rock bottom, a downturn, pick me up, low mood, low character, low blow, feeling down, over the moon, I’m above you. I can make up new phrases using the same metaphor and you will know what I mean without apparently thinking about it. These things suggest that the connection between goodness and upness is still active in our minds; these things aren’t idioms.

Intuitions that phrases like ‘pin the wall with posters’ are wrong follow simple rules that we are introspectively oblivious to.

You can say ‘splatter paint on the wall’ or ‘splatter the wall with paint’. You can say ‘pin posters on the wall’. This seems analogous to ‘splatter paint on the wall’, so why don’t we use the same alternative form with that?

The answer is that the first form implies that you were changing the paint or the poster by putting it on the wall, whereas the second form implies that you were changing the wall by putting paint or a poster on it. Painting a wall changes the nature of the wall in our eyes, while pinning posters on it doesn’t.

This explanation holds across the many other examples of this pattern, and similar explanations hold for others. You photograph a wall with your camera, but don’t photograph your camera at the wall. You fling a cat into a room, but you don’t fling a room with a cat. You can load hay into a cart or load a cart with hay.

Some situations it makes sense to frame in a different way and others not. Other conceptual differences that matter with verbs for instance include whether the action was purposeful or accidental, physica...
Advice for AI makers
by Stuart_Armstrong — 2010-01-14T11:32:04.429Z

A friend of mine is about to launch himself heavily into the realm of AI programming. The details of his approach aren't important; probabilities dictate that he is unlikely to score a major success. He's asked me for advice, however, on how to design a safe(r) AI. I've been pointing him in the right directions and sending him links to useful posts on this blog and the SIAI.

Do people here have any recommendations they'd like me to pass on? Hopefully, these may form the basis of a condensed 'warning pack' for other AI makers.

Addendum: Advice along the lines of "don't do it" is vital and good, but unlikely to be followed. Coding will nearly certainly happen; is there any way of making it less genocidally risky?
Outline of a lower bound for consciousness
by PhilGoetz — 2010-01-13T05:27:48.087Z

This is a summary of an article I'm writing on consciousness, and I'd like to hear opinions on it. It is the first time anyone has been able to defend a numeric claim about subjective consciousness.

ADDED: Funny no one pointed out this connection, but the purpose of this article is to create a nonperson predicate.

1. Overview

I propose a test for the absence of consciousness, based on the claim that a necessary, but not sufficient, condition for a symbol-based knowledge system to be considered conscious is that it has exactly one possible symbol grounding, modulo symbols representing qualia. This supposition, plus a few reasonable assumptions, leads to the conclusion that a symbolic artificial intelligence using Boolean truth-values and having an adult vocabulary must have on the order of 106 assertions before we need worry whether it is conscious.

Section 2 will explain the claim about symbol-grounding that this analysis is based on. Section 3 will present the math and some reasonable assumptions for computing the expected number of randomly-satisfied groundings for a symbol system. Section 4 will argue that a Boolean symbol system with a human-level vocabulary must have millions of assertions in order for it to be probable that no spurious symbols groundings exist.

2. Symbol grounding

2.1. A simple representational system

Consider a symbolic reasoning system whose knowledge base K consists of predicate logic assertions, using atoms, predicates, and variables. We will ignore quantifiers. In addition to the knowledge base, there is a relatively small set of primitive rules that say how to derive new assertions from existing assertions, which are not asserted in K, but are implemented by the inference engine interpreting K. Any predicate that occurs in a primitive rule is called a primitive predicate.

The meaning of primitive predicates is specified by the program that implements the inference engine. The meaning of predicates other than primitive rules is defined within K, in terms of the primitive predicates, in the same way that LISP code defines a semantics for functions based on the semantics of LISP's primitive functions. (If this is objectionable, you can devise a representation in which the only predicates are primitive predicates (Shapiro 2000).)

This still leaves the semantics of the atoms undefined. W...
High Status and Stupidity: Why?
by Eliezer Yudkowsky — 2010-01-12T16:36:56.176Z

Michael Vassar once suggested: "Status makes people effectively stupid, as it makes it harder for them to update their public positions without feeling that they are losing face."

To the extent that status does, in fact, make people stupid, this is a rather important phenomenon for a society like ours in which practically all decisions and beliefs pass through the hands of very-high-status individuals (a high "cognitive Gini coefficient").

Does status actually make people stupid? It's hard to say because I haven't tracked many careers over time. I do have a definite and strong impression, with respect to many high-status individuals, that it would have been a lot easier to have an intelligent conversation with them, if I'd approached them before they made it big. But where does that impression come from, since I haven't actually tracked them over time? (Fundamental question of rationality: What do you think you know and how do you think you know it?) My best guess for why my brain seems to believe this: I know it's possible to have intelligent conversations with smart grad students, and I get the strong impression that high-status people used to be those grad students, but now it's much harder to have intelligent conversations with them than with smart grad students.

Hypotheses:

Vassar's hypothesis: Higher status increases the amount of face you lose when you change your mind, or increases the cost of losing face.

The open-mindedness needed to consider interesting new ideas is (was) only an evolutionary advantage for low-status individuals seeking a good idea to ride to high status. Once high status is achieved, new ideas are high-risk gambles with less relative payoff - the optimal strategy is to be mainstream. I think Robin Hanson had a post about this but I can't recall the title.

Intelligence as such is a high-cost feature which is no longer necessary once status is achieved. We can call this the Llinas Hypothesis.

High-status individuals were intelligent when they were young; the observed disparity is due solely to the standard declines of aging.

High-status individuals spend more time on dinners and politics, and less time on problem-solving and reading; they exercise their minds less.

High-status individuals are under less pressure to perform, in general.

High-status individuals ar

...
We desire genetically attributed success
by cousin_it — 2010-01-12T13:56:02.692Z

A certain girl friend of mine isn't very successful in romantic relationships. But whenever I advise her to take a more rational approach to hunting men, she says my perfectly sensible suggestions still make her feel uneasy because things ought to happen naturally. Is this kind of thinking a cognitive mistake? I'm not sure...

A little reflection turns up other examples where people don't just desire success, but specifically natural, genetically attributed success. In ego-heavy areas like game design, everybody starts out wanting to be the master creator rather than humble participant, even if collaboration yields a much bigger pie. (Linky on how an egoless environment aids game creation.)

This behavior is most easily explained by status-seeking, but I feel there's something else at play: a kind of lingering desire to prove to myself that I was worthy from the start. It might be self-signaling or an unexpected application of Robin Hanson's idea that we enjoy clear fitness signals on general principle. Not sure on this point either - it sounds plausible but weird.

I first got the idea for this post while thinking about cooperation. Duh, if I join some collaborative endeavor, it had better be about proving my worth! But when we insist on getting status rather than (say) money, we may be passing over opportunities to cooperate, so at least in some cases it might be helpful to override the evolution-provided default.
Why are obvious meanings veiled?
by KatjaGrace — 2010-01-12T13:30:22.000Z

Why do people use veiled language even when both parties likely know the real message? For instance if a boy asks a girl up for coffee after a date, nobody is likely to miss the cliched connotation, so why not be direct? The same question goes for many other threats, bribes, requests and propositions. Where meaning is reasonably ambiguous, plausible deniability seems a good explanation. However in many cases denial wouldn’t be that plausible and would make you look fairly silly anyway.

In The Stuff of Thought, Steven Pinker offers six possible explanations for these cases, the last of which I found particularly interesting: People are not embarrassed nearly as much by everyone knowing their failings as long as they aren’t common knowledge (everyone knows that everyone knows etc). Pinker suggests veiled language can offer enough uncertainty that while the other party knows they are very likely being offered sex for instance (which is all you need them to know), they are still unsure of whether you know that they know this, and so on. Plausible deniability of common knowledge means if they decline you, you can carry on with yours pride intact more easily, because status is about what everyone thinks everyone thinks etc your status is, and that hasn’t changed.

This has some problems. Does any vagueness preclude mutual knowledge? We don’t act as though it does; there is always some uncertainty. Plus we take many private observations into account in judging others’ status, though you could argue that this is to judge how they are usually judged, so any aspect of a person you believe others haven’t mostly seen should not inform you on their status. Pinker suggests that a larger gap between the level of vagueness that precludes mutual knowledge and that which allows plausible deniability is helped by people attributing their comprehension of veiled suggestions to their own wonderful social intuition, which makes them less sure that the other knows what they understood.

But veiled comments often seem to allow no more uncertainty than explicit ones. For instance, ‘it would be great if you would do the washing up’ is about as obvious as ‘do the washing up’, but somehow more polite because you are informing not commanding, though the listener arguably has less choice because angrily proclaiming that they are not...
The things we know that we know ain't so
by PhilGoetz — 2010-01-11T21:59:22.776Z

We're all familiar with false popular memes that spread faster than they can be stomped out: You only use 10% of your brain. Al Gore said he invented the internet. Perhaps it doesn't surprise you that some memes in popular culture can't be killed. But does the same thing happen in science?

Most of you have probably heard of Broca's aphasia and Wernicke's aphasia. Every textbook and every college course on language and the brain describes the connection between damage to these areas, and the speech deficits named after them.

Also, both are probably wrong. Both areas were mistakenly associated with their aphasias because they are near or surrounded by other areas which, when damaged, cause the aphasias. Yet our schools continue teaching the traditional, erroneous story; including a lecture in 9.14 at MIT given in 2005. Both the Wikipedia entry on Wernicke's aphasia and the Wikipedia entry on Broca's aphasia are still in error; the Wikipedia entry on Wernicke's area has got it straight.

Is it because this information is considered unimportant? Hardly; it's probably the only functional association you will find in every course and every book on the brain.

Is it because the information is too new to have penetrated the field? No; see the dates on the references below.

In spite of this failure in education, are the experts thoroughly familiar with this information? Possibly not; this 2006 paper on Broca's area by a renowned expert does not mention it. (In its defense, it references many other studies in which damage to Broca's area is associated with language deficits.)

So:

Am I wrong, and the evidence still implicates Broca's area and Wernicke's area in their aphasias above other areas?

If I'm right, why can't the new understanding displace the old understanding?

Is this a general failure in the way we do science? Can you think of other examples where an important discovery can't penetrate its field?

References

Bogen JE, Bogen GM (1976). Wernicke's region—Where is it? Ann. N. Y. Acad. Sci. 280: 834–43.

Dronkers, N. F., Shapiro, J. K., Redfern, B., & Knight, R. T. (1992). The role of Broca’s area in Broca’s aphasia.
Journal of Clinical and Experimental Neuropsychology, 14, 52–53.

Dronkers NF., Redfern B B., Knight R T. (2000). The neural architecture of la...
Consciousness Explained, chapter 3
by PhilGoetz — 2010-01-11T17:39:55.261Z

A Visit to the Phenonemological Garden

This chapter categorizes and discusses mental phenomena. It emphasizes that we don't re-draw the outer world inside our heads for a little person to look at.

1. Welcome to the phenom

"Kant distinguished "phenomena," things as they appear, from "noumena," things as they are in themselves, and during the development of the natural or physical sciences in the 19th century, the term phenomenology came to refer to the merely descriptive study of any subject matter."

["Phenomenology" seems similar to what I recently called "mere curve-fitting" in a comment exchange about gravity, as something that lets us make accurate predictions about a phenomenon, while still leaving it in the realm of metaphysics (not integrated causally into the rest of physics).]

D will use the term "phenom" to denote the (true) ontology of conscious phenomena. He divides it into

sense perceptions

imagined perceptions

affect

2. Our experience of the external world

D tells an interesting fable about a philosopher who denied that the mechanical reproduction of an orchestra's sounds could be possible, due to a failure of his own imagination. He then challenges the naive assumption that, after all the sound of all the different instruments have been translated into a stream of pulses, they get converted back into all the different orchestral sounds in the brain. The distinctive sounds of different instruments, once thought to be unanalyzable, are due to the superposition of a number of different frequencies and amplitudes of pure tones. This leads into the argument that we don't experience sounds in our heads - we're not going to go to the effort to break sound down into frequencies just to get it inside our heads, and then build the original waveform back up again. Similarly, we imagine we hear word boundaries in speech; yet there are no gaps between words in the acoustic energy profile.

Vision: We don't draw pictures in our heads and then look at them. That would lead to an infinite regress. Besides, it's dark in there. Our visual perception is a conglomeration of different objects at different resolutions tagged with different visual properties.

3. Our experience of the internal world

The gist of this section seems to be that we don't imagine things by, eg., drawing pictures in our heads. [I rem...
Dennett's "Consciousness Explained": Chpt 2
by PhilGoetz — 2010-01-10T23:38:55.449Z

Chapter 2: Explaining Consciousness

This chapter is all about dualism and why it's bad. I find this chapter incoherent, because Dennett never defines dualism carefully, and confuses it with the theistic views historically associated with dualism.

[Long stretches of my own ideas will be in brackets, like this.]

1. Should consciousness be demystified?

D begins with a defense against those who don't want an explanation of consciousness. I'm not even going to read this section.

2. The mystery of consciousness

"What could be more obvious or certain to each of us than that he or she is a conscious subject of experience, an enjoyer of perceptions and sensations, a sufferer of pain, an entertainer of ideas, and a conscious deliberator? ... How can living physical bodies in the physical world produce such phenomena?"

3. The attractions of mind stuff

D says Searle describes functionalism by saying that if a computer program reproduced the entire functional structure of a human wine taster's cognitive system, functionalism says that it would reproduce all the mental properties, including enjoyment. This is followed by a [long, pointless, popular] discussion of whether volcanos, hurricanes, etc., have souls.

The "mind stuff" position D sets up in order to knock down is not John Searle's "mind stuff" position, nor Roger Penrose's, but something spiritual and non-physical. He says that the mind-stuff view is that "the conscious mind... cannot just be the brain... because nothing in the brain could appreciate wine." [D should pursue this further, because the summary given of the "mind stuff" view is too incoherent to attack.]

4. Why dualism is forlorn

D says that the "mind stuff" view is that the brain is "composed not of ordinary matter", and is dualism. Materialists, OTOH, say we can account for all mental phenomena using the same principles that explain other things. Dualism is bad; materialism is good.

D says that if dualism were true, then the mind would need to act on our body, so we could move our arms. But to move, a physical thing needs energy; and how can non-matter produce energy? Dualism is thus dead, QED.

[IMHO this muddies the waters terribly, in a way I've seen them muddied many times before. The dichotomy between dualism and materialism is a false dichotomy. There are no "materialists" who believe only i...
Why does failed indulgence cause guilt?
by KatjaGrace — 2010-01-10T13:33:10.000Z

When luxurious products disappoint, people feel more guilty than when utilitarian products do:

The primary insights this research provides are as follows: (1) a negative experience with the choice of a product with superior utilitarian and inferior hedonic benefits (e.g., a highly functional cell phone with poor attractiveness) over a product with superior hedonic and inferior utilitarian benefits evokes feelings of sadness, disappointment, and anger, (2) a negative experience with the choice of a product with superior hedonic and inferior utilitarian benefits (e.g., a highly attractive cell phone with poor functionality) over a product with superior utilitarian and inferior hedonic benefits evokes feelings of guilt and anxiety.

This is interesting because the failure of the product to satisfy isn’t caused by the indulgence of the buyer’s decision to buy it. Yet it’s as thoug the blame goes to the last decision the buyer made, and the problem with that decision is taken to be whatever felt bad about it at the time, however unrelated to the failure at hand. Or does the disappointment seem like punishment somehow for the original greed?

How general is this pattern? I think I feel more guilty when my less admirable intentions fail. Can you think of examples or counterexamples?
Dennett's "Consciousness Explained": Prelude
by PhilGoetz — 2010-01-10T07:31:03.775Z

I'm starting Dennett's "Consciousness Explained". Dennett says, in the introduction, that he believes he has solved the problem of consciousness. Since several people have referred to his work here with approval, I'm going to give it a go. I'm going to post chapter summaries as I read, for my own selfish benefit, so that you can point out when you disagree with my understanding of it. "D" will stand for Dennett.

If you loathe the C-word, just stop now. That's what the convenient break just below is for. You are responsible for your own wasted time if you proceed.

Chpt. 1: Prelude: How are Hallucinations Possible?

D describes the brain in a vat, and asks how we can know we aren't brains in vats. This dismays me, as it is one of those questions that distracts people trying to talk about consciousness, that has nothing to do with the difficult problems of consciousness.

Dennett states, without presenting a single number, that the bandwidth needs for reproducing our sensory experience would be so great that it is impossible (his actual word); and that this proves that we are not brains in vats. Sigh.

He then asks how hallucinations are possible: "How on earth can a single brain do what teams of scientists and computer animators would find to be almost impossible?" Sigh again. This is surprising to Dennett because he believes he has just established that the bandwidth needs for consciousness are too great for any computer to provide; yet the brain sometimes (during hallucinations) provides nearly that much bandwidth. D has apparently forgotten that the brain provides exactly, by definition, the consciousness bandwidth of information to us all the time.

D recounts Descartes' remarkably prescient discussion of the bellpull as an analogy for how the brain could send us phantom misinformation; but dismisses it, saying, "there is no way the brain as illusionist could store and manipulate enough false information to fool an inquiring mind." Sigh. Now not only consciousness, but also dreams, are impossible. However, D then comes back to dreams, and is aware they exist and are hallucinations; so either he or I is misunderstanding this section.

On p. 12 he suggests something interesting: Perception is driven both bottom-up (from the senses) and top-down (from our expectations). A hallucin...