trentmkelly/LessWrong-43k huggingface dataset:
How An Algorithm Feels From Inside
by Eliezer Yudkowsky — 2008-02-11T02:35:20.000Z

"If a tree falls in the forest, and no one hears it, does it make a sound?" I remember seeing an actual argument get started on this subject—a fully naive argument that went nowhere near Berkeleyan subjectivism. Just:

"It makes a sound, just like any other falling tree!"
"But how can there be a sound that no one hears?"

The standard rationalist view would be that the first person is speaking as if "sound" means acoustic vibrations in the air; the second person is speaking as if "sound" means an auditory experience in a brain. If you ask "Are there acoustic vibrations?" or "Are there auditory experiences?", the answer is at once obvious. And so the argument is really about the definition of the word "sound".

I think the standard analysis is essentially correct. So let's accept that as a premise, and ask: Why do people get into such an argument? What's the underlying psychology?

A key idea of the heuristics and biases program is that mistakes are often more revealing of cognition than correct answers. Getting into a heated dispute about whether, if a tree falls in a deserted forest, it makes a sound, is traditionally considered a mistake.

So what kind of mind design corresponds to that error?

In Disguised Queries I introduced the blegg/rube classification task, in which Susan the Senior Sorter explains that your job is to sort objects coming off a conveyor belt, putting the blue eggs or "bleggs" into one bin, and the red cubes or "rubes" into the rube bin. This, it turns out, is because bleggs contain small nuggets of vanadium ore, and rubes contain small shreds of palladium, both of which are useful industrially.

Except that around 2% of blue egg-shaped objects contain palladium instead. So if you find a blue egg-shaped thing that contains palladium, should you call it a "rube" instead? You're going to put it in the rube bin—why not call it a "rube"?

But when you switch off the light, nearly all bleggs glow faintly in the dark. And blue egg-shaped objects that contain palladium are just as likely to glow in the dark as any other blue egg-shaped object.

So if you find a blue egg-shaped object that contains palladium, and you ask "Is it a blegg?", the answer depends on what you have to do with the answer: If you ask "Which bin does the object go in?", then you choose as if the o...
Neural Categories
by Eliezer Yudkowsky — 2008-02-10T00:33:17.000Z

In Disguised Queries, I talked about a classification task of "bleggs" and "rubes". The typical blegg is blue, egg-shaped, furred, flexible, opaque, glows in the dark, and contains vanadium. The typical rube is red, cube-shaped, smooth, hard, translucent, unglowing, and contains palladium. For the sake of simplicity, let us forget the characteristics of flexibility/hardness and opaqueness/translucency. This leaves five dimensions in thingspace: Color, shape, texture, luminance, and interior.

Suppose I want to create an Artificial Neural Network (ANN) to predict unobserved blegg characteristics from observed blegg characteristics. And suppose I'm fairly naive about ANNs: I've read excited popular science books about how neural networks are distributed, emergent, and parallel just like the human brain!! but I can't derive the differential equations for gradient descent in a non-recurrent multilayer network with sigmoid units (which is actually a lot easier than it sounds).

Then I might design a neural network that looks something like this:

Network 1 is for classifying bleggs and rubes. But since "blegg" is an unfamiliar and synthetic concept, I've also included a similar Network 1b for distinguishing humans from Space Monsters, with input from Aristotle ("All men are mortal") and Plato's Academy ("A featherless biped with broad nails").

A neural network needs a learning rule. The obvious idea is that when two nodes are often active at the same time, we should strengthen the connection between them—this is one of the first rules ever proposed for training a neural network, known as Hebb's Rule.

Thus, if you often saw things that were both blue and furred—thus simultaneously activating the "color" node in the + state and the "texture" node in the + state—the connection would strengthen between color and texture, so that + colors activated + textures, and vice versa. If you saw things that were blue and egg-shaped and vanadium-containing, that would strengthen positive mutual connections between color and shape and interior.

Let's say you've already seen plenty of bleggs and rubes come off the conveyor belt. But now you see something that's furred, egg-shaped, and—gasp!—reddish purple (which we'll model as a "color" activation level of -2/3). You haven't yet tested th...
Disguised Queries
by Eliezer Yudkowsky — 2008-02-09T00:05:28.000Z

Imagine that you have a peculiar job in a peculiar factory: Your task is to take objects from a mysterious conveyor belt, and sort the objects into two bins. When you first arrive, Susan the Senior Sorter explains to you that blue egg-shaped objects are called "bleggs" and go in the "blegg bin", while red cubes are called "rubes" and go in the "rube bin".

Once you start working, you notice that bleggs and rubes differ in ways besides color and shape. Bleggs have fur on their surface, while rubes are smooth. Bleggs flex slightly to the touch; rubes are hard. Bleggs are opaque; the rube's surface slightly translucent.

Soon after you begin working, you encounter a blegg shaded an unusually dark blue—in fact, on closer examination, the color proves to be purple, halfway between red and blue.

Yet wait! Why are you calling this object a "blegg"? A "blegg" was originally defined as blue and egg-shaped—the qualification of blueness appears in the very name "blegg", in fact. This object is not blue. One of the necessary qualifications is missing; you should call this a "purple egg-shaped object", not a "blegg".

But it so happens that, in addition to being purple and egg-shaped, the object is also furred, flexible, and opaque. So when you saw the object, you thought, "Oh, a strangely colored blegg." It certainly isn't a rube... right?

Still, you aren't quite sure what to do next. So you call over Susan the Senior Sorter.

"Oh, yes, it's a blegg," Susan says, "you can put it in the blegg bin."
You start to toss the purple blegg into the blegg bin, but pause for a moment. "Susan," you say, "how do you know this is a blegg?"
Susan looks at you oddly. "Isn't it obvious? This object may be purple, but it's still egg-shaped, furred, flexible, and opaque, like all the other bleggs. You've got to expect a few color defects. Or is this one of those philosophical conundrums, like 'How do you know the world wasn't created five minutes ago complete with false memories?' In a philosophical sense I'm not absolutely certain that this is a blegg, but it seems like a good guess."
"No, I mean..." You pause, searching for words. "Why is there a blegg bin and a rube bin? What's the difference between bleg

...
The Cluster Structure of Thingspace
by Eliezer Yudkowsky — 2008-02-08T00:07:15.000Z

The notion of a "configuration space" is a way of translating object descriptions into object positions. It may seem like blue is "closer" to blue-green than to red, but how much closer? It's hard to answer that question by just staring at the colors. But it helps to know that the (proportional) color coordinates in RGB are 0:0:5, 0:3:2 and 5:0:0. It would be even clearer if plotted on a 3D graph.

In the same way, you can see a robin as a robin—brown tail, red breast, standard robin shape, maximum flying speed when unladen, its species-typical DNA and individual alleles. Or you could see a robin as a single point in a configuration space whose dimensions described everything we knew, or could know, about the robin.

A robin is bigger than a virus, and smaller than an aircraft carrier—that might be the "volume" dimension. Likewise a robin weighs more than a hydrogen atom, and less than a galaxy; that might be the "mass" dimension. Different robins will have strong correlations between "volume" and "mass", so the robin-points will be lined up in a fairly linear string, in those two dimensions—but the correlation won't be exact, so we do need two separate dimensions.

This is the benefit of viewing robins as points in space: You couldn't see the linear lineup as easily if you were just imagining the robins as cute little wing-flapping creatures.

A robin's DNA is a highly multidimensional variable, but you can still think of it as part of a robin's location in thingspace—millions of quaternary coordinates, one coordinate for each DNA base—or maybe a more sophisticated view that . The shape of the robin, and its color (surface reflectance), you can likewise think of as part of the robin's position in thingspace, even though they aren't single dimensions.

Just like the coordinate point 0:0:5 contains the same information as the actual HTML color blue, we shouldn't actually lose information when we see robins as points in space. We believe the same statement about the robin's mass whether we visualize a robin balancing the scales opposite a 0.07-kilogram weight, or a robin-point with a mass-coordinate of +70.

We can even imagine a configuration space with one or more dimensions for every distinct characteristic of an object, so that the position of an object's point in this space corr...
Typicality and Asymmetrical Similarity
by Eliezer Yudkowsky — 2008-02-06T21:20:50.000Z

Birds fly. Well, except ostriches don't. But which is a more typical bird—a robin, or an ostrich?

Which is a more typical chair: A desk chair, a rocking chair, or a beanbag chair?

Most people would say that a robin is a more typical bird, and a desk chair is a more typical chair. The cognitive psychologists who study this sort of thing experimentally, do so under the heading of "typicality effects" or "prototype effects" (Rosch and Lloyd 1978). For example, if you ask subjects to press a button to indicate "true" or "false" in response to statements like "A robin is a bird" or "A penguin is a bird", reaction times are faster for more central examples. (I'm still unpacking my books, but I'm reasonably sure my source on this is Lakoff 1986.) Typicality measures correlate well using different investigative methods—reaction times are one example; you can also ask people to directly rate, on a scale of 1 to 10, how well an example (like a specific robin) fits a category (like "bird").

So we have a mental measure of typicality—which might, perhaps, function as a heuristic—but is there a corresponding bias we can use to pin it down?

Well, which of these statements strikes you as more natural: "98 is approximately 100", or "100 is approximately 98"? If you're like most people, the first statement seems to make more sense. (Sadock 1977.) For similar reasons, people asked to rate how similar Mexico is to the United States, gave consistently higher ratings than people asked to rate how similar the United States is to Mexico. (Tversky and Gati 1978.)

And if that still seems harmless, a study by Rips (1975) showed that people were more likely to expect a disease would spread from robins to ducks on an island, than from ducks to robins. Now this is not a logical impossibility, but in a pragmatic sense, whatever difference separates a duck from a robin and would make a disease less likely to spread from a duck to a robin, must also be a difference between a robin and a duck, and would make a disease less likely to spread from a robin to a duck.

Yes, you can come up with rationalizations, like "Well, there could be more neighboring species of the robins, which would make the disease more likely to spread initially, etc.," but be careful not to try too hard to rationalize the probability ratings of...
Similarity Clusters
by Eliezer Yudkowsky — 2008-02-06T03:34:22.000Z

Once upon a time, the philosophers of Plato's Academy claimed that the best definition of human was a "featherless biped". Diogenes of Sinope, also called Diogenes the Cynic, is said to have promptly exhibited a plucked chicken and declared "Here is Plato's man." The Platonists promptly changed their definition to "a featherless biped with broad nails".

No dictionary, no encyclopedia, has ever listed all the things that humans have in common. We have red blood, five fingers on each of two hands, bony skulls, 23 pairs of chromosomes—but the same might be said of other animal species. We make complex tools to make complex tools, we use syntactical combinatorial language, we harness critical fission reactions as a source of energy: these things may serve out to single out only humans, but not all humans—many of us have never built a fission reactor. With the right set of necessary-and-sufficient gene sequences you could single out all humans, and only humans—at least for now—but it would still be far from all that humans have in common.
But so long as you don't happen to be near a plucked chicken, saying "Look for featherless bipeds" may serve to pick out a few dozen of the particular things that are humans, as opposed to houses, vases, sandwiches, cats, colors, or mathematical theorems.

Once the definition "featherless biped" has been bound to some particular featherless bipeds, you can look over the group, and begin harvesting some of the other characteristics—beyond mere featherfree twolegginess—that the "featherless bipeds" seem to share in common. The particular featherless bipeds that you see seem to also use language, build complex tools, speak combinatorial language with syntax, bleed red blood if poked, die when they drink hemlock.

Thus the category "human" grows richer, and adds more and more characteristics; and when Diogenes finally presents his plucked chicken, we are not fooled: This plucked chicken is obviously not similar to the other "featherless bipeds".

(If Aristotelian logic were a good model of human psychology, the Platonists would have looked at the plucked chicken and said, "Yes, that's a human; what's your point?")

If the first featherless biped you see is a plucked chicken, then you may end up thinking that the verbal label "human" denotes a plucked chicken; so I c...
Buy Now Or Forever Hold Your Peace
by Eliezer Yudkowsky — 2008-02-04T21:42:51.000Z

The Intrade prediction market is giving Hillary a 53% chance and Obama a 47% chance of winning the Democratic presidential nomination. Hillary is down 7.5 percentage points in just the last day. (Note: Between when I wrote the above, and when I posted this, Hillary went up to 54.)

From what I've read on Intrade, you can fund your account with up to $250 using a credit card, and it should land in your account immediately. (More than this takes time.) Also, remember that you can sell contracts at any time afterward - you don't have to wait months to collect your payout.

If you think that Hillary is going to do better than the polls on Super Tuesday, and you're going to sneer afterward and say that Intrade was "just tracking the polls", buy Hillary now.

If you think that Obama is going to do better than the polls on Super Tuesday, and you're going to gloat about how prediction markets didn't call this surprise in advance, buy Obama now.

If you don't do either, then clearly you do not really believe that you know anything the prediction markets don't. (Or you don't understand expected utility, or your utilities over final outcomes drop off improbably fast in the vicinity of your current wealth minus fifty bucks - you don't have to bet the full $250.) It is free money, going now for anyone who genuinely thinks they know better than the prediction markets what will happen next.

Prediction markets do not have supernatural insight. If they give the candidates fifty-fifty odds, it means that the market collectively doesn't know what will happen next. Even if you're well-calibrated, you get surprised on 90% probabilities one time out of ten.

The point is not that prediction markets are a good predictor but that they are the best predictor. If you think you can do better, why ain'cha rich? Any person, group, or method that does better can pump money out of the prediction markets.

If prediction markets react to polls, they're getting new information, that they didn't predict in advance, which happens. Being the best predictor doesn't make you omniscient.

Everyone's going to find it real easy to make a better prediction afterward, but if you think you can call it in advance, there's FREE MONEY GOING NOW.

Buy now, or forever hold your peace.
Extensions and Intensions
by Eliezer Yudkowsky — 2008-02-04T21:34:52.000Z

"What is red?"
"Red is a color."
"What's a color?"
"A color is a property of a thing."

But what is a thing? And what's a property? Soon the two are lost in a maze of words defined in other words, the problem that Steven Harnad once described as trying to learn Chinese from a Chinese/Chinese dictionary.

Alternatively, if you asked me "What is red?" I could point to a stop sign, then to someone wearing a red shirt, and a traffic light that happens to be red, and blood from where I accidentally cut myself, and a red business card, and then I could call up a color wheel on my computer and move the cursor to the red area. This would probably be sufficient, though if you know what the word "No" means, the truly strict would insist that I point to the sky and say "No."

I think I stole this example from S. I. Hayakawa—though I'm really not sure, because I heard this way back in the indistinct blur of my childhood. (When I was 12, my father accidentally deleted all my computer files. I have no memory of anything before that.)

But that's how I remember first learning about the difference between intensional and extensional definition. To give an "intensional definition" is to define a word or phrase in terms of other words, as a dictionary does. To give an "extensional definition" is to point to examples, as adults do when teaching children. The preceding sentence gives an intensional definition of "extensional definition", which makes it an extensional example of "intensional definition".

In Hollywood Rationality and popular culture generally, "rationalists" are depicted as word-obsessed, floating in endless verbal space disconnected from reality.

But the actual Traditional Rationalists have long insisted on maintaining a tight connection to experience:

"If you look into a textbook of chemistry for a definition of lithium, you may be told that it is that element whose atomic weight is 7 very nearly. But if the author has a more logical mind he will tell you that if you search among minerals that are vitreous, translucent, grey or white, very hard, brittle, and insoluble, for one which imparts a crimson tinge to an unluminous flame, this mineral being triturated with lime or witherite rats-bane, and then fused, can be partly dissolved in muriatic acid; and if this solution be evaporated, and the residue be extracted wit

...
Words as Hidden Inferences
by Eliezer Yudkowsky — 2008-02-03T23:36:23.000Z

Suppose I find a barrel, sealed at the top, but with a hole large enough for a hand. I reach in, and feel a small, curved object. I pull the object out, and it's blue—a bluish egg. Next I reach in and feel something hard and flat, with edges—which, when I extract it, proves to be a red cube. I pull out 11 eggs and 8 cubes, and every egg is blue, and every cube is red.

Now I reach in and I feel another egg-shaped object. Before I pull it out and look, I have to guess: What will it look like?

The evidence doesn't prove that every egg in the barrel is blue, and every cube is red. The evidence doesn't even argue this all that strongly: 19 is not a large sample size. Nonetheless, I'll guess that this egg-shaped object is blue—or as a runner-up guess, red. If I guess anything else, there's as many possibilities as distinguishable colors—and for that matter, who says the egg has to be a single shade? Maybe it has a picture of a horse painted on.

So I say "blue", with a dutiful patina of humility. For I am a sophisticated rationalist-type person, and I keep track of my assumptions and dependencies—I guess, but I'm aware that I'm guessing... right?

But when a large yellow striped feline-shaped object leaps out at me from the shadows, I think, "Yikes! A tiger!" Not, "Hm... objects with the properties of largeness, yellowness, stripedness, and feline shape, have previously often possessed the properties 'hungry' and 'dangerous', and thus, although it is not logically necessary, it may be an empirically good guess that aaauuughhhh CRUNCH CRUNCH GULP."

The human brain, for some odd reason, seems to have been adapted to make this inference quickly, automatically, and without keeping explicit track of its assumptions.

And if I name the egg-shaped objects "bleggs" (for blue eggs) and the red cubes "rubes", then, when I reach in and feel another egg-shaped object, I may think: Oh, it's a blegg, rather than considering all that problem-of-induction stuff.

It is a common misconception that you can define a word any way you like.

This would be true if the brain treated words as purely logical constructs, Aristotelian classes, and you never took out any more information than you put in.

Yet the brain goes on about its work of categorization, whether or not we consciously appr...
The Parable of Hemlock
by Eliezer Yudkowsky — 2008-02-03T02:01:43.000Z

"All men are mortal. Socrates is a man. Therefore Socrates is mortal."
— Aristotle(?)

Socrates raised the glass of hemlock to his lips...
"Do you suppose," asked one of the onlookers, "that even hemlock will not be enough to kill so wise and good a man?"
"No," replied another bystander, a student of philosophy; "all men are mortal, and Socrates is a man; and if a mortal drink hemlock, surely he dies."
"Well," said the onlooker, "what if it happens that Socrates isn't mortal?"
"Nonsense," replied the student, a little sharply; "all men are mortal by definition; it is part of what we mean by the word 'man'. All men are mortal, Socrates is a man, therefore Socrates is mortal. It is not merely a guess, but a logical certainty."
"I suppose that's right..." said the onlooker. "Oh, look, Socrates already drank the hemlock while we were talking."
"Yes, he should be keeling over any minute now," said the student.
And they waited, and they waited, and they waited...
"Socrates appears not to be mortal," said the onlooker.
"Then Socrates must not be a man," replied the student. "All men are mortal, Socrates is not mortal, therefore Socrates is not a man. And that is not merely a guess, but a logical certainty."

The fundamental problem with arguing that things are true "by definition" is that you can't make reality go a different way by choosing a different definition.

You could reason, perhaps, as follows: "All things I have observed which wear clothing, speak language, and use tools, have also shared certain other properties as well, such as breathing air and pumping red blood. The last thirty 'humans' belonging to this cluster, whom I observed to drink hemlock, soon fell over and stopped moving. Socrates wears a toga, speaks fluent ancient Greek, and drank hemlock from a cup. So I predict that Socrates will keel over in the next five minutes."

But that would be mere guessing. It wouldn't be, y'know, absolutely and eternally certain. The Greek philosophers—like most prescientific philosophers—were rather fond of certainty.

Luckily the Greek philosophers have a crushing rejoinder to your questioning. You have misunderstood...