trentmkelly/LessWrong-43k huggingface dataset:
Do Scientists Already Know This Stuff?
by Eliezer Yudkowsky — 2008-05-17T02:25:59.000Z

poke alleges:

"Being able to create relevant hypotheses is an important skill and one a scientist spends a great deal of his or her time developing. It may not be part of the traditional description of science but that doesn't mean it's not included in the actual social institution of science that produces actual real science here in the real world; it's your description and not science that is faulty."

I know I've been calling my younger self "stupid" but that is a figure of speech; "unskillfully wielding high intelligence" would be more precise. Eliezer18 was not in the habit of making obvious mistakes—it's just that his "obvious" wasn't my "obvious".

No, I did not go through the traditional apprenticeship. But when I look back, and see what Eliezer18 did wrong, I see plenty of modern scientists making the same mistakes. I cannot detect any sign that they were better warned than myself.

Sir Roger Penrose—a world-class physicist—still thinks that consciousness is caused by quantum gravity. I expect that no one ever warned him against mysterious answers to mysterious questions—only told him his hypotheses needed to be falsifiable and have empirical consequences. Just like Eliezer18.

"Consciousness is caused by quantum gravity" has testable implications: It implies that you should be able to look at neurons and discover a coherent quantum superposition (whose collapse?) contributes to information-processing, and that you won't ever be able to reproduce a neuron's input-output behavior using a computable microanatomical simulation...

...but even after you say "Consciousness is caused by quantum gravity", you don't anticipate anything about how your brain thinks "I think therefore I am!" or the mysterious redness of red, that you did not anticipate before, even though you feel like you know a cause of it. This is a tremendous danger sign, I now realize, but it's not the danger sign that I was warned against, and I doubt that Penrose was ever told of it by his thesis advisor. For that matter, I doubt that Niels Bohr was ever warned against it when it came time to formulate the Copenhagen Interpretation.

As far as I can tell, the reason Eliezer18 and Sir Roger Penrose and Niels Bohr were not warned, is that no standard warning exists.

I did not generalize the concept of "mysterious answers t...
Science Isn't Strict Enough
by Eliezer Yudkowsky — 2008-05-16T06:51:16.000Z

Once upon a time, a younger Eliezer had a stupid theory. Eliezer18 was careful to follow the precepts of Traditional Rationality that he had been taught; he made sure his stupid theory had experimental consequences. Eliezer18 professed, in accordance with the virtues of a scientist he had been taught, that he wished to test his stupid theory.

This was all that was required to be virtuous, according to what Eliezer18 had been taught was virtue in the way of science.

It was not even remotely the order of effort that would have been required to get it right.

The traditional ideals of Science too readily give out gold stars. Negative experimental results are also knowledge, so everyone who plays gets an award. So long as you can think of some kind of experiment that tests your theory, and you do the experiment, and you accept the results, you've played by the rules; you're a good scientist.

You didn't necessarily get it right, but you're a nice science-abiding citizen.

(I note at this point that I am speaking of Science, not the social process of science as it actually works in practice, for two reasons. First, I went astray in trying to follow the ideal of Science—it's not like I was shot down by a journal editor with a grudge, and it's not like I was trying to imitate the flaws of academia. Second, if I point out a problem with the ideal as it is traditionally preached, real-world scientists are not forced to likewise go astray!)

Science began as a rebellion against grand philosophical schemas and armchair reasoning. So Science doesn't include a rule as to what kinds of hypotheses you are and aren't allowed to test; that is left up to the individual scientist. Trying to guess that a priori, would require some kind of grand philosophical schema, and reasoning in advance of the evidence. As a social ideal, Science doesn't judge you as a bad person for coming up with heretical hypotheses; honest experiments, and acceptance of the results, is virtue unto a scientist.

As long as most scientists can manage to accept definite, unmistakable, unambiguous experimental evidence, science can progress. It may happen too slowly—it may take longer than it should—you may have to wait for a generation of elders to die out—but eventually, the ratchet of knowledge clicks forward another notch. Year by year, decade by dec...
When Science Can't Help
by Eliezer Yudkowsky — 2008-05-15T07:24:25.000Z

Once upon a time, a younger Eliezer had a stupid theory. Let's say that Eliezer18's stupid theory was that consciousness was caused by closed timelike curves hiding in quantum gravity. This isn't the whole story, not even close, but it will do for a start.

And there came a point where I looked back, and realized:

I had carefully followed everything I'd been told was Traditionally Rational, in the course of going astray. For example, I'd been careful to only believe in stupid theories that made novel experimental predictions, e.g., that neuronal microtubules would be found to support coherent quantum states.

Science would have been perfectly fine with my spending ten years trying to test my stupid theory, only to get a negative experimental result, so long as I then said, "Oh, well, I guess my theory was wrong."

From Science's perspective, that is how things are supposed to work—happy fun for everyone. You admitted your error! Good for you! Isn't that what Science is all about?

But what if I didn't want to waste ten years?

Well... Science didn't have much to say about that. How could Science say which theory was right, in advance of the experimental test? Science doesn't care where your theory comes from—it just says, "Go test it."

This is the great strength of Science, and also its great weakness.

Gray Area asked:

Eliezer, why are you concerned with untestable questions?

Because questions that are easily immediately tested are hard for Science to get wrong.

I mean, sure, when there's already definite unmistakable experimental evidence available, go with it. Why on Earth wouldn't you?

But sometimes a question will have very large, very definite experimental consequences in your future—but you can't easily test it experimentally right now—and yet there is a strong rational argument.

Macroscopic quantum superpositions are readily testable: It would just take nanotechnologic precision, very low temperatures, and a nice clear area of interstellar space. Oh, sure, you can't do it right now, because it's too expensive or impossible for today's technology or something like that—but in theory, sure! Why, maybe someday they'll run whole civilizations on macroscopically superposed quantum computers, way out in a well-swept volume of a Great Void. (Asking what quantum...
Science Doesn't Trust Your Rationality
by Eliezer Yudkowsky — 2008-05-14T02:13:46.000Z

Scott Aaronson suggests that Many-Worlds and libertarianism are similar in that they are both cases of bullet-swallowing, rather than bullet-dodging:

Libertarianism and MWI are both are grand philosophical theories that start from premises that almost all educated people accept (quantum mechanics in the one case, Econ 101 in the other), and claim to reach conclusions that most educated people reject, or are at least puzzled by (the existence of parallel universes / the desirability of eliminating fire departments).

Now there's an analogy that would never have occurred to me.

I've previously argued that Science rejects Many-Worlds but Bayes accepts it. (Here, "Science" is capitalized because we are talking about the idealized form of Science, not just the actual social process of science.)

It furthermore seems to me that there is a deep analogy between (small-'l') libertarianism and Science:

Both are based on a pragmatic distrust of reasonable-sounding arguments.

Both try to build systems that are more trustworthy than the people in them.

Both accept that people are flawed, and try to harness their flaws to power the system.

The core argument for libertarianism is historically motivated distrust of lovely theories of "How much better society would be, if we just made a rule that said XYZ." If that sort of trick actually worked, then more regulations would correlate to higher economic growth as society moved from local to global optima. But when some person or interest group gets enough power to start doing everything they think is a good idea, history says that what actually happens is Revolutionary France or Soviet Russia.

The plans that in lovely theory should have made everyone happy ever after, don't have the results predicted by reasonable-sounding arguments. And power corrupts, and attracts the corrupt.

So you regulate as little as possible, because you can't trust the lovely theories and you can't trust the people who implement them.

You don't shake your finger at people for being selfish. You try to build an efficient system of production out of selfish participants, by requiring transactions to be voluntary. So people are forced to play positive-sum games, because that's how they get the other party to sign the contract. With violence restrained and contracts enforced, individual selfishness can power a global...
The Dilemma: Science or Bayes?
by Eliezer Yudkowsky — 2008-05-13T08:16:28.000Z

"Eli: You are writing a lot about physics recently. Why?"
—Shane Legg (and several other people)

"In light of your QM explanation, which to me sounds perfectly logical, it seems obvious and normal that many worlds is overwhelmingly likely. It just seems almost too good to be true that I now get what plenty of genius quantum physicists still can't. [...] Sure I can explain all that away, and I still think you're right, I'm just suspicious of myself for believing the first believable explanation I met."
—Recovering irrationalist

RI, you've got no idea how glad I was to see you post that comment.

Of course I had more than just one reason for spending all that time posting about quantum physics. I like having lots of hidden motives, it's the closest I can ethically get to being a supervillain.

But to give an example of a purpose I could only accomplish by discussing quantum physics...

In physics, you can get absolutely clear-cut issues. Not in the sense that the issues are trivial to explain. But if you try to apply Bayes to healthcare, or economics, you may not be able to formally lay out what is the simplest hypothesis, or what the evidence supports. But when I say "macroscopic decoherence is simpler than collapse" it is actually strict simplicity; you could write the two hypotheses out as computer programs and count the lines of code. Nor is the evidence itself in dispute.

I wanted a very clear example—Bayes says "zig", this is a zag—when it came time to break your allegiance to Science.

"Oh, sure," you say, "the physicists messed up the many-worlds thing, but give them a break, Eliezer! No one ever claimed that the social process of science was perfect. People are human; they make mistakes."

But the physicists who refuse to adopt many-worlds aren't disobeying the rules of Science. They're obeying the rules of Science.

The tradition handed down through the generations says that a new physics theory comes up with new experimental predictions that distinguish it from the old theory. You perform the test, and the new theory is confirmed or falsified. If it's confirmed, you hold a huge celebration, call the newspapers, and hand out Nobel Prizes for everyone; any doddering old emeritus professors who refuse to convert are qu...
The Failures of Eld Science
by Eliezer Yudkowsky — 2008-05-12T10:32:06.000Z

This time there were no robes, no hoods, no masks. Students were expected to become friends, and allies. And everyone knew why you were in the classroom. It would have been pointless to pretend you weren't in the Conspiracy.

Their sensei was Jeffreyssai, who might have been the best of his era, in his era. His students were either the most promising learners, or those whom the beisutsukai saw political advantage in molding.

Brennan fell into the latter category, and knew it. Nor had he hesitated to use his Mistress's name to open doors. You used every avenue available to you, in seeking knowledge; that was respected here.

"—for over thirty years," Jeffreyssai said. "Not one of them saw it; not Einstein, not Schrödinger, not even von Neumann." He turned away from his sketcher, and toward the classroom. "I pose to you to the question: How did they fail?"

The students exchanged quick glances, a calculus of mutual risk between the wary and the merely baffled. Jeffreyssai was known to play games.

Finally Hiriwa-called-the-Black leaned forward, jangling slightly as her equation-carved bracelets shifted on her ankles. "By your years given, sensei, this was two hundred and fifty years after Newton. Surely, the scientists of that era must have grokked the concept of a universal law."

"Knowing the universal law of gravity," said the student Taji, from a nearby seat, "is not the same as understanding the concept of a universal law." He was one of the promising ones, as was Hiriwa.

Hiriwa frowned. "No... it was said that Newton had been praised for discovering the first universal. Even in his own era. So it was known." Hiriwa paused. "But Newton himself would have been gone. Was there a religious injunction against proposing further universals? Did they refrain out of respect for Newton, or were they waiting for his ghost to speak? I am not clear on how Eld science was motivated—"

"No," murmured Taji, a laugh in his voice, "you really, really aren't."

Jeffreyssai's expression was kindly. "Hiriwa, it wasn't religion, and it wasn't lead in the drinking water, and they didn't all have Alzheimers, and they weren't sitting around all day reading webcomics. Forget the catalogue of horrors out of ancient times. Just think in t...
Many Worlds, One Best Guess
by Eliezer Yudkowsky — 2008-05-11T08:32:18.000Z

If you look at many microscopic physical phenomena—a photon, an electron, a hydrogen atom, a laser—and a million other known experimental setups—it is possible to come up with simple laws that seem to govern all small things (so long as you don’t ask about gravity). These laws govern the evolution of a highly abstract and mathematical object that I’ve been calling the “amplitude distribution,” but which is more widely referred to as the “wavefunction.”

Now there are gruesome questions about the proper generalization that covers all these tiny cases. Call an object “grue” if it appears green before January 1, 2020 and appears blue thereafter. If all emeralds examined so far have appeared green, is the proper generalization, “Emeralds are green” or “Emeralds are grue”?

The answer is that the proper generalization is “Emeralds are green.” I’m not going to go into the arguments at the moment. It is not the subject of this essay, and the obvious answer in this case happens to be correct. The true Way is not stupid: however clever you may be with your logic, it should finally arrive at the right answer rather than a wrong one.

In a similar sense, the simplest generalizations that would cover observed microscopic phenomena alone take the form of “All electrons have spin 12” and not “All electrons have spin 12 before January 1, 2020” or “All electrons have spin 12 unless they are part of an entangled system that weighs more than 1 gram.”

When we turn our attention to macroscopic phenomena, our sight is obscured. We cannot experiment on the wavefunction of a human in the way that we can experiment on the wavefunction of a hydrogen atom. In no case can you actually read off the wavefunction with a little quantum scanner. But in the case of, say, a human, the size of the entire organism defeats our ability to perform precise calculations or precise experiments—we cannot confirm that the quantum equations are being obeyed in precise detail.

We know that phenomena commonly thought of as “quantum” do not just disappear when many microscopic objects are aggregated. Lasers put out a flood of coherent photons, rather than, say, doing something completely different. Atoms have the chemical chara...
If Many-Worlds Had Come First
by Eliezer Yudkowsky — 2008-05-10T07:43:55.000Z

Not that I’m claiming I could have done better, if I’d been born into that time, instead of this one…

Macroscopic decoherence, a.k.a. many-worlds, was first proposed in a 1957 paper by Hugh Everett III. The paper was ignored. John Wheeler told Everett to see Niels Bohr. Bohr didn’t take him seriously.

Crushed, Everett left academic physics, invented the general use of Lagrange multipliers in optimization problems, and became a multimillionaire.

It wasn’t until 1970, when Bryce DeWitt (who coined the term “many-worlds”) wrote an article for Physics Today, that the general field was first informed of Everett’s ideas. Macroscopic decoherence has been gaining advocates ever since, and may now be the majority viewpoint (or not).

But suppose that decoherence and macroscopic decoherence had been realized immediately following the discovery of entanglement, in the 1920s. And suppose that no one had proposed collapse theories until 1957. Would decoherence now be steadily declining in popularity, while collapse theories were slowly gaining steam?

Imagine an alternate Earth, where the very first physicist to discover entanglement and superposition said, “Holy flaming monkeys, there’s a zillion other Earths out there!”

In the years since, many hypotheses have been proposed to explain the mysterious Born probabilities. But no one has yet suggested a collapse postulate. That possibility simply has not occurred to anyone.

One day, Huve Erett walks into the office of Biels Nohr…

“I just don’t understand,” Huve Erett said, “why no one in physics even seems interested in my hypothesis. Aren’t the Born statistics the greatest puzzle in modern quantum theory?”

Biels Nohr sighed. Ordinarily, he wouldn’t even bother, but something about the young man compelled him to try.

“Huve,” says Nohr, “every physicist meets dozens of people per year who think they’ve explained the Born statistics. If you go to a party and tell someone you’re a physicist, chances are at least one in ten they’ve got a new explanation for the Born statistics. It’s one of the most famous problems in modern science, and worse, it’s a problem that everyone thinks they can understand. To get attention, a new Born hypothesis has to be… pretty darn good.”

“And this,” Huve says, “this isn’t good?”

Huve gestures to the paper he’d brought to Biels Nohr. It is a short paper. The title reads, “The Solution to the Born Problem.” The ...
Collapse Postulates
by Eliezer Yudkowsky — 2008-05-09T07:49:15.000Z

Macroscopic decoherence—also known as “many-worlds”—is the idea that the known quantum laws that govern microscopic events simply govern at all levels without alteration. Back when people didn’t know about decoherence—before it occurred to anyone that the laws deduced with such precision for microscopic physics might apply universally—what did people think was going on?

The initial reasoning seems to have gone something like:

When my calculations showed an amplitude of −13i for this photon to get absorbed, my experimental statistics showed that the photon was absorbed around 107 times out of 1,000, which is a good fit to 19, the square of the modulus.

to

The amplitude is the probability (by way of the squared modulus).

to

Once you measure something and know it didn’t happen, its probability goes to zero.

Read literally, this implies that knowledge itself—or even conscious awareness— causes the collapse. Which was in fact the form of the theory put forth by Werner Heisenberg!

But people became increasingly nervous about the notion of importing dualistic language into fundamental physics—as well they should have been! And so the original reasoning was replaced by the notion of an objective “collapse” that destroyed all parts of the wavefunction except one, and was triggered sometime before superposition grew to human-sized levels.

Now, once you’re supposing that parts of the wavefunction can just vanish, you might think to ask:

Is there only one survivor? Maybe there are many surviving worlds, but they survive with a frequency determined by their integrated squared modulus, and so the typical surviving world has experimental statistics that match the Born rule.

Yet collapse theories considered in modern academia only postulate one surviving world. Why?

Collapse theories were devised in a time when it simply didn’t occur to any physicists that more than one world could exist! People took for granted that measurements had single outcomes—it was an assumption so deep it was invisible, because it was what they saw happening. Collapse theories were devised to explain why measurements had single outcomes, rather than (in full generality) why experimental statistics matched the Born rule.

For similar reasons, the “collapse postulates” considered academically suppose that collapse occurs before any human beings get superposed. But experiments are steadily ruling out the possibilit... ,
Quantum Non-Realism
by Eliezer Yudkowsky — 2008-05-08T05:27:23.000Z

"Does the moon exist when no one is looking at it?"

—Albert Einstein, asked of Niels Bohr

Suppose you were just starting to work out a theory of quantum mechanics.

You begin to encounter experiments that deliver different results depending on how closely you observe them. You dig underneath the reality you know, and find an extremely precise mathematical description that only gives you the relative frequency of outcomes; worse, it’s made of complex numbers. Things behave like particles on Monday and waves on Tuesday.

The correct answer is not available to you as a hypothesis, because it will not be invented for another thirty years.

In a mess like that, what’s the best you could do?

The best you can do is the strict “shut up and calculate” interpretation of quantum mechanics. You’ll go on trying to develop new theories, because doing your best doesn’t mean giving up. But we’ve specified that the correct answer won’t be available for thirty years, and that means none of the new theories will really be any good. Doing the best you could theoretically do would mean that you recognized that, even as you looked for ways to test the hypotheses.

The best you could theoretically do would not include saying anything like, “The wavefunction only gives us probabilities, not certainties.” That, in retrospect, was jumping to a conclusion; the wavefunction gives us a certainty of many worlds existing. So that part about the wavefunction being only a probability was not-quite-right. You calculated, but failed to shut up.

If you do the best that you can do without the correct answer being available, then, when you hear about decoherence, it will turn out that you have not said anythingincompatible with decoherence. Decoherence is not ruled out by the data and the calculations. So if you refuse to affirm, as positive knowledge, any proposition which was not forced by the data and the calculations, the calculations will not force you to say anything incompatible with decoherence. So too with whatever the correct theory may be, if it is not decoherence. If you go astray, it must be from your own impulses.

But it is hard for human beings to shut up and calculate—really shut up and calculate. There is an overwhelming tendency to treat our ignorance as if it were positive knowledge.

I don’t know if any conversations like this ever really took place, but this is how ignorance becomes knowle... ,