trentmkelly/LessWrong-43k huggingface dataset:
Narrow your answer space
by Vaniver — 2010-12-28T11:38:56.007Z

Mark Rosewater, a designer for Magic: The Gathering, writes a lot about how "restrictions breed creativity." The explanation he gives is simple: when someone is building a house, the more tools they have, the better off they are. But when someone is looking for something, the more space they have to explore, the worse off they are. This applies to answer space: the more narrowly defined your problem is, the easier it is to search an answers; you'll find both more answers and better answers by looking in a well-chosen smaller space. Oftentimes the hardest problems to find good answers for are the ones with the widest scope.1

Most problems require some sort of creative thinking to overcome, and perhaps the greatest gains from this method come from applying it to your life goals. Imagine someone with a simple goal:2 they want to improve themselves. That's admirable, but sort of bland and massively broad. It would be helpful to have a way to work from a bland, broad goal to a better goal- but what's better, in this context? We know that restrictions help, but what sort of restrictions help the most?

Choosing Restrictions

In Getting Things Done, David Allen argues that to-do lists should only include 'tasks.' That is, only write down clearly identified next actions towards achieve specific goals. "Call Adam" isn't a task, but "Call Adam about hotel reservations for the conference" is. This serves to reduce mental load (once you've written down the second, you can remove the task entirely from your mind, while you still need to keep a lot in memory for the first), to reduce the need to plan while doing, and to reduce the ugh field associated with getting started. A good place for a goal to be, then, is a place where looking at the goal causes you to imagine the next task, even if you lost the to-do list where you had written down the next task and then purposefully forgotten it. So, actionable is a restriction that helps (even if the action is "wait for X," it's a good idea to know what X is, so you can look out for it!).

But at the same time, it helps when our goals are a sentence or a paragraph long, rather than a list of every subgoal and task. They should be kept simple, for the sake of both communication and flexibility. Finally, to encapsulate what's useful about restricting creativity in general, it should be specific. There are many actionable goals which pres...
Being Rational and Being Productive: Similar Core Skills?
by John_Maxwell — 2010-12-28T10:11:01.210Z

A synthesis of How to Actually Change Your Mind and PJ Eby, written for a general audience.

Several years ago I started suspecting that I needed glasses. At first, I was afraid. I began trying to convince myself that my vision was normal.
But then I stopped to reflect. If I went to see the eye doctor, either he would recommend glasses for me or he wouldn't. If he didn't recommend glasses for me then my life would be the same. But if he did recommend glasses, I would get a vision upgrade. Therefore, I reasoned, I should eagerly await my doctor visit.
By following the principle of letting control flow from thoughts to emotions, I gained two benefits. First, my beliefs about my vision weren't being distorted by my desire for it to be normal. And second, my emotion of eagerness for a potential vision upgrade meant that I wasn't tempted to put off visiting the doctor.
My glasses example might seem kind of mundane, but it demonstrates how thinking before emoting helps with two core human objectives: Being Correct and Getting Things Done.
Many of the cognitive biases that distort human reasoning can be explained by emotions that get in the way of our thought process. For example, status quo bias occurs when we are unreasonably skeptical of arguments that suggest we should change the status quo. The emotion that distorts our reasoning in this case is our fear of things that are new and unfamiliar. This is the bias that made me try to convince myself that I didn't need glasses.
When it comes to Getting Things Done, both productivity and procrastination are emotional states. Being able to turn these off and on would be useful.
So having control flow from thoughts to emotions has strong theoretical potential to help humans be less biased and more productive. But is it possible in practice?
Yes. The trick is to notice and reflect on negative emotions. Negative emotions like fear, guilt, shame, and regret are hardly ever useful and frequently interfere with our reasoning and working.
Sometimes that's all that's necessary. For example, once I was arguing with a friend about global warming and I started to become afraid that he might actually be right. Fortunately I noticed my fear and reminded myself that if my friend was right about global warming, I wanted to agree with him.&nbs...
Certainty estimates in areas outside one's expertise
by JoshuaZ — 2010-12-27T20:56:11.131Z

One issue that I've noticed in discussions on Less Wrong is that I'm much less certain about the likely answers to specific questions than some other people on Less Wrong. But the questions where this seems to be most pronounced are mathematical questions that are close to my area of expertise (such as whether P = NP). In areas outside my expertise, my apparent confidence is apparently often higher. Thus, for example at a recent LW meet-up I expressed a much lower probability estimate that cold fusion is real than what others in the conversation estimated. This suggests that I may be systematically overestimating my confidence in areas that I don't study as much, essentially a variant of the Dunning-Krueger effect. Have other people here experienced the same pattern with their own confidence estimates?
Dark Arts 101: Using presuppositions
by PhilGoetz — 2010-12-27T17:16:10.541Z

Sun Tzu said, "The supreme art of war is to subdue the enemy without fighting." This is also true in rhetoric. The best way to get a belief accepted is to fool people into thinking that they have already accepted it.

(Note, first-year students, that I did not say, "The best way to convince people of a belief". Do not try to convince people! It will not work; and it may start them thinking.)

An excellent way of doing this is to embed your desired conclusion as a presupposition to an enticing argument. If you are debating abortion, and you wish people to believe that human and non-human life are qualitatively different, begin by saying, "We all agree that killing humans is immoral. So when does human life begin?" People will be so eager to jump into the debate about whether a life becomes "human" at conception, the second trimester, or at birth (I myself favor "on moving out of the house"), they won't notice that they agreed to the embedded presupposition that the problem should be phrased as a binary category membership problem, rather than as one of tradeoffs or utility calculations.

Consider the recent furor over whether WikiLeaks leader Julian Assange is a journalist, or can be prosecuted for espionage. I don't know who initially asked this question. The earliest posing of the question that I can find that relates it to the First Amendment is this piece from Fox News on Dec. 8; but Marc Thiessen's column in the Washington Post of Aug. 3 has similar implications. Note that this question presupposes that First Amendment protection applies only to journalists! There is no legal precedent for this that I'm aware of; yet if people spend enough time debating whether Julian Assange is a journalist, they will have unknowingly convinced themselves that ordinary citizens have no First Amendment rights. (We can only hope that this was an artful stroke made from the shadows by some great master of the Dark Arts, and not a mere snowballing of an ignorant question.)
Efficient Induction
by paulfchristiano — 2010-12-27T10:40:38.829Z

(By which I mean, induction over efficient hypotheses.)

A standard prior is "uniform" over the class of computable functions (ie, uniform over infinite strings which have a prefix which compiles). Why is this a natural choice of prior? Well, we've looked at the universe for a really long time, and it seems to be computable. The Church-Turing thesis says we have no reason to choose a bigger support for our prior.

Why stop there? There is a natural generalization of the Church-Turing thesis, naturally called the extended Church-Turing thesis, which asserts that the universe is computable in polynomial time. In fact, we have a strong suspicion that physics should be local, which means more or less precisely that updates can be performed using a linear size logical circuit. Maybe we should restrict our prior further, looking only for small circuits.

(As we realized only slightly later, this extended hypothesis is almost certainly false, because in the real world probabilities are quantum. But if we replace "circuit" by "quantum circuit," which is a much less arbitrary change than it seems at face value, then we are good. Are further changes forthcoming? I don't know, but I suspect not.)

So we have two nice questions. First, what does a prior over efficiently computable hypotheses look like? Second, what sort of observation about the world could cause you to modify Solomonoff induction? For that matter, what sort of physical evidence ever convinced us that Solomonoff induction was a good idea in the first place, rather than a broader prior? I suspect that both of these questions have been tackled before, but google has failed me so now I will repeat some observations.

Defining priors over "polynomially large" objects is a little more subtle than usual Solomonoff induction. In some sense we need to penalize a hypothesis both for its description complexity and its computational complexity. Here is a first try:

A hypothesis consists of an initial state of some length N, and a logical circuit of size M which takes N bits to K+N bits. The universe evolves by repeatedly applying the logical circuit to compute both a "next observation" (the first K bits) and the new state of the universe (the last N bits). The probability of a hypothesis drops off exponentially with its length.

How reasonable is this? Why, not at all. The size of our hypothesis is the size of the universe, so...
Neutral AI
by PhilGoetz — 2010-12-27T06:10:05.878Z

Unfriendly AI has goal conflicts with us. Friendly AI (roughly speaking) shares our goals. How about an AI with no goals at all?

I'll call this "neutral AI". Cyc is a neutral AI. It has no goals, no motives, no desires; it is inert unless someone asks it a question. It then has a set of routines it uses to try to answer the question. It executes these routines, and terminates, whether the question was answered or not. You could say that it had the temporary goal to answer the question. We then have two important questions:

Is it possible (or feasible) to build a useful AI that operates like this?

Is an AI built in this fashion significantly less-dangerous than one with goals?

Many people have answered the first question "no". This would probably include Hubert Dreyfus (based on a Heideggerian analysis of semantics, which was actually very good but I would say misguided in its conclusions because Dreyfus mistook "what AI researchers do today" for "what is possible using a computer"), Phil Agre, Rodney Brooks, and anyone who describes their work as "reactive", "behavior-based", or "embodied cognition". We could also point to the analogous linguistic divide. There are two general approaches to natural language understanding, one descending from generative grammars and symbolic AI and embodied by James Allen's book Natural Language Understanding, and in the "program in the knowledge" camp that would answer the first question "yes". The other approach has more kinship with construction grammars and machine learning, and is embodied by Manning & Schutze's Foundations of Statistical Natural Language Processing, and its practitioners would be more likely to answer the first question "no". (Eugene Charniak is noteworthy for having been prominent in both camps.)

The second question, I think, hinges on two sub-questions:

Can we prevent an AI from harvesting more resources than it should for a question?

Can we prevent an AI from conceiving the goal of increasing its own intelligence as a subgoal to answering a question?

The Jack Williamson story "With Folded Hands" (1947) tells how humanity was enslaved by robots given the order to protect humanity, who became... overprotective. Or suppose a physicist asked an AI, "Does the Higgs boson exist?" You don't want it to use the Earth to ...
Rational insanity
by PhilGoetz — 2010-12-27T05:04:24.954Z

My theory on why North Korea has stepped up its provocation of South Korea since their nuclear missle tests is that they see this as a tug-of-war.

Suppose that North Korea wants to keep its nuclear weapons program. If they hadn't sunk a ship and bombed a city, world leaders would currently be pressuring North Korea to stop making nuclear weapons. Instead, they're pressuring North Korea to stop doing something (make provocative attacks) that North Korea doesn't really want to do anyway. And when North Korea (temporarily) stops attacking South Korea, everybody can go home and say they "did something about North Korea". And North Korea can keep on making nukes.
The Fallacy of Dressing Like a Winner
by benelliott — 2010-12-26T20:59:10.172Z

Imagine you are a sprinter, and your one goal in life is to win the 100m sprint in the Olympics. Naturally, you watch the 100m sprint winners of the past in the hope that you can learn something from them, and it doesn't take you long to spot a pattern.

Every one of them can be seen wearing a gold medal around their neck. Not only is there a strong correlation, you also examine the rules of the olympics and find that 100% of winner must wear a gold medal at some point, there is no way that someone could win and never wear a gold medal. So you go out and buy a gold medal from a shop, put it around your neck, and sit back, satisfied.

For another example, imagine that you are now in charge of running a large oil rig. Unfortunately, some of the drilling equipment is old and rusty, and every few hours a siren goes off alerting the divers that they need to go down again and repair the damage. This is clearly not an acceptable state of affairs, so you start looking for solutions.

You think back to a few months ago, before things got this bad, and you remember how the siren barely ever went off at all. In fact, from you knowledge of how the equipment works, the there were no problems, the siren couldn't go off. Clearly the solution the problem is to unplug the siren.

(I would like to apologise in advance for my total ignorance of how oil rigs actually work, I just wanted an analogy)

Both these stories demonstrate a mistake which I call 'Dressing Like a Winner' (DLAW). The general form of the error is when and indicator of success gets treated as an instrumental value, and then sometimes as a terminal value which completely subsumes the thing it was supposed to indicate. As someone noted this can also be seen as a sub-case of the correlative fallacy. This mistake is so obviously wrong that it is pretty much non-existant in near mode, which is why the above stories seem utterly ridiculous. However, once we switch into the more abstract far mode, even the most ridiculous errors become dangerous. In the rest of this post I will point out three places where I think this error occurs.

Changing our minds

In a debate between two people, it is usually the case that whoever is right is unlikely to change their mind. This is not only an empirically observable correlation, but it's also intuitively obvious, would you change...
Alternative Places to Get Ideas (Also, "In Defense of Food")
by Raemon — 2010-12-26T18:36:35.552Z

I discovered Less Wrong a few months ago (courtesy of Harry Potter and the Methods of Rationality), and I am extremely grateful to have such a thoughtful place to have discussions and to learn new things. But one of the more significant hazards I've become aware of is confirmation bias. And since I began coming here a lot, I find that I evaluate new information through a "Less Wrong Lens" which officially means "well researched and thought out" which is perfectly fine but also includes some new or reinforced biases.

I recently realized the extent of the problem while reading "In Defense of Food". The basic premise (or most relevant one) of the book is that while science may one day be able to determine what is healthy on a nutrient-by-nutrient basis and let us craft whatever artificial foods we want, we are not nearly at that point yet. Every few years the prevailing beliefs of the nutritionist and scientific communities change, people scurry to catch up, and regardless, since transition from a "traditional" to "scientific" diet, certain nutrition-related diseases have gotten more prevalent, not less.

His argument is that traditional diets have often had thousands of years to evolve to match the needs and available food sources of populations. So while the variables and interactions may be complex enough that we don't know why, until science DOES figure it out, individual people are better off sticking to the diets of the past. At the same time, corporations that are interested only in marketing as much food as quickly as possible benefit from constantly changing scientific attitudes. (Disclaimer: yes, I'm oversimplifying again, but my point isn't even necessarily about the merits of the book so bear with me).

By the end of the book, I did agree with his basic premises. I'm not sure his solution is the single best one, but it's a large step up from the diet that the average westerner is going to have. It wasn't explicitly anti-science, just attempting to be realistic about what science can realistically accomplish, and what unique pitfalls come from giving science (or more importantly, politicians and corporations on whom scientists are dependent for funding) the position of power that religion and other cultural institutions once had.

But the first half of the book does have a pretty obvious goal, not of discrediting science per se, but "taking the wind out of ...
Tallinn-Evans $125,000 Singularity Challenge
by Kaj_Sotala — 2010-12-26T11:21:22.649Z

Michael Anissimov posted the following on the SIAI blog:

Thanks to the generosity of two major donors; Jaan Tallinn, a founder of Skype and Ambient Sound Investments, and Edwin Evans, CEO of the mobile applications startup Quinly, every contribution to the Singularity Institute up until January 20, 2011 will be matched dollar-for-dollar, up to a total of $125,000.

Interested in optimal philanthropy — that is, maximizing the future expected benefit to humanity per charitable dollar spent? The technological creation of greater-than-human intelligence has the potential to unleash an “intelligence explosion” as intelligent systems design still more sophisticated successors. This dynamic could transform our world as greatly as the advent of human intelligence has already transformed the Earth, for better or for worse. Thinking rationally about these prospects and working to encourage a favorable outcome offers an extraordinary chance to make a difference. The Singularity Institute exists to do so through its research, the Singularity Summit, and public education.

We support both direct engagements with the issues as well as the improvements in methodology and rationality needed to make better progress. Through our Visiting Fellows program, researchers from undergrads to Ph.Ds pursue questions on the foundations of Artificial Intelligence and related topics in two-to-three month stints. Our Resident Faculty, up to four researchers from three last year, pursues long-term projects, including AI research, a literature review, and a book on rationality, the first draft of which was just completed. Singularity Institute researchers and representatives gave over a dozen presentations at half a dozen conferences in 2010. Our Singularity Summit conference in San Francisco was a great success, bringing together over 600 attendees and 22 top scientists and other speakers to explore cutting-edge issues in technology and science.

We are pleased to receive donation matching support this year from Edwin Evans of the United States, a long-time Singularity Institute donor, and Jaan Tallinn of Estonia, a more recent donor and supporter. Jaan recently gave a talk on the Singularity and his life at a entrepreneurial group in Finland. Here’s what Jaan has to say about us:

“We became the dominant species on this planet by being the most intelligent species aro...
A Christmas topic: I have thoughts regarding Chanukah and need logic help from Atheists
by Marius — 2010-12-25T14:24:40.784Z

Essentially, I want to make sure my logic is sound, from the point of view of smart rational people who do not believe in the existence of supernatural miracles.

The Chanukah story: 175-134 BCE. Hellenic Assyrians (Antiochus IV) had conquered Israel, and passed a variety of laws oppressing the freedom of worship of the Jews there. They defiled the Temple and forbade the study of sacred texts. The Maccabees led a Jewish revolt against the Assyrians, and eventually drove them out of Israel. Immediately upon retaking the Temple, they cleaned and rededicated it; they relit the sacred flame using a small vial of kosher oil and sent for more oil (which was 8 days distant). The small vial was expected to last only one night, but miraculously lasted 8 days until more supplies arrived.

Now recently, several Reform rabbis have stated that the fact that the first surviving written record of the miracle is from the Gemara (500 CE) indicates that the miracle was invented around 500 CE. I am not an Orthodox Jew, but I do believe that the Gemara represents the sages writing down oral traditions, and am annoyed by the tendency among certain Reform rabbis to assume that everything was invented at the time it was written regardless of the evidence for or against it.

The texts with the potential to document events follow:

Maccabees 1 (~100 BCE): purely historical/nonreligious. The book was originally written in Hebrew, but that text does not survive. A Greek translation exists, and the text avoids all mention of religious and spiritual matters. For instance, it speaks briefly and euphemistically about the temple, stating that the Jews captured the "temple hill" and rededicated the "citadel", avoiding mention of the temple itself.

Maccabees 2 (~30 BCE): we possess what claims to be a 1-volume abridgement of a 5-volume original (which does not survive and is not referenced elsewhere). The surviving abridgement mentions the temple rededication and a variety of bizarre miracles including the public appearance of angels. It does not mention the miracle of the oil, however. The abridged text includes a number of theologic innovations which bear more similarity to Catholic beliefs than to Jewish or Protestant ones; it is unknown whether these were present in the origi...