trentmkelly/LessWrong-43k huggingface dataset:
Why we should fear the Paperclipper [Link]
by XiXiDu — 2011-02-14T19:24:54.280Z

The Scenario

A programmer has constructed an artificial intelligence based on an architecture similar to Marcus Hutter's AIXI model (see below for a few details). This AI will maximize the reward given by a utility function the programmer has given it. Just as a test, he connects it to a 3D printer and sets the utility function to give reward proportional to the number of manufactured paper-clips.

At first nothing seems to happen: the AI zooms through various possibilities. It notices that smarter systems generally can make more paper-clips, so making itself smarter will likely increase the number of paper-clips that will eventually be made. It does so. It considers how it can make paper-clips using the 3D printer, estimating the number of possible paper-clips. It notes that if it could get more raw materials it could make more paper-clips. It hence figures out a plan to manufacture devices that will make it much smarter, prevent interference with its plan, and will turn all of Earth (and later the universe) into paper-clips. It does so.

Only paper-clips remain.

Link: aleph.se/andart/archives/2011/02/why_we_should_fear_the_paperclipper.html
Algorithms as Case Studies in Rationality
by abramdemski — 2011-02-14T18:27:47.170Z

This post springs out of a very long line of thought, which I will only summarise small parts of.

It began with the puzzling realisation that the algorithm which computers use to perform symbolic integration is radically different from the guess-and-check method taught in schools. My first reaction was, why are we not taught the systematic way of doing it? This is true for other areas of mathematics as well. In a few cases, such as solving quadratic equations or systems of linear equations, students are eventually taught the fast way. However, in many cases, the existence of an algorithm is not even mentioned.

My point, however, is not to criticise the educational practice; in fact, I agree with the idea of teaching mathematics as an exploration of ideas rather than an application of formulaic solution methods. Rather, I would like to encourage people to eventually learn the algorithms, and try to apply them. A good algorithm is a study in rational behaviour, and I think we can take home a lesson from each.

I'll just give two examples which I find particularly fascinating: Knuth-Bendix completion and the summary-product algorithm. The first is most relevant to fast mathematical reasoning. The second is relevant to studying the reasonableness of fast-and-messy probabilistic reasoning, the way humans do it.

Knuth-Bendix Completion

Knuth-Bendix completion ("K-B" from now on) is a fascinating formalisation of the mathematical idea of simplification. Everyone should recognise the phrase "simplify your answer" from high school homework assignments. I never suspected that simplification could be a really powerful method for mathematical reasoning.

K-B is a method for reasoning about equality. Equality is one of the simplest relationships we can have between two things, yet the naive way of reasoning about it results in an explosion of possible inferences. If there are N things in our universe, there are N2 equality statements which may be true or false. We can substitute equals for equals all day and get nowhere. So why does it seems like such a simple relation? I think the K-B algorithm gives a good answer to that.

We start with the basic axioms for the domain we are reasoning about. The domain consists of a set of expressions, some of which are equal to others. The axioms provide some general equalities; for example, they might be the trigonometric identities, if we a...
IBM's "Watson" program to compete against "Jeopardy" champions tonight
by NihilCredo — 2011-02-14T15:28:39.168Z

It was mentioned before on LessWrong, but I feel people might appreciate a reminder:

http://www-03.ibm.com/innovation/us/watson/what-is-watson/countdown-to-jeopardy.html

It's a bit of a cheesy PR thing - I'd be a lot more interested if they connected the program on the Internet and allowed anyone to try and ask them general questions, rather than mixing the program with voice recognition and (heh) buzzer-pushing. Trivia tests are also probably one of the easier challenges to deal with, since keyword filtering alone is very efficient in narrowing down the candidate space.

Still, I'm going to watch it if I can: if anybody knows of a streaming link that is accessible to non-US viewers, that would be appreciated.

(Silly aside: is anyone else annoyed by how "Jeopardy" pretends to invert the traditional question-answer format, while what it does is simply moving the "what is" from the former to the latter, even if the result makes no sense? I suppose to US people this is a rather old complaint, but I learnt about the show today and I'm rather bugged by this feature.)
Meetup posts as discussion threads, please
by Jonii — 2011-02-14T11:49:40.201Z

As of now, 4 of 10 newest promoted posts are about meetups, as well 4 of 10 newest posts overall. For casual readers like me, having frontpage flooded by this much irrelevant information, _especially promoted-section_, seems really, really discouraging. LW has tendency to contain too much useless meta-discussion compared to the actual rationality-related one, but having frontpage flooded by meta-discussion like this seems rather unbeliveable. Please, let's try to keep at least the promoted-section rationality-related.
Why Do We Engage in Moral Simplification?
by Wei Dai — 2011-02-14T01:16:40.615Z

It appears to me that much of human moral philosophical reasoning consists of trying to find a small set of principles that fit one’s strongest moral intuitions, and then explaining away or ignoring the intuitions that do not fit those principles. For those who find such moral systems attractive, they seem to have the power of actually reducing the strength of, or totally eliminating those conflicting intuitions.

In Fake Utility Functions, Eliezer described an extreme version of this, the One Great Moral Principle, or Amazingly Simple Utility Function, and suggested that he was partly responsible for this phenomenon by using the word “supergoal” while describing Friendly AI. But it seems to me this kind of simplification-as-moral-philosophy has a history much older than FAI.

For example, hedonism holds that morality consists of maximizing pleasure and minimizing pain, utilitarianism holds that everyone should have equal weight in one’s morality, and egoism holds that moralist consists of satisfying one’s self-interest. None of these fits all of my moral intuitions, but each does explain many of them. The puzzle this post presents is: why do we have a tendency to accept moral philosophies that do not fit all of our existing values? Why do we find it natural or attractive to simplify our moral intuitions?

Here’s my idea: we have a heuristic that in effect says, if many related beliefs or intuitions all fit a certain pattern or logical structure, but a few don’t, the ones that don’t fit are probably caused by cognitive errors and should be dropped and regenerated from the underlying pattern or structure.

As an example where this heuristic is working as intended, consider that your intuitive estimates of the relative sizes of various geometric figures probably roughly fit the mathematical concept of “area”, in the sense that if one figure has a greater area than another, you’re likely to intuitively judge that it’s bigger than the other. If someone points out this structure in your intuitions, and then you notice that in a few cases your intuitions differ from the math, you’re likely to find that a good reason to change those intuitions.

I think this idea can explain why different people end up believing in different moral philosophies. For example, many members of this community are di...
Open Thread: Mathematics
by bentarm — 2011-02-13T23:50:50.941Z

In Luke's recent post on what sort of posts we would like to see more of, one suggestion was "Open Thread: Math". This suggestion has been voted up by (at least) 12 people. Since it's going to take me less than 2 minutes to type this post, I figured I might as well just go ahead and post the thread, rather than vote up the suggestion.

So, this is an open thread on mathematics. As things stand, I have no idea what the rules should be (I don't know what the people who voted up the post suggestion expected the rules to be), but I guess the general principle should be that we have maths questions which are vaguely related to LW-type ideas, as there are plenty of more appropriate fora for general mathematical discussion already out there.
Social Necessity of Drinking
by Raemon — 2011-02-13T22:52:44.211Z

It's been over a year since I graduated from college, but only recently have I felt like I'm officially entering the "adult world." Navigating the social arenas of the adult world requires the same basic skillsets as the college world, but a lot of the rules are different and I'm struggling to learn them. Among them is how to drink socially.

As a general rule, I don't drink. I don't like the taste of alcohol. I don't like paying the exorbinant prices that alcohol costs. I don't like the feeling of my brain slowing down and making it harder to string sentences together. I don't mind the physical disorientation - that part's pretty fun. But that part also seems to be slightly frowned upon in an "adult" setting. I'm not opposed to it for any particular moral reasons.

When I do drink, I prefer to get it over with as fast as possible, whether I'm officially drinking a "shot" or not. In college that at least had a sort of "daring" quality that was respected. But it's pretty obviously taboo at classy cocktail parties and even somewhat taboo at "casual adult" parties.

So there's a few separate questions I have:

1) Are there any good, cached buzzword phrases I can use that'll make it socially acceptable to not drink? "I just don't like it" seems to draw disdainful stares, and while I haven't tried it I get the sense that saying I'm morally opposed to it would make me look even more like a stick in the mud. Saying "it's ridiculously expensive" makes me look like a cheapskate.

2) If I must drink socially, is there a breakdown of the general social conventions I should be aware of so I don't need to have them pointed out to me over the course of the next few years?

3) Is there any particularly interesting analysis of *why* drinking is so important to social interaction? Knowing the underlying causes might at least give me some better appreciation for why I have to learn this other than "because!"
Farmington Hills, MI Less Wrong meetup: Sunday, February 20
by Psy-Kosh — 2011-02-13T18:42:18.958Z

Where: Udipi Restaurant, on Orchard Lake road, between 12 and 13 mile road (much closer to 13 mile)

When: Sunday, February 20, 12:30pm (This can be moved a bit later or such if that's better for anyone)

Unless anyone who would otherwise come except for the date/time being bad, that's what I'm going to go with.

If you plan on showing up, go ahead and comment here so I'll have an idea about that. (Although if you don't comment and you find that you do still want to show up, don't feel that your not having commented is a reason not to show up.)

EDIT: From the discussion thread, some people in Ann Arbor would come if they had a ride. So if anyone is coming from Ann Arbor and is willing to give a ride, please mention that. Thanks.
Exercise and motivation
by NancyLebovitz — 2011-02-13T13:56:33.060Z

I realized very recently that what gets me to exercise is whether it makes moving more of a pleasure, and, of course, if it's a pleasure in itself.

Any hint of "prove you're a worthwhile person by how much you can make yourself endure" or "not being fat is the most important thing in the world"[1] is apt to be demotivating.

It might be a good thing if being able to react well in emergencies[2] were a strong motivation for me, but the truth is, I've had a pretty easy life, and the only thing that motivates me in that range is wanting to be able to walk safely on ice.

So, what tends to increase your pleasure in movement? Do you know of any systems organized around enjoyment?

[1] I know they never say that, but if an exercise system has fat loss as the only or first reason listed for engaging in it, that's how I interpret it.

[2] This interview with Scott Sonnon is absolutely the most rational thing I've seen on the subject. He's focused very hard on doing things that work rather than things that seem as though they might indicate that something will work. And his emphasis is that the best exercise program is one that you will keep doing, both because you're willing to stay with it and because it doesn't hurt you is what inspired this post.

This was posted here and here-- there are good comments in both places. It's suboptimal to have more than one comment stream, but I've got friends who will only read at one of the sites, and I'd expect that you guys are more likely to comment here than at either of those.

And also, what are your actual motivations? Was it hard to discover them, and if so, did it help when you did?

A follow-up: I've been doing The Five Tibetans (a cross between yoga and calisthenics) for a while, and I've had a hard time with the fourth one. My arms are relatively short and my shoulders and chest are tight.

I tried taking pleasure seriously when I did it, and that, combined with "relax, breathe, feel the earth, do nothing extra" from Restore Yourself with Tai Chi made a huge fast improvement. Instead of struggling with the move, I was able to get a significant amount of weight on to my feet and feel a coherent stretch across the front of my body instead of separate parts making unpleasant efforts.

My reaction to pain was better-- I could use it as a signal to change what I was doing instead of feeling as though the universe was out to get me or as thou...
BOOK DRAFT: 'Ethics and Superintelligence' (part 1)
by lukeprog — 2011-02-13T10:09:12.769Z

I'm researching and writing a book on meta-ethics and the technological singularity. I plan to post the first draft of the book, in tiny parts, to the Less Wrong discussion area. Your comments and constructive criticisms are much appreciated.

This is not a book for a mainstream audience. Its style is that of contemporary Anglophone philosophy. Compare to, for example, Chalmers' survey article on the singularity.

Bibliographic references are provided here.

Part 1 is below...

Chapter 1: The technological singularity is coming soon.

The Wright Brothers flew their spruce-wood plane for 200 feet in 1903. Only 66 years later, Neil Armstrong walked on the moon, more than 240,000 miles from Earth.

The rapid pace of progress in the physical sciences drives many philosophers to science envy. Philosophers have been researching the core problems of metaphysics, epistemology, and ethics for millennia and not yet come to consensus about them like scientists have for so many core problems in physics, chemistry, and biology.

I won’t argue about why this is so. Instead, I will argue that maintaining philosophy’s slow pace and not solving certain philosophical problems in the next two centuries may lead to the extinction of the human species.

This extinction would result from a “technological singularity” in which an artificial intelligence (AI) of human-level general intelligence uses its intelligence to improve its own intelligence, which would enable it to improve its intelligence even more, which would lead to an “intelligence explosion” feedback loop that would give this AI inestimable power to accomplish its goals. If so, then it is critically important to program its goal system wisely. This project could mean the difference between a utopian solar system of unprecedented harmony and happiness, and a solar system in which all available matter is converted into parts for a planet-sized computer built to solve difficult mathematical problems.

The technical challenges of designing the goal system of such a superintelligence are daunting.[1] But even if we can solve those problems, the question of which goal system to give the superintelligence remains. It is a question of philosophy; it is a question of ethics.

Philosophy has impacted billions of humans through religion, culture, and government. But now

...
Bridging Inferential Gaps and Explaining Rationality to Other People
by atucker — 2011-02-13T06:22:06.488Z

This post is going in discussion until I get it edited enough that I feel like its post-worthy, or if it does well.

Core Post:

Rationality has helped me do a lot of things (in the past year: being elected President of my robotics team, getting a girlfriend, writing good college apps (and getting into a bunch of good schools), etc.), and I feel sort of guilty for not helping other people use it.

I had made progress on a lot of those fronts before, but a bunch of things fell into place in a relatively short period of time after I started trying to optimize them. Some of my friends have easyish problems, but unsolicited risky counterintuitive advice is uncouth and unhelpful.

More pressingly, I want to pass on a lot of rationality knowledge to people I know before I graduate high school. Being in a fairly good Math/Science/Computer Science Magnet Program, I have access to a lot of smart, driven people who have a lot of flexibility in their lives and I think it would be a shame if there were things I could tell them that would make them do a lot better. On top of that, I want to pass on this knowledge within my robotics team so that they continue doing well.

Basically, I want to learn how to explain useful rationality concepts to other people in a non-annoying and effective way. As far as I can tell, many people want to do similar things, and find it difficult to do so.

I suspect that this topic is broad enough that it would be hard for a single person to tackle it in one post. So that people don't need to have enough information for an entire post (which, would be awesome by the way) before they talk about it, here's a thread to respond to.

I'd particularly like to encourage people who have successfully bridged inferential distances to reply with where people started and how the conversation went. Please. An example:

In my Origins of Science (basically a philosophy) class, a conversation like this (paraphrased, happened a few days ago) took place. I'm not sure where the other people in the class started, but it got them to the point that they understood how you model reality, but that beliefs are supposed to reflect reality, and you can't just make things up entirely.

W: "I feel like if people want to think God exists, then God exists for them, but if they want to ignore him then he won't."

me: "But that's not how existing works. In our tho

...
Science: Do It Yourself
by alyssavance — 2011-02-13T04:47:46.252Z

In the nerd community, we have lots of warm, fuzzy associations around 'science'. And, of course, science is indeed awesome. But, seeing how awesome science is, shouldn't we try to have more of it in our lives? When was the last time we did an experiment to test a theory?

Here, I will try to introduce a technique which I have found to be very useful. It is based on the classical scientific method, but I call it "DIY Science", to distinguish it from university science. The point of DIY Science is that science is not that hard to do, and can be used to answer practical questions as well as abstract ones. Particle physics looks hard to do, since you need expensive, massive accelerators and magnets and stuff. However, fortunately, some of the fields in which it is easiest to do science are some of the most practical and interesting. Anyone smart and rational can start doing science right now, from their home computer.

One of the key ingredients of DIY Science is to discard the more useless trappings of university science, for these frequently do more harm than good. Science doesn't need journals and universities. Science doesn't need beakers and test tubes. Science doesn't need p < 0.05, although I have found p-tests to be occasionally useful. The point of science is not to conform to these stereotypes of academia, but to discover something you didn't know before. (To our detriment, this is the opposite of how science is taught, as noted by Paul Graham: "So hackers start original, and get good, and scientists start good, and get original.")

Instead, as an simple first example, consider this question:

- I want to get rich, or to be specific, have a net worth of over $100M USD. How do people get rich?

Here, we have an opportunity: We don't know something, and we want to find out what it is. To answer this question, our first intuition might be to Google "how do people get rich?". This isn't a horrible method, but by just asking someone else, we are not doing any science. Googling or asking a friend isn't the scientific method; it's the medieval method. (In medieval times, we would just have gone to the Church and asked, and the Church would have replied, "Pray diligently to the LORD and have faith, and you will be prosperous." Different people, same thing.)

In fields like physics, where lots of science is already being done by others, this will probably be OK. ...
Link: Cryonics and the Creation of a Durable Morality
by lsparrish — 2011-02-12T18:10:32.172Z

From Mike Darwin's new blog:

DCD has lead to a fracture within the medical community [7,8] wherein some centers, such as the University of Pittsburgh Medical Center, have taken patients who want ventilator support withdrawn, placed femoral cannulae under local (spinal) anesthesia, turned off the ventilator after effectively anesthetizing the patient, waited until the patient’s heart stops, and then restarted circulation with CPB. They also, of course, give paralytic neuromuscular blocking drugs (as is routine in all visceral organ retrieval) to prevent the thoracoabdominal incision, and the terminal drop in blood pressure (when the organs are removed), from causing muscle vesiculations (twitching) or actual limb movement as a result of stimulation of the nocioceptive pathways in the spinal cord (pain is a local phenomenon first and a central nervous system one secondly with the process proceeding up the spinal cord to the brain). [9,10]

To be blunt, this procedure resulted in all hell breaking out. [11,12,13] Bioethicists, such James Bernat and Leslie Whetstine, accused the surgeons and neurologists involved in this undertaking of every ethical evil, including homicide.[14,15] A compromise position is to restore circulation in the body using a special balloon-tipped aortic catheter that prevents ‘all ‘ flow to the brain. This results in a ‘resolution’ to the ‘paradox’ of removing organs from a patient with a ‘viable, or potentially viable brain.’ Of course, from our perspective as cryonicists, this whole exercise is nothing more or less than a procedural contortion designed to avoid confronting the reality that death is not a binary condition, and that if you are going to allow people to withdraw from medical care they no longer want, and that they (rightfully) consider an assault, then the corollary to that is that they also get to decide when they are dead. [16] That means that they have the perfect right to ask for, and receive a treatment (i.e., in the presence of informed consent) whereby they are anesthetized, cooled, subjected to blood washout, and their organs removed – at which point they are indeed DEAD, in the sense that their non-functional condition is now irreversible, or not going to be reversed, because they do not want it to be. When, exactly, they become irrecoverable from an infor

...