trentmkelly/LessWrong-43k huggingface dataset:
Division of cognitive labour in accordance with researchers' ability
by Stefan_Schubert — 2014-01-16T09:28:02.920Z

The variance in productivity between people is much greater in some areas than in others. The most productive cleaner is probably less than twice as productive as the average cleaner, whereas the most productive entrepreneur arguably is many times more productive than the average entrepreneur. One area where this difference in productivity is particularly great is research - especially in its more theoretical parts. A physicist once told me that Einstein put forth four theories worthy of the Nobel Prize in one year alone (1905, his Annus Mirabilis). That's a level of productivity that, on any reasonable measure, dwarfs that of whole armies of lesser physicists. The same pattern recurs in many disciplines (though it is perhaps not always quite as strong).

Such a recurring pattern cries out for an explanation. In some areas the reason might be hero-worship (postmodernist philosophy springs to mind), but the pattern is strong in many serious disciplines as well, so that can't be the whole explanation.

Instead, I would argue that the reason is that science is a social activity in a much deeper sense than cleaning is. A scientific problem is "hard" due to the fact that most scientists are not sufficiently talented to solve it given our present knowledge. In most sciences, there will normally at any given point be a lot of such problems which are too hard for most scientists, but solvable by a few really smart people.

I'm not saying that but for them, there would be no progress. Clearly there would, but we would need more knowledge - clues, as it were - to solve the hard problems. Thus progress would go much slower.

If this picture - rather an elitist one - is right, what are the consequences for policy? It seems to me that the most obvious consequence is that we need to make sure that the most talented people's time is spent in a rational way. Firstly, it does not seem to me that they should spend time teaching students - their time is far too valuable for that. Overall, it's a bit strange that researchers are expected to teach since they really are two quite different jobs.

That's not my main point, however. My main point is that they should focus on what what they're best at: producing new ideas. Research today involves so much more - applying for grants, writing up articles that have to conform to all sorts of disciplinary conventions, attending seminars...
Things I Wish They'd Taught Me When I Was Younger: Why Money Is Awesome
by ChrisHallquist — 2014-01-16T07:27:47.032Z

There are some things money can't buy. They are the exceptions that prove the rule.

For the pedants, to say something is an exception that proves the rule is to say that when you look at the exceptions, they're so unusual that it reinforces the point that the rule is generally valid even though it isn't universally valid. In the case of money, there's a reason people don't say things like "there are some things hand-knit scarves can't be bartered for" or "Hand-knit scarves can't be bartered for happiness."

Eliezer once described the sequences as the letter he wishes he could have written to his former self. When I think of the letter I wish I could write to my former self, the value of money is at the top of the list of things I'd include.

You can give a cynical, Hansonian explanation of why we don't tell young people enough about the awesomeness of money, and I suppose there'd be some truth to it. But I'm not sure that was my main problem. Growing up, my dad spent a lot of time urging me to go into a high-paying career, to the point giving me advice on what medical specialty to go into. He just didn't do a great job of selling me on it. It wasn't until I learned some economics that I really came to understand why money is so awesome.

(Disclaimer: I don't actually know that much economics, and in fact have never taken an economics course. I just know more than my former self.)

The first thing to understand about money is that the range of things you can get for it is really incredibly huge. Econ bloggers Tyler Cowen and Alex Tabarrok periodically do posts called "markets in everything" where they highlight some of the weirder examples of this, but the weird examples matter less than the obvious examples people just don't think about much. There's a tendency to associate money with a narrow range of things rich people stereotypically spend their money on. Or, in my case growing up in an upper middle-class family, there were the family vacations and boats that my dad seemed to mainly spend his money on, which were nice but didn't seem particularly worth planning my career around.

Yet not only is the range of things you can get with money huge, even with things you can get without money, spending money on them is often a better way of acquiring them. The reason for this is comparative advantage, a concept that gets discussed a lot in the context of nation...
Dangers of steelmanning / principle of charity
by gothgirl420666 — 2014-01-16T06:35:31.625Z

As far as I can tell, most people around these parts consider the principle of charity and its super saiyan form, steelmanning, to be Very Good Rationalist Virtues. I basically agree and I in fact operate under these principles more or less automatically now. HOWEVER, no matter how good the rule is, there are always exceptions, which I have found myself increasingly concerned about.

This blog post that I found in the responses to Yvain's anti-reactionary FAQ argues that even though the ancient Romans had welfare, this policy was motivated not for concern for the poor or for a desire for equality like our modern welfare policies, but instead "the Roman dole was wrapped up in discourses about a) the might and wealth of Rome and b) goddess worship... The dole was there because it made the emperor more popular and demonstrated the wealth of Rome to the people. What’s more, the dole was personified as Annona, a goddess to be worshiped and thanked."

So let's assume this guy is right, and imagine that an ancient Roman travels through time to the present day. He reads an article by some progressive arguing (using the rationale one would typically use) that Obama should increase unemployment benefits. "This makes no sense," the Roman thinks to himself. "Why would you give money to someone who doesn't work for it? Why would you reward lack of virtue? Also, what's this about equality? Isn't it right that an upper class exists to rule over a lower class?" Etc.

But fortunately, between when he hopped out of the time machine and when he found this article, a rationalist found him and explained to him steelmanning and the principle of charity. "Ah, yes," he thinks. "Now I remember what the rationalist said. I was not being so charitable. I now realize that this position kind of makes sense, if you read between the lines. Giving more unemployment benefits would, now that I think about it, demonstrate the power of America to the people, and certainly Annona would approve. I don't know why whoever wrote this article didn't just come out and say that, though. Maybe they were confused".

Hopefully you can see what I'm getting at. When you regularly use the principle of charity and steelmanning, you run the risk of:

1. Sticking rigidly to a certain worldview/paradigm/established belief set, even as you find yourself willing to consider more and more concre...
LessWrong Help Desk - free paper downloads and more (2014)
by jsalvatier — 2014-01-16T05:51:40.710Z

Over the last year, VincentYu, gwern and others have provided many papers for the LessWrong community (87% success rate in 2012) through previous help desk threads. We originally intended to provide editing, research and general troubleshooting help, but article downloads are by far the most requested service.

If you're doing a LessWrong relevant project we want to help you. If you need help accessing a journal article or academic book chapter, we can get it for you. If you need some research or writing help, we can help there too.

Turnaround times for articles published in the last 20 years or so is usually less than a day. Older articles often take a couple days.

Please make new article requests in the comment section of this thread.

If you would like to help out with finding papers, please monitor this thread for requests. If you want to monitor via RSS like I do, many RSS readers will give you the comment feed if you give it the URL for this thread (or use this link directly).

If you have some special skills you want to volunteer, mention them in the comment section.
The first AI probably won't be very smart
by Jonathan Paulson — 2014-01-16T01:37:49.048Z

Claim: The first human-level AIs are not likely to undergo an intelligence explosion.

1) Brains have a ton of computational power: ~86 billion neurons and trillions of connections between them. Unless there's a "shortcut" to intelligence, we won't be able to efficiently simulate a brain for a long time. http://io9.com/this-computer-took-40-minutes-to-simulate-one-second-of-1043288954 describes one of the largest computers in the world simulating 1s of brain activity in 40m (i.e. this "AI" would think 2400 times slower than you or me). The first AIs are not likely to be fast thinkers.
2) Being able to read your own source code does not mean you can self-modify. You know that you're made of DNA. You can even get your own "source code" for a few thousand dollars. No humans have successfully self-modified into an intelligence explosion; the idea seems laughable.
3) Self-improvement is not like compound interest: if an AI comes up with an idea to modify it's source code to make it smarter, that doesn't automatically mean it will have a new idea tomorrow. In fact, as it picks off low-hanging fruit, new ideas will probably be harder and harder to think of. There's no guarantee that "how smart the AI is" will keep up with "how hard it is to think of ways to make the AI smarter"; to me, it seems very unlikely.
Results from MIRI's December workshop
by Benya — 2014-01-15T22:29:57.368Z

Last week (Dec. 14-20), MIRI ran its 6th research workshop on logic, probability, and reflection. Writing up mathematical results takes time, and in the past, it's taken quite a while for results from these workshops to become available even in draft form. Because of this, at the December workshop, we tried something new: taking time during and in the days immediately after the workshop to write up results in quick and somewhat dirty form, while they still feel fresh and exciting.

In total, there are seven short writeups. Here's a list, with short descriptions of each. Before you get started on these writeups, you may want to read John Baez's blog post about the workshop, which gives an introduction to the two main themes of the workshop.

Theme 1: Scientific induction in mathematics

One of the main themes was using Bayesian probability to represent uncertainty about mathematical statements. Like human mathematicians, an AI will be able to outright prove or disprove many mathematical statements, but there will also be many that it will be uncertain about, and the obvious thing to try to get a handle on dealing with such uncertainty is to assign a probability to each such statement. This would mean choosing some sort of prior, and then updating on evidence: For example, if you're not sure whether the twin prime conjecture is true, then each time you discover a new twin prime larger than all that you have seen before, you should ever so slightly increase the probability you assign to the conjecture.

But what sort of prior should we choose? That's the problem that people at the December workshop tried to make some progress on. Here is an interesting problem which the best previous proposal, due to Abram Demski, fails on: Suppose that Q(x) is a predicate symbol, and suppose that you find evidence that Q(x) is true of exactly 90% of all numbers between x=1 and x=10^100. Then we would expect that if you plug in some arbitrary number in this range, say n = floor(7^7^7^7 * pi) mod 10^100, then the posterior probability of Q(n), after conditioning on your evidence, would be 0.9. But it turns out that in Demski's proposal, the probability of Q(n) would be approximately 0.5.

To learn more about this, see "Scientific Induction in Probabilistic Mathematics", written up by Jeremy Hahn. Jeremy writes: "At the workshop, after pinning down the above example of undesired behaivio...
Meetup : Berkeley meetup: 5-minute exercises
by Nisan — 2014-01-15T21:02:26.223Z

Discussion article for the meetup : Berkeley meetup: 5-minute exercises

WHEN:
15 January 2014 07:00:00PM (-0800)

WHERE:
2030 Addison, Berkeley, CA

At today's meetup, I want to try a kind of exercise I learned / shamelessly stole from User:apophenia . The idea is to try a rationality / cognition exercise that's as valuable as can be while only taking 5 minutes. See this example:

http://lesswrong.com/lw/irr/the_best_15_words/

Please arrive between 7pm and 7:30pm today. The exercise will begin at 7:30pm. It won't take very long, and we will hang out afterward. The CFAR office is at 2030 Addison, 3rd floor, Berkeley, near the Downtown Berkeley BART. If you find yourself locked out, text me at

http://i.imgur.com/Vcafy.png

Even though this takes place at CFAR, it's not a CFAR-sponsored event.

Discussion article for the meetup : Berkeley meetup: 5-minute exercises
Recreational Cryonics
by topynate — 2014-01-15T20:21:36.011Z

We recently saw a post in Discussion by ChrisHallquist, asking to be talked out of cryonics. It so happened that I'd just read a new short story by Greg Egan which gave me the inspiration to write the following:

It is likely that you would not wish for your brain-state to be available to all-and-sundry, subjecting you to the possibility of being simulated according to their whims. However, you know nothing about the ethics of the society that will exist when the technology to extract and run your brain-state is developed. Thus you are taking a risk of a negative outcome that may be less attractive to you than mere non-existence.

I had little expectation of this actually convincing anyone, but thought it was a fairly novel contribution. When jowen's plea for a refutation went unanswered, I began attempting one myself. What I ended up with closes the door on the scenario I outlined, but opens one I find rather more disturbing.

I think I'd better start by explaining why I wrote my comment the way that I did.

Normally, when being simulated is raised as a negative possibility (referred to in SF as being 'deathcubed', and carrying the implication not so much of torture as of arbitrariness), it's in the context of an AI doing so. Now there's a pretty good argument against being deathcubed by an AI, as follows:

Any AI that would do this is unFriendly. The vast majority of uFAIs have goals incompatible with human life but not in any way concerned with it. Humans are just matter that can be better used for something else; likewise simulations use computational resources better used for something else. Therefore there is little to fear in the way of being tortured by an AI.

I sidestepped that entire argument (but I'll return to it in a minute) by referring to "the ethics of the society that will exist". In other words, without making it explicit, I created the image of a community of agents, probably human-like agents, each with ownership of resources that they could use according to a moral code and subject to some degree of enforcement. I assumed a naturalistic polity, rather than a hegemony.

With the assumptions behind my scenario laid bare, it should now be apparent that it is no more stable than a world in which everyone owns nuclear weapons. If the resources to do these simulations are dispersed amongst that many people, someone wi...
The non-Independence of superficially-Irrelevant Alternatives
by roystgnr — 2014-01-15T18:46:29.053Z

A new article in Biology Letters shows that under some conditions in which animals appear to behave "irrationally" (by apparently failing to conform to the Independence of Irrelevant Alternatives or even the Transitivity axioms of decision theory), the animal behavior may in fact resemble utility-maximizing strategies which also appear to violate those axioms. The optimal strategies' current preferences are altered in response to the information conveyed by the current presence or absence of various alternatives.

A press release about the article is available from the lead author's university, U. Bristol. A news piece summarizing it is at Nature's website.

These results probably shouldn't surprise the most careful rational thinkers. For instance, "I prefer A to B when I believe more B is likely to be available later, and B to C when I believe more B is likely to be available later" clearly does not necessarily entail "I prefer A to B under all circumstances, and B to C under all circumstances". But there is clearly a pitfall of tempting model oversimplification here. Although this new paper and most discussion of it is carefully putting scare quotes around "irrational", the papers being cited purport to show "intransitive preferences", "violations of rational choice", "irrational decision-making", and "irrational choices", without obvious irony.
To capture anti-death intuitions, include memory in utilitarianism
by Kaj_Sotala — 2014-01-15T06:27:28.901Z

EDIT: Mestroyer was the first one to find a bug that breaks this idea. Only took a couple of hours, that's ethics for you. :)

In the last Stupid Questions Thread, solipsist asked

Making a person and unmaking a person seem like utilitarian inverses, yet I don't think contraception is tantamount to murder. Why isn't making a person as good as killing a person is bad?

People raised valid points, such as ones about murder having generally bad effects on society, but most people probably have the intuition that murdering someone is bad even if the victim was a hermit whose death was never found out by anyone. It just occurred to me that the way to formalize this intuition would also solve more general problems with the way that the utility functions in utilitarianism (which I'll shorten to UFU from now on) behave.

Consider these commonly held intuitions:

If a person is painlessly murdered and a new (equally happy) person is instantly created in their place, this is worse than if there was a single person who lived for the whole time.

If a living person X is painlessly murdered at time T, then this is worse than if the X's parents had simply chosen not to have a child at time T-20, even though both acts would have resulted in X not existing at time T+1.

Also, the next intuition isn't necessarily commonly held, since it's probably deemed mostly science fictiony, but in transhumanist circles one also sees:

If someone is physically dead, but not information-theoretically dead and a close enough replica of them can be constructed and brought back, then bringing them back is better than creating an entirely new person.

Assume that we think the instrumental arguments in favor of these intuitions (like societies with fewer murders being better off for everyone) are insufficient - we think that the intuitions should hold even if disregarding them had no effect on anything else. Now, many forms of utilitarianism will violate these intuitions, saying that in all cases both of the offered scenarios are equally good or equally bad.

The problem is that UFUs ignore the history of the world, looking only at individual states. By analogy to stochastic processes, we could say that UFUs exhibit the Markov property: that is to say, the value of a state depends only on that state, not on the sequence of events that preceded it. When deciding whether a possible world at ti

...
Thought Crimes
by Scott Garrabrant — 2014-01-15T05:23:32.825Z

Cross-posted on By Way of Contradiction

In my morals, at least up until recently, one of the most obvious universal rights was freedom of thought. Agents should be allowed to think whatever they want, and should not be discouraged for doing so. This feels like a terminal value to me, but it is also instrumentally useful. Freedom of thought encourages agents to be rational and search for the truth. If you are punished for believing something true, you might not want to search for truth. This could slow science and hurt everyone. On the other hand, religions often discourage freedom of thought, and this is a major reason for my moral problems with religions. It is not just that religions are wrong, everyone is wrong about lots of stuff. It is that many religious beliefs restrict freedom of thought by punishing doubters with ostracizing or eternal suffering. I recognize that there are some "religions" which do not exhibit this flaw (as much).

Recently, my tune has changed. There are two things which have caused me to question the universality of the virtue of freedom of thought:

1) Some truths can hurt society

Topics like unfriendly artificial intelligence make me question the assumption that I always want intellectual progress in all areas. If we as modern society were to choose any topic which restricting thought about might be very useful, UFAI seems like a good choice. Maybe the freedom of thought in this issue might be a necessary casualty to avoid a much worse conclusion.

2) Simulations

This is the main point I want to talk about. If we get to the point where minds can simulate other minds, then we run into major issues. Should one mind be allowed to simulate another mind and torture it? It seems like the answer should be no, but this rule seems very hard to enforce without sacrificing not only free thought, but what would seem like the most basic right to privacy. Even today, people can have preferences over the thoughts of other people, but our intuition tells us that the one who is doing the thinking should get the final say. If the mind is simulating another mind, shouldn't the simulated mind also have rights? What makes advanced minds simulating torture so much worse than a human today thinking about torture. (Or even worse, thinking about 3^^^^3 people with dust specks in their eyes. (That was a joke, I know we cant actually think about 3^^^^3 people.))

Th...