trentmkelly/LessWrong-43k huggingface dataset:
"Smarter than us" is out!
by Stuart_Armstrong — 2014-02-25T15:50:35.350Z

We're pleased to announce the release of "Smarter Than Us: The Rise of Machine Intelligence", commissioned by MIRI and written by Oxford University’s Stuart Armstrong, and available in EPUB, MOBI, PDF, and from the Amazon and Apple ebook stores.

What happens when machines become smarter than humans? Forget lumbering Terminators. The power of an artificial intelligence (AI) comes from its intelligence, not physical strength and laser guns. Humans steer the future not because we’re the strongest or the fastest but because we’re the smartest. When machines become smarter than humans, we’ll be handing them the steering wheel. What promises—and perils—will these powerful machines present? This new book navigates these questions with clarity and wit.

Can we instruct AIs to steer the future as we desire? What goals should we program into them? It turns out this question is difficult to answer! Philosophers have tried for thousands of years to define an ideal world, but there remains no consensus. The prospect of goal-driven, smarter-than-human AI gives moral philosophy a new urgency. The future could be filled with joy, art, compassion, and beings living worthwhile and wonderful lives—but only if we’re able to precisely define what a “good” world is, and skilled enough to describe it perfectly to a computer program.

AIs, like computers, will do what we say—which is not necessarily what we mean. Such precision requires encoding the entire system of human values for an AI: explaining them to a mind that is alien to us, defining every ambiguous term, clarifying every edge case. Moreover, our values are fragile: in some cases, if we mis-define a single piece of the puzzle—say, consciousness—we end up with roughly 0% of the value we intended to reap, instead of 99% of the value.

Though an understanding of the problem is only beginning to spread, researchers from fields ranging from philosophy to computer science to economics are working together to conceive and test solutions. Are we up to the challenge?

Special thanks to all those at the FHI, MIRI and Less Wrong who helped with this work, and those who voted on the name!
Innovation's low-hanging fruits: on the demand or supply sides?
by Stuart_Armstrong — 2014-02-25T14:58:16.582Z

Cross-posted at Practical Ethics.

This is an addendum to a previous post, which argued that we may be underestimating the impact of innovation because we have so much of it. I noted that we underestimated the innovative aspect of the CD because many other technologies partially overlapped with it, such as television, radio, cinema, ipod, walkman, landline phone, mobile phone, laptop, VCR and Tivo's. Without these overlapping technologies, we could see the CD's true potential and estimate it higher as an innovation. Many different technologies could substitute for each other.

But this argument brings out a salient point: if so many innovations overlap or potentially overlap, then there must be many more innovations that purposes for innovations. Tyler Cowen made the interesting point that the internet isn't as innovative as the flushing toilet (or indeed the television). He certainly has a point here: imagine society without toilets or youtube, which would be most tolerable (or most survivable)?

But the flush toilet can only be invented once. We might have access to talking super toilets with multi-coloured fountains - but all the bells and whistles are less useful that the original flushing toilet aspect. That's because flush toilets responded effectively to a real human need: how to dispose of human waste in urban areas. Once that problem is solved, further innovation is mainly wasted.

This suggests that while we may indeed be plucking the innovation low-hanging fruits, it might not be because we lack a supply of innovation, but because we're exhausting the easy demand for innovation. What current needs do we have that we're waiting for innovation to solve? What's problems are we facing that are as important as removing human waste from urban areas?

There seem to be very few. Maybe solving death and disease: and we can make a very strong case that medical innovation is indeed slowing. Poverty is another one; but it's not like we know of a specific technological innovation that would solve poverty, if only someone would develop it. We might want easy access to space, or effective alternative energies: but the way people and governments spend their money confirms that this is not a top priority for many. Even if we had teleporters, would future Tyler Cowens be writing that they're not as innovative as the car - and would they be correct, in that a t...
The innovation tree, overshadowed in the innovation forest
by Stuart_Armstrong — 2014-02-25T14:11:30.234Z

Cross-posted at Practical Ethics.

Many have pronounced that the era of innovation dead, peace be to its soul. From Tyler Cowen's decree that we've picked all the low hanging fruit of innovation, through Robert Gordon's idea that further innovation growth is threatened by "six headwinds", to Gary Karparov's and Peter Thiel's theory that risk aversion has stifled innovation, there is no lack of predictions about the end of discovery.

I don't propose to address the issue with something as practical and useful as actual data. Instead, staying true to my philosophical environment, I propose a thought experiment that hopefully may shed some light. The core idea is that we might be underestimating the impact of innovation because we have so much of it.

Imagine that technological innovation had for some reason stopped around the 1945 - with one exception: the CD and CD player/burner. Fast forwards a few decades, and visualise society. We can imagine a society completely dominated by the CD. We'd have all the usual uses for the CD - music, songs and similar - of course, but also much more.

Without mass television, the CD and radio would become the premier sources of entertainment for everyone. Newspapers would experiment with bundling CDs with their subscriptions, giving high-quality live sound recordings from major events. The walkman culture and all the transformations it brought would instead have been based around the portable CD player. Bosses at large companies would probably get into the habit of recording motivational messages and sending them to all employees. Syndicated columnists would record themselves on CDs to distribute to the fast-paced workers who didn't have time to be at a radio at a certain time, or who were snobish about high quality audio. Underground movements would stay in contact with smuggled CDs (as would various youth cultures) and families would stay in touch with mailed CD messages. Various corporations would experiments with ways of sending data via CD, maybe by connecting them up to some sort of typing machine. CD-geek movements would learn to interpret the data just by listening, without needing a translation machine. Coded messages would be sent on CDs, and vast amounts of government secrets and mundane data would be stored on it. Many other ideas would no doubt spring up to use this ubiquitous technology - CD contracts, anyone?

Any ...
Single player extensive-form games as a model of UDT
by cousin_it — 2014-02-25T10:43:12.746Z

This post was inspired by Benja's SUDT post. I'm going to describe another simplified model of UDT which is equivalent to Benja's proposal, and is based on standard game theory concepts as described in this Wikipedia article.

First let's define what is a "single player extensive-form game with chance moves and imperfect information":

A "single player extensive-form game" is a tree of nodes. Each leaf node is a utility value. A play of the game starts at the root and ends at some leaf node.
Some non-leaf nodes are "chance nodes", with probabilities assigned to branches going out of that node. All other non-leaf nodes are "decision nodes", where the player can choose which branch to take. (Thanks to badger for helping me fix an error in this part!)
"Imperfect information" means the decision nodes are grouped into "information sets". The player doesn't know which node they're currently at, only which information set it belongs to.
"Imperfect recall" is a special case of imperfect information, where knowing the current information set doesn't even allow the player to figure out which information sets were previously visited, like in the Absent-Minded Driver problem.
We will assume that the player can use "behavioral strategies", where the player can make a random choice at each node independently, rather than "mixed strategies", which randomize over the set of pure strategies for the entire game. See Piccione and Rubinstein's paper for more on this difference. (Thanks to Coscott for pointing out that assumption!)
The behavioral strategy with the highest expected utility will be taken as the solution of the game.

Now let's try using that to solve some UDT problems:

Absent-Minded Driver is the simplest case, since it's already discussed in the literature as a game of the above form. It's strange that not everyone agrees that the best strategy is indeed the best, but let's skip that and move on.

Psy-Kosh's non-anthropic problem is more tricky, because it has multiple players. We will model it as a single-player game anyway, putting the decision nodes of the different players in sequence and grouping them together into information sets in the natural way. The resulting game tree is complicated, but the solution is the same ...
[LINK] Sean Carrol's reflections on his debate with WL Craig on "God and Cosmology"
by Shmi — 2014-02-25T00:56:34.368Z

I previously mentioned this debate a month ago and predicted that Sean Carroll is unlikely to do very well. The debate happened last Friday and Sean posted his post-debate reflections on his popular blog (the full video will be posted soon). Some excerpts:

I think it went well, although I can easily think of several ways I could have done better. On the substance, my major points were that the demand for “causes” and “explanations” is completely inappropriate for modern fundamental physics/cosmology, and that theism is not taken seriously in professional cosmological circles because it is hopelessly ill-defined (no matter what happens in the universe, you can argue that God would have wanted it that way). He defended two of his favorite arguments, the “cosmological argument” and the fine-tuning argument; no real surprises there. In terms of style, from my perspective things got a bit frustrating, because the following pattern repeated multiple times: Craig would make an argument, I would reply, and Craig would just repeat the original argument.

The cosmological argument has two premises: (1) If the universe had a beginning, it has a transcendent cause; and (2) The universe had a beginning. [...] My attitude toward the above two premises is that (2) is completely uncertain, while the “obvious” one (1) is flat-out false. Or not even false, as I put it, because the notion of a “cause” isn’t part of an appropriate vocabulary to use for discussing fundamental physics. [Emphasis mine]

The Aristotelian analysis of causes is outdated when it comes to modern fundamental physics; what matters is whether you can find a formal mathematical model that accounts for the data.

Sean goes over a couple of mistakes he thinks he made in the debate, basically being blindsided by WLC bringing up obscure papers and misinterpreting them to suit his argument.

Sean's reflections are very detailed and worth reading, though I found them hard to summarize. It looks like WLC did his homework better than SC, but it's hard to tell whether it mattered until the video is made public and various interested parties gave their feedback. Another couple of quotes, with my emphasis:

For my closing statement, I couldn’t think of many responses to Craig’s closing statement that wouldn’t have simply be ...
Interest in a Christchurch (New Zealand) Meetup
by free_rip — 2014-02-24T22:20:24.948Z

I'm considering starting a Christchurch LessWrong Meetup and would like to get a measure of interest in the area. Including me, there are already three people interested, so you're sure to meet someone new! Please comment if you'd be at all interested.

I'd also like to find out if Sunday afternoon/evening is a good time for you. Considering Chch is pretty small, I'd like to find a time everyone can make it.

Location would likely be James Hight Library at Canterbury University (there are bookable private discussion rooms, nearby food places, and anyone can access it).

Topics up for grabs too, leave a suggestion if you'd like.
A medium for more rational discussion
by Adam Zerner — 2014-02-24T17:20:49.248Z

It would be cool if online discussions allowed you to 1) declare your claims, 2) declare how your claims depend on each other (ie. make a dependency tree), 3) discuss the claims, and 4) update the status of the claim by saying whether or not you agree with it, and using something like the text shorthand for uncertainty to say how confident you are in your agreement/disagreement.

I think that mapping out these things visually would allow for more productive conversation. And it would also allow newcomers to the discussion to quickly and easily get up to date, rather than having to sift through tons of comments. On this note, there should also probably be something like an answer wiki for each claim to summarize the arguments and say what the consensus is.

I get the feeling that it should be flexible though. That probably means that it should be accompanied by the normal commenting system. Sometimes you don't actually know what your claims are, but need to "talk it out" in order to figure out what they are. Sometimes you don't really know how they depend on each other. And sometimes you have something tangential to say (on that note, there should probably be an area for tangential comments, or at least a way to flag them as tangential).

As far who would be interested in this, obviously this Less Wrong community would be interested, and I think that there are definitely some other online communities that would (Hacker News, some subreddits...).

Also, this may be speculating, but I would hope that it would develop a reputation for the most effective way to have a productive discussion. So much so that people would start saying, "go outline your argument on [name]". Maybe there'd even be pressure for politicians to do this. If so, then I think this could put pressure on society to be more rational.

What do you guys think?

EDIT: If anyone is actually interested in building this, you definitely have my permission (don't worry about "stealing the idea"). I want to build it, but 1) I don't think I'm a good enough programmer yet, and 2) I'm busy with my startup.

EDIT: Another idea: if you think that a statement commits an established fallacy, then you should be able to flag it (like this). And if enough other people agree, then the statement is underlined or highlighted or something. The advantage to this is that it makes the discussion less "bulky". A simple vers...
How to teach to magical thinkers?
by polymathwannabe — 2014-02-24T13:43:20.443Z

I'm afraid I haven't properly designed the Muggles Studies course I introduced at my local Harry Potter fan club. Last Sunday we finally had our second class (after wasted months of insistence and delays), and I introduced some very basic descriptions of common biases, while of course emphasizing the need to detect them in ourselves before trying to detect them in other people. At some point, which I didn't completely notice, the discussion changed from an explanation of the attribution bias into a series of multicultural examples in favor of moral relativity. I honestly don't know how that happened, but as more and more attendants voiced their comments, I started to fear someone would irreversibly damage the lessons I was trying to teach. They basically stopped short of calling the scientific method a cultural construct, at which point I'm sure I would have snapped. I don't know what to make of this. Some part of me tries to encourage me and make me put more effort into showing these people the need for more reductionism in their worldview, but another part of me just wants to give them up as hopeless postmodernists. What should I do?
What are some related communities online?
by casebash — 2014-02-24T13:04:51.261Z

I think that it might be useful to create a list of related communities online that people might want to check out. Suggestions much appreciated.

Directly Related to rationalism

Skeptics Stack Exchange - useful for confirming factual claims you are skeptical of. Requires specific, answerable questions

Straight dope - similar to Skeptics, but less strict on the questions accepted

Change my view - place to get your view challenged

Cognitive Science - you are expected to do your reading first, but useful if your want to lean about what the research says

Philosophy Reddit

Rational Wiki: I only just learned about the existence of this site, so I don't know how good it is yet

Secondary:

Stacks: StackOverflow, Computer Science StackExchange, Theoretical computer science, MathOverflow, Math StackExchange, Cross Validated, Physics Stackexchange

Ask Science
LINK: In favor of niceness, community, and civilisation
by Solvent — 2014-02-24T04:13:10.620Z

Scott, known on LessWrong as Yvain, recently wrote a post complaining about an inaccurate rape statistic.

Arthur Chu, who is notable for winning money on Jeopardy recently, argued against Scott's stance that we should be honest in arguments in a comment thread on Jeff Kaufman's Facebook profile, which can be read here.

Scott just responded here, with a number of points relevant to the topic of rationalist communities.

I am interested in what LW thinks of this.

Obviously, at some point being polite in our arguments is silly. I'd be interested in people's opinions of how dire the real world consequences have to be before it's worthwhile debating dishonestly.
SUDT: A toy decision theory for updateless anthropics
by Benya — 2014-02-23T23:50:38.330Z

The best approach I know for thinking about anthropic problems is Wei Dai's Updateless Decision Theory (UDT). We aren't yet able to solve all problems that we'd like to—for example, when it comes to game theory, the only games we have any idea how to solve are very symmetric ones—but for many anthropic problems, UDT gives the obviously correct solution. However, UDT is somewhat underspecified, and cousin_it's concrete models of UDT based on formal logic are rather heavyweight if all you want is to figure out the solution to a simple anthropic problem.

In this post, I introduce a toy decision theory, Simple Updateless Decision Theory or SUDT, which is most definitely not a replacement for UDT but makes it easy to formally model and solve the kind of anthropic problems that we usually apply UDT to. (And, of course, it gives the same solutions as UDT.) I'll illustrate this with a few examples.

This post is a bit boring, because all it does is to take a bit of math that we already implicitly use all the time when we apply updateless reasoning to anthropic problems, and spells it out in excruciating detail. If you're already well-versed in that sort of thing, you're not going to learn much from this post. The reason I'm posting it anyway is that there are things I want to say about updateless anthropics, with a bit of simple math here and there, and while the math may be intuitive, the best thing I can point to in terms of details are the posts on UDT, which contain lots of irrelevant complications. So the main purpose of this post is to save people from having to reverse-engineer the simple math of SUDT from the more complex / less well-specified math of UDT.

(I'll also argue that Psy-Kosh's non-anthropic problem is a type of counterfactual mugging, I'll use the concept of l-zombies to explain why UDT's response to this problem is correct, and I'll explain why this argument still works if there aren't any l-zombies.)

*

I'll introduce SUDT by way of a first example: the counterfactual mugging. In my preferred version, Omega appears to you and tells you that it has thrown a very biased coin, which had only a 1/1000 chance of landing heads; however, in this case, the coin has in fact fallen heads, which is why Omega is talking to you. It asks you to choose between two options, (H) and (T). If you choose (H), Omega will create a Friendly AI; if you choose (T...
Beware garblejargon
by KnaveOfAllTrades — 2014-02-23T22:35:28.134Z

Brienne on Facebook:

"There seems to be a whole class of words that have some sort of anti-reasoning field. Everything's going fine, and then one of these wibbly wobbly distortion devices pops up and wrecks your whole field for a decade or three.

Properties of garblejargon:

(1) Early on, you gain status by using it.
(2) Later, you lose status by not using it.
(3) It reliably causes the user to confuse the map with the territory.
(4) It feels really satisfying for most users.
(5) When people attempt to explain the same thing without using the garblejargon, they either become overtly incoherent, or give accounts that directly contradict most similar attempts by others.

Other examples of garblejargon: emergence, truth-maker, universals (maybe)."

(Slightly edited for Less Wrong.)

I'd note that 'accounts that directly contradict most similar attempts by others' might often agree with the original attempts in obvious literal denotation but differ significantly in subtle connotation or the cognitive processes they cue, such that the difference in sense amounts to a huge difference in the ways of thinking promoted by the attempt to explain vs. by the original usage.

What are other examples of garblejargon?
In Favor of Niceness, Community, and Civilization
by Scott Alexander — 2014-02-23T22:24:18.000Z

[Content warning: Discussion of social justice, discussion of violence, spoilers for Jacqueline Carey books.]

[Edit 10/25: This post was inspired by a debate with a friend of a friend on Facebook who has since become somewhat famous. I’ve renamed him here to “Andrew Cord” to protect his identity.]

I.

Andrew Cord criticizes me for my bold and controversial suggestion that maybe people should try to tell slightly fewer blatant hurtful lies:

I just find it kind of darkly amusing and sad that the “rationalist community” loves “rationality is winning” so much as a tagline and yet are clearly not winning. And then complain about losing rather than changing their tactics to match those of people who are winning.

Which is probably because if you *really* want to be the kind of person who wins you have to actually care about winning something, which means you have to have politics, which means you have to embrace “politics the mindkiller” and “politics is war and arguments are soldiers”, and Scott would clearly rather spend the rest of his life losing than do this.

That post [the one debunking false rape statistics] is exactly my problem with Scott. He seems to honestly think that it’s a worthwhile use of his time, energy and mental effort to download evil people’s evil worldviews into his mind and try to analytically debate them with statistics and cost-benefit analyses.

He gets *mad* at people whom he detachedly intellectually agrees with but who are willing to back up their beliefs with war and fire rather than pussyfooting around with debate-team nonsense.

It honestly makes me kind of sick. It is exactly the kind of thing that “social justice” activists like me *intend* to attack and “trigger” when we use “triggery” catchphrases about the mewling pusillanimity of privileged white allies.

In other words, if a fight is important to you, fight nasty. If that means lying, lie. If that means insults, insult. If that means silencing people, silence.

It always makes me happy when my ideological opponents come out and say eloquently and openly what I’ve always secretly suspected them of believing.

My natural instinct is to give some of the reasons why I think Andrew is wrong, starting with the history of the “noble lie” concept and moving on to some examples of why it didn’t work very well, and why it might not be expected not to work so ...