trentmkelly/LessWrong-43k huggingface dataset:
Is the orthogonality thesis at odds with moral realism?
by ChrisHallquist — 2013-11-05T20:47:52.979Z

Continuing my quest to untangle people's confusions about Eliezer's metaethics... I've started to wonder if maybe some people have the intuition that the orthogonality thesis is at odds with moral realism.

I personally have a very hard time seeing why anyone would think that, perhaps in part because of my experience in philosophy of religion. Theistic apologists would love to be able to say, "moral realism, therefore a sufficiently intelligent being would also be good." It would help patch some obvious holes in their arguments and help them respond to things like Stephen Law's Evil God Challenge. But they mostly don't even try to argue that, for whatever reason.

You did see philosophers claiming things like that back in the bad old days before Kant, which raises the question of what's changed. I suspect the reason is fairly mundane, though: before Kant (roughly), it was not only dangerous to be an atheist, it was dangerous to question that the existence of God could be proven through reason (because it would get you suspected of being an atheist). It was even dangerous to advocated philosophical views that might possibly undermine the standard arguments for the existence of God. That guaranteed that philosophers could used whatever half-baked premises they wanted in constructing arguments for the existence of God, and have little fear of being contradicted.

Besides, even if you think an all-knowing would also necessarily be perfectly good, it still seems perfectly possible to have an otherwise all-knowing being with a horrible blind spot regarding morality.

On the other hand, in the comments of a post on the orthogonality thesis, Stuart Armstrong mentions that:

I've read the various papers [by people who reject the orthogonality thesis], and they all orbit around an implicit and often unstated moral realism. I've also debated philosophers on this, and the same issue rears its head - I can counter their arguments, but their opinions don't shift. There is an implicit moral realism that does not make any sense to me, and the more I analyse it, the less sense it makes, and the less convincing it becomes. Every time a philosopher has encouraged me to read a particular work, it's made me find their moral realism less likely, because the arguments are always weak.

This is not super-enlightening, partly because Stuart is talking about people whose views he admits he d...
[Link] "A Long-run Perspective on Strategic Cause Selection and Philanthropy" by Nick Beckstead and Carl Shulman
by Pablo — 2013-11-05T18:27:04.851Z

A philanthropist who will remain anonymous recently asked us about what we would do if we didn’t face financial constraints. We gave a detailed answer that we thought we might as well share with others, who may also find our perspective interesting. We gave the answer largely in hope of creating some interest in our way of thinking about philanthropy and some of the causes that we find interesting for further investigation, and because we thought the answer would be fruitful for conversation.

Link
Meetup : Moscow, Now 2 Sigma More Awesome
by Slava Matyukhin — 2013-11-05T10:33:46.570Z

Discussion article for the meetup : Moscow, Now 2 Sigma More Awesome

WHEN:
10 November 2013 04:00:00PM (+0400)

WHERE:
Russia, Moscow, ulitsa L'va Tolstogo 16

We have lots of various content prepared this time, 2 standard deviations more than usual.

We're planning to have (not necessarily in this order):

a report from a participant of our ongoing willpower group;

a report about "Feeling Good" by David Burns;

an exercise on discovering your terminal values;

a section on how to make conversations more productive by learning to recognize several failure modes of arguing.

There might be a calibration session if we have time for it.

There might also be a cake, but it can turn out to be a lie.

As usual, we're going to have our ongoing Prediction Market, Scavenger Hunt, positive reinforcements (now with cashew as an option, not just chocolate!) and pizza.

We intend to broadcast this meetup on ustream or justin.tv, but considering the amount of chaos and the fact we never did it before, it's unlikely to turn out well. So join us in person if you can.

If you are going for the first time:

We gather in the Yandex office. Look the second revolving door with the sign “Яндекс”, here is the photo of the entrance. You need to pass the first entrance and the bicycle parking on your way. Here is an additional guide on how to get there: link.

You can fill this one minute form (in Russian) to share your contact information.

We start at 16:00 and sometimes finish at night.

Discussion article for the meetup : Moscow, Now 2 Sigma More Awesome
No Universally Compelling Arguments in Math or Science
by ChrisHallquist — 2013-11-05T03:32:42.920Z

Last week, I started a thread on the widespread sentiment that people don't understand the metaethics sequence. One of the things that surprised me most in the thread was this exchange:

Commenter: "I happen to (mostly) agree that there aren't universally compelling arguments, but I still wish there were. The metaethics sequence failed to talk me out of valuing this."

Me: "But you realize that Eliezer is arguing that there aren't universally compelling arguments in any domain, including mathematics or science? So if that doesn't threaten the objectivity of mathematics or science, why should that threaten the objectivity of morality?"

Commenter: "Waah? Of course there are universally compelling arguments in math and science."

Now, I realize this is just one commenter. But the most-upvoted comment in the thread also perceived "no universally compelling arguments" as a major source of confusion, suggesting that it was perceived as conflicting with morality not being arbitrary. And today, someone mentioned having "no universally compelling arguments" cited at them as a decisive refutation of moral realism.

After the exchange quoted above, I went back and read the original No Universally Compelling Arguments post, and realized that while it had been obvious to me when I read it that Eliezer meant it to apply to everything, math and science included, it was rather short on concrete examples, perhaps in violation of Eliezer's own advice. The concrete examples can be found in the sequences, though... just not in that particular post.

First, I recommend reading The Design Space of Minds-In-General if you haven't already. TLDR; the space of minds in general ginormous and includes some downright weird minds. The space of human minds is a teeny tiny dot in the larger space (in case this isn't clear, the diagram in that post isn't remotely drawn to scale). Now with that out of the way...

There are minds in the space of minds-in-general that do not recognize modus ponens.

Modus ponens is the rule of inference that says that if you have a statement of the form "If A then B", and also have "A", then you can derive "B". It's a fundamental part of logic. But there are possible mind that reject it. A brilliant illustration of this point can be found in Lewis Carroll's dialog "What the Tortoise Said to Achilles" (for those not in the know, Carroll was a mat...
2013 Census/Survey: call for changes and additions
by Scott Alexander — 2013-11-05T03:10:53.855Z

I have finally gotten the survey to a point where I'm pretty happy with it. I have no big changes I want to make this year. But as is the tradition, please take a week to discuss what minor changes you want to the survey (within the limits of what Google Docs and finite time can do) and I will try to comply. In particular, we can continue the tradition that any question you request can be added to the Extra Credit section unless it's illegal or horribly offensive.

You can find last year's survey results here and you can find the very preliminary version of this year's survey (so far exactly the same as last year's) here.

EDIT: I don't particularly like the IQ test or the Big Five test used last year. If you have any better replacements for either, tell me and I'll put them in.

EDIT2: CFAR, you added seven questions last year. Let me know what you want to do with those this year. Keep them? Remove them? Replace them?
New vs. Business-as-Usual Future
by katydee — 2013-11-05T02:13:00.679Z

What, in a broad sense, does the future look like? We don't know, and while many have historically made predictions, the track record for such predictions is less than impressive. I have noted that there appear to be two main types of view about the future-- the "new future" and the "business-as-usual future." In order to simplify this discussion, let's restrict it only to the coming century-- the period between 2013 and 2113.

The "new future" is, generally speaking, the idea that the coming century is going to be very different from the present; the "business-as-usual future" is, generally speaking, the idea that the coming century is going to be very similar to the present.

Here are some characteristics of the new future:

Some large-scale event occurs that alters human experience forever-- an intelligence explosion leading to a technological singularity, existential risks leading to human suppression or extinction, global climate change on a massive scale, etc.

Society changes a lot, and in fundamental ways that are difficult to understand. Daily life is vastly altered.

If future history even exists after the dramatic change, it views the coming century as being a critical moment where everything became vastly different, on par with or exceeding the significance of the development of agriculture.

Here are some characteristics of the business-as-usual future:

The intelligence explosion doesn't happen. AI continues to advance in much the same way that it has for the last several decades. More human-capable tasks become automated, but in slow and predictable ways. Intelligence amplification doesn't happen or doesn't yield generally useful results.

The world doesn't end. Global warming ends up being just another doomsday scare. Perhaps a lot of people die in the Third World, but the rest of the world adapts and keeps going much like it has for the last ever. Yellowstone doesn't explode. No asteroids hit the earth. There isn't a nuclear war.

Society doesn't change very much except in superficial ways. Daily life is more or less the same.

Wars might happen. Nations might collapse. But wars have been happening and nations have been collapsing for thousands of years. By and large, the coming century is viewed by future history as not particularly unlike those that came before.

Reference class forecasting seems to indicate that the business-as-usual future is qu...
Meetup : Rescheduled November Meetup
by Nova_Division — 2013-11-05T01:56:49.065Z

Discussion article for the meetup : Rescheduled November Meetup

WHEN:
24 November 2013 07:00:00PM (-0500)

WHERE:
491 Lindbergh Place NE Apt 618 Atlanta, GA 30324

Update, 11/24/13.

New venue! This is a private residence without cats. The address, again is:

Post Lindbergh Apartments
491 Lindbergh Place NE Apt 618
Atlanta, GA 30324

Gate code: 607

Once you enter the gate, make a left and continue till you reach a swimming pool. Apartment 618 is behind the pool. You can park wherever there's space.

Please call Katie at 678-416-0248 with questions.

We've had a last minute hitch in our plans for the meetup, courtesy of the flu! Unfortunately, one of the roommates at our host's house has come down with the flu, and in the interest of protecting our health from contagions, the meetup has been changed to SUNDAY, November 24th, at 7 (SEVEN) PM. Venue TBA soon.

Come join us! We'll be doing our normal eclectic mix of self-improvement brainstorming, educational mini-presentations, structured discussion, unstructured discussion, and social fun and games times! Check out ATLesswrong's facebook group, if you haven't already: https://www.facebook.com/groups/Atlanta.Lesswrong/ where you can connect with Atlanta Lesswrongers, suggest a topics for discussion at this meetup, and join our book club or study group!

Discussion article for the meetup : Rescheduled November Meetup
Meetup : Durham NC/Triangle Area Meetup: Productive Other-Optimization
by curiousepic — 2013-11-04T23:33:57.741Z

Discussion article for the meetup : Durham NC/Triangle Area Meetup: Productive Other-Optimization

WHEN:
07 November 2013 07:00:00PM (-0500)

WHERE:
2411 N Roxboro St, Durham NC 27704

Bring your problems, worries, and knowledge gaps, great and small, and we will apply our communal knowledge to solving them - and try to recognize when we're committing the Typical Mind Fallacy.

It will likely help to write down some problems beforehand! Feel free to post them here as well, to get our thoughts going. If anyone is particularly eager to go first, they have priority - otherwise we will choose randomly, and if all goes well, continue working through others as part of future meetups. The format will be:

Announce problem, get clarifications

5 minutes of quiet solution-searching

20-30 minutes of discussion

Repeat

After 3 cycles, 5 minutes more of discussion of each

11/17, 7:00 PM, at the House with the Red Door, 2411 N Roxboro St, parking on Ellerbee. Hangouts, libations, and/or games to subsequently ensue in-situ or at Fullsteam!

Discussion article for the meetup : Durham NC/Triangle Area Meetup: Productive Other-Optimization
Meetup : Meetup Bolder CO
by yakurbe0112 — 2013-11-04T17:05:17.789Z

Discussion article for the meetup : Meetup Bolder CO

WHEN:
07 November 2013 07:00:00PM (-0700)

WHERE:
Old Chicago 1102 Pearl St, Boulder, CO

We didn't meet up last week for the discucion so we will do it this week. The topic is what we want from our meetup group.

Discussion article for the meetup : Meetup Bolder CO
From Philosophy to Math to Engineering
by lukeprog — 2013-11-04T15:43:55.704Z

Cross-posted from the MIRI blog.

For centuries, philosophers wondered how we could learn what causes what. Some argued it was impossible, or possible only via experiment. Others kept hacking away at the problem, clarifying ideas like counterfactual and probability and correlation by making them more precise and coherent.

Then, in the 1990s, a breakthrough: Judea Pearl and others showed that, in principle, we can sometimes infer causal relations from data even without experiment, via the mathematical machinery of probabilistic graphical models.

Next, engineers used this mathematical insight to write software that can, in seconds, infer causal relations from a data set of observations.

Across the centuries, researchers had toiled away, pushing our understanding of causality from philosophy to math to engineering.

And so it is with Friendly AI research. Current progress on each sub-problem of Friendly AI lies somewhere on a spectrum from philosophy to math to engineering.

We began with some fuzzy philosophical ideas of what we want from a Friendly AI (FAI). We want it to be benevolent and powerful enough to eliminate suffering, protect us from natural catastrophes, help us explore the universe, and otherwise make life awesome. We want FAI to allow for moral progress, rather than immediately reshape the galaxy according to whatever our current values happen to be. We want FAI to remain beneficent even as it rewrites its core algorithms to become smarter and smarter. And so on.

Small pieces of this philosophical puzzle have been broken off and turned into math, e.g. Pearlian causal analysis and Solomonoff induction. Pearl's math has since been used to produce causal inference software that can be run on today's computers, whereas engineers have thus far succeeded in implementing (tractable approximations of) Solomonoff induction only for very limited applications.

Toy versions of two pieces of the "stable self-modification" problem were transformed into math problems in de Blanc (2011) and Yudkowsky & Herreshoff (2013), though this was done to enable further insight via formal analysis, not to assert that these small pieces of the philosophical problem had been solved to the level of math.

Thanks to Patrick LaVictoire and other MIRI workshop participants,1 Douglas Hofstadter's FAI-relevant philosophical idea of "superrationality"

...
Meetup : Urbana-Champaign: Thinking Fast and Slow Pages 70-140 Discussion Meetup
by Mestroyer — 2013-11-04T10:39:19.077Z

Discussion article for the meetup : Urbana-Champaign: Thinking Fast and Slow Pages 70-140 Discussion Meetup

WHEN:
10 November 2013 02:00:00PM (-0600)

WHERE:
40.109545,-88.227318

Coordinates are: 40.109545,-88.227318 Meetup will be held in the Courtyard Cafe in the Illini Union, on the ground floor, at 2PM.

Discussion will be on pages 70-140 of Thinking Fast and Slow.

We will probably also discuss the raven paradox after having a week to think about it and read outside opinions, because we were unsatisfied with our response to it last time.

Cross-posted on the mailing list.

Discussion article for the meetup : Urbana-Champaign: Thinking Fast and Slow Pages 70-140 Discussion Meetup
Meetup : Brussels monthly meetup: memory!
by Roxolan — 2013-11-04T07:05:56.800Z

Discussion article for the meetup : Brussels monthly meetup: memory!

WHEN:
09 November 2013 01:00:00PM (+0100)

WHERE:
Rue des Alexiens 55 1000 Bruxelles

This month's topic: memory. Read some interesting research? Have a niggling worry that your entire life started last week and any memory prior to that is a fabrication? Come share! I'm experimenting with the memory palace/method of loci; if I'm successful I might try and do the party trick.

As on every second Saturday of the month, we will meet at 1 pm at La Fleur en papier doré, close to the Brussels Central station. The meeting will be in English to facilitate both French and Dutch speaking members.

If you are coming for the first time, please consider filling out this one minute form, to share your contact information: https://docs.google.com/forms/d/1qSvI1NWkFSsfIJhUMORb_Wd8fdJTVPhdw49grDQwRTI/viewform

The Brussels meetups use a Google Group: https://groups.google.com/forum/#!forum/lesswrong-brussels

Discussion article for the meetup : Brussels monthly meetup: memory!
The Inefficiency of Theoretical Discovery
by lukeprog — 2013-11-03T21:26:52.468Z

Previously: Why Neglect Big Topics.

Why was there no serious philosophical discussion of normative uncertainty until 1989, given that all the necessary ideas and tools were present at the time of Jeremy Bentham?

Why did no professional philosopher analyze I.J. Good’s important “intelligence explosion” thesis (from 19591) until 2010?

Why was reflectively consistent probabilistic metamathematics not described until 2013, given that the ideas it builds on go back at least to the 1940s?

Why did it take until 2003 for professional philosophers to begin updating causal decision theory for the age of causal Bayes nets, and until 2013 to formulate a reliabilist metatheory of rationality?

By analogy to financial market efficiency, I like to say that “theoretical discovery is fairly inefficient.” That is: there are often large, unnecessary delays in theoretical discovery.

This shouldn’t surprise us. For one thing, there aren’t necessarily large personal rewards for making theoretical progress. But it does mean that those who do care about certain kinds of theoretical progress shouldn’t necessarily think that progress will be hard. There is often low-hanging fruit to be plucked by investigators who know where to look.

Where should we look for low-hanging fruit? I’d guess that theoretical progress may be relatively easy where:

Progress has no obvious, immediately profitable applications.

Relatively few quality-adjusted researcher hours have been devoted to the problem.

New tools or theoretical advances open up promising new angles of attack.

Progress is only valuable to those with unusual views.

These guesses make sense of the abundant low-hanging fruit in much of MIRI’s theoretical research, with the glaring exception of decision theory. Our September decision theory workshop revealed plenty of low-hanging fruit, but why should that be? Decision theory is widely applied in multi-agent systems, and in philosophy it’s clear that visible progress in decision theory is one way to “make a name” for oneself and advance one’s career. Tons of quality-adjusted researcher hours have been devoted to the problem. Yes, new theoretical advances (e.g. causal Bayes nets and program equilibrium) open up promising new angles of attack, but they don’t seem necessary to much of the low-hanging fruit disco...
Meetup : London social meetup, 10/11/13
by philh — 2013-11-03T17:27:18.049Z

Discussion article for the meetup : London social meetup, 10/11/13

WHEN:
10 November 2013 02:00:00PM (+0000)

WHERE:
Shakespeare's Head, Africa House, 64-68 Kingsway, London WC2B 6BG, UK

LW London is having another social meetup, and you should come! There's no fixed topic of discussion, so we talk about whatever strikes our fancy, which can range from fanfiction to cryonics by way of logic puzzles and utilitarian population ethics.

We'll meet in the Shakespeare's Head near Holborn tube station, at 2pm. There'll be a sign identifying us; also, feel free to contact me through email (philip.hazelden@gmail.com) or phone (07792009646) if you have questions or difficulty finding us.

Also check out our google group.

Discussion article for the meetup : London social meetup, 10/11/13
A Workflow with Spaced Repetition
by Emile — 2013-11-03T15:58:41.117Z

This is a detailed description of my reading and learning workflow. You may find ideas to adopt, or maybe you can tell me what I could be doing differently!

Overview

I've been using Spaced Repetition on and Off for the past few years, and have built a solid Anki habit this last three months, to the point where now I wonder how I could read books without entering the important points into Anki.

I recommend getting a habit of using Spaced Repetition, it's a small habit that doesn't require too much willpower (it can feel like a game, if done right!), and is useful in the long term.

Daily routine: transit

I have a dozen or so Anki decks. Some I consider “valuable” (Algorithms, Driving Code, Git commands), some less so (Paris Metro, Hiragana and Katakana, Vim commands, …). I also carry around a book, notebook and four-color pen.

On any downtime (waiting for transit, waiting in line in a store, standing in crowded transit…), I’ll review my decks, starting with those with the most due cards.

On some days I may not finish all the decks, but that’s no big deal; with an hour and a half of transit per day, I’ll get to them eventually.

If I can sit for a bit of time, and don’t have too many outstanding cards, I’ll usually read a book (or work on stuff in my notebook if I have some stuff that needs brainstorming).

Reading books

If I’m reading fiction, I’m relaxing, I don’t need to try to remember anything :)

If I’m reading non-fiction, I’ll usually have an index card as a bookmark and place to take notes - things to look up, summaries and rephrasings, diagrams, page numbers of parts to come back to, and of course things to enter in Anki (though I’ll sometimes just directly enter them in my phone).

I’ll reread my notes when I finished the book or a big chapter, or when I come back to the book after a long time, and eventually enter them in Anki (usually with Anki's web interface, which is quicker than typing on a phone).

Reading online material

I have a bunch of Google Docs where I take notes on various topics (why Google Docs? I can search them, share them if needed, work with them from various places). If I’m reading something I want to remember, I’ll usually have a corresponding google doc open in another window (so I can see both at the same time - hunting through ...