trentmkelly/LessWrong-43k huggingface dataset:
Harry Potter and the Methods of Rationality discussion thread, March 2015, chapter 114 + chapter 115
by Gondolinian — 2015-03-03T18:02:44.923Z

This is a new thread to discuss Eliezer Yudkowsky’s Harry Potter and the Methods of Rationality and anything related to it. This thread is intended for discussing chapter 114, and also, as a special case due to the exceptionally close posting times, chapter 115.

There is a site dedicated to the story at hpmor.com, which is now the place to go to find the authors notes and all sorts of other goodies. AdeleneDawner has kept an archive of Author’s Notes. (This goes up to the notes for chapter 76, and is now not updating. The authors notes from chapter 77 onwards are on hpmor.com.)

Spoiler Warning: this thread is full of spoilers. With few exceptions, spoilers for MOR and canon are fair game to post, without warning or rot13. More specifically:

You do not need to rot13 anything about HP:MoR or the original Harry Potter series unless you are posting insider information from Eliezer Yudkowsky which is not supposed to be publicly available (which includes public statements by Eliezer that have been retracted).

If there is evidence for X in MOR and/or canon then it’s fine to post about X without rot13, even if you also have heard privately from Eliezer that X is true. But you should not post that “Eliezer said X is true” unless you use rot13.
[POLL] LessWrong group on YourMorals.org (2015)
by gwern — 2015-03-03T03:08:32.748Z

In 2011, InquilineKea posted a Discussion topic on YourMorals.org, a psychology research website which provides scores of psychology scales/inventories/surveys/tests to the general public to gather large samples. Niftily, YourMorals lets users sign up for particular groups, and then when you take tests, you can see your own results alongside group averages of liberals/conservatives/libertarians & $GROUP. A lot of time has passed and I think most LWers don't know about it, so I'm reposting so people can use it.

The regular research has had interesting results like showing a distinct pattern of cognitive traits and values associated with libertarian politics, but there's no reason one can't use it for investigating LWers in more detail; for example, going through the results, "we can see that many of us consider purity/respect to be far less morally significant than most", and we collectively seem to have Conscientiousness issues. (I also drew on it recently for a gay marriage comment.) If there were more data, it might be interesting to look at the results and see where LWers diverge the most from libertarians (the mainstream group we seem most psychologically similar to), but unfortunately for a lot of the tests, there's too little to bother with (LW n<10). Maybe more people could take it.

You can sign up using http://www.yourmorals.org/setgraphgroup.php?grp=623d5410f705f6a1f92c83565a3cfffc

All quizzes: http://www.yourmorals.org/all_morality_values_quizzes.php

Big 5: http://www.yourmorals.org/bigfive_process.php

(You can see some of my results at http://www.gwern.net/Links#profile )
Superintelligence 25: Components list for acquiring values
by KatjaGrace — 2015-03-03T02:01:11.071Z

This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, and an index of posts so far see the announcement post. For the schedule of future topics, see MIRI's reading guide.

Welcome. This week we discuss the twenty-fifth section in the reading guide: Components list for acquiring values.

This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. Some of my own thoughts and questions for discussion are in the comments.

There is no need to proceed in order through this post, or to look at everything. Feel free to jump straight to the discussion. Where applicable and I remember, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).

Reading: “Component list” and “Getting close enough” from Chapter 13

Summary

Potentially important choices to make before building an AI (p222)

What goals does it have?

What decision theory does it use?

How do its beliefs evolve? In particular, what priors and anthropic principles does it use? (epistemology)

Will its plans be subject to human review? (ratification)

Incentive wrapping: beyond the main pro-social goals given to an AI, add some extra value for those who helped bring about the AI, as an incentive (p222-3)

Perhaps we should indirectly specify decision theory and epistemology, like we have suggested doing with goals, rather than trying to resolve these issues now. (p224-5)

An AI with a poor epistemology may still be very instrumentally smart, but for instance be incapable of believing the universe could be infinite (p225)

We should probably attend to avoiding catastrophe rather than maximizing value (p227) [i.e. this use of our attention is value maximizing..]

If an AI has roughly the right values, decision theory, and epistemology maybe it will correct itself anyway and do what we want in the long run (p227)

Another view

Paul Christiano argues (today) that decision theory doesn't need to be sorted out before creating human-level AI. Here's a key bit, but you might need to look at the rest of the post to understand his idea well:

Really, I’d like to leave these questions up to an AI. That is, whatever work Iwould do in order to answer these questions, an A

...
Announcement: The Sequences eBook will be released in mid-March
by Rob Bensinger — 2015-03-03T01:58:45.893Z

The Sequences are being released as an eBook, titled Rationality: From AI to Zombies, on March 12.

We went with the name "Rationality: From AI to Zombies" (based on shminux's suggestion) to make it clearer to people — who might otherwise be expecting a self-help book, or an academic text — that the style and contents of the Sequences are rather unusual. We want to filter for readers who have a wide-ranging interest in (/ tolerance for) weird intellectual topics. Alternative options tended to obscure what the book is about, or obscure its breadth / eclecticism.

The book's contents

Around 340 of Eliezer's essays from 2009 and earlier will be included, collected into twenty-six sections ("sequences"), compiled into six books:

Map and Territory: sequences on the Bayesian conceptions of rationality, belief, evidence, and explanation.

How to Actually Change Your Mind: sequences on confirmation bias and motivated reasoning.

The Machine in the Ghost: sequences on optimization processes, cognition, and concepts.

Mere Reality: sequences on science and the physical world.

Mere Goodness: sequences on human values.

Becoming Stronger: sequences on self-improvement and group rationality.

The six books will be released as a single sprawling eBook, making it easy to hop back and forth between different parts of the book. The whole book will be about 1,800 pages long. However, we'll also be releasing the same content as a series of six print books (and as six audio books) at a future date.

The Sequences have been tidied up in a number of small ways, but the content is mostly unchanged. The largest change is to how the content is organized. Some important Overcoming Bias and Less Wrong posts that were never officially sorted into sequences have now been added — 58 additions in all, forming four entirely new sequences (and also supplementing some existing sequences). Other posts have been removed — 105 in total. The following old sequences will be the most heavily affected:

Map and Territory and Mysterious Answers to Mysterious Questions are being merged, expanded, and reassembled into a new set of introductory sequences, with more focus placed on cognitive biases. The name 'Map and Territory' will be re-applied to this entire collection of sequences, constituting the first book.

Qua

...
What Scarcity Is and Isn't
by  — 2015-03-03T01:41:51.477Z

Sequence summary: This is a series of 18 articles on the most fundamental concepts of economics: scarcity, opportunity cost, marginalism, and self-interest. These are the atoms, molecules, cells, the core things you need to have a grip on to move on with the science, and, if all goes well, we will move on, but first it is absolutely vital to get a strong grasp of the fundamentals. Though these are basic concepts, they are not easy to understand. If you want evidence of this, open a newspaper....

Previous: Imagining Scarcity

A familiar example of something scarce is ordering in a restaurant. There are so many enticing options, but you can only order one. You want more than what you can choose to have. Whatever option you choose, you could have chosen another one, and that which you did not choose is now beyond your reach. Even if you come back and order that other choice the next day, it's not the same day, you're not quite the same you. All this means that your options are scarce.

Or say you're packing to go hiking. You have only so much space in your pack. That means if you want to bring the flashlight, you won't have room for those extra snack bars, not if you also want to fit in the spare batteries for the camera. You have to pick one and make a tradeoff - that is, your space is scarce.

(Maybe you could bring a bigger pack. But that would be more unwieldy, and with all the stuff in it, it will be heavier. So there's your tradeoff - more stuff, or a more difficult hike?)

And don't forget, you need a tent. It rains pretty hard round these parts. Do you want to spend more money on the super high-tech tent with the poly-something supercalifragilistic anti-water technology? Or do you want the cheaper one, even though there's a chance you might get wet? Again, you have to make a choice, trading off one good for another because your money is scarce.

If only you had more money! If only you had more space! If only you could carry more weight! If only more restaurants would offer a tasting menu so you could sample everything and not torture you with the eternal mystery of what all these fascinating things actually are!

All these things are scarce.

Now that we have some examples, we can pin down a more precise definition of what makes something scarce. But first, I'm going to state very explicitly what does not make something scarce. Space is not scarce just bec...
Proposal: Modeling goal stability in machine learning
by orthonormal — 2015-03-03T01:31:36.000Z

Summary: We might learn some interesting things if we can construct a model of goal stability, wireheading, and corrigibility in a present-day machine learning algorithm. I outline one way that we could potentially do this, and ask if you'd like to help!

So far, MIRI has focused on doing crisp proofs about simple mathematical models (with unbounded computation or other unfeasible features) rather than building and studying messy heuristic analogues of the topics of interest. There are some excellent reasons for this focus, but there are also some topics which we could safely and usefully study in both ways. Here's one such example.

The intuitive phenomenon we'd eventually like to study is corrigibility. An agent with sufficient self-reflective capacity will generally want to prevent changes to its goal system (see e.g. section 2.2 of this paper), even if these changes come from its developers, and this gets to be a problem as soon as an AI is capable of resisting, manipulating, or deceiving its developers; but perhaps some special kinds of goal systems would be better suited for allowing such correction.

There are several stages to building up a useful model of corrigibility, some of them almost certainly done already in the literature and others not.

First, we could represent goal stability and accidental wireheading, perhaps as follows:

We start with a machine learning algorithm trying to optimize a criterion (e.g. a cost function) over its training data, and being tested on the application of its model to testing data.

Next, the system should be able to modify the criterion itself, at least partially. (For example, some of its outputs are parameters of the next iteration of the criterion, while others are parameters of the next iteration of the model.)

I'd expect that some initially complex criteria collapse to trivial criteria after iterated modifications (i.e. wireheading) while others remain stable (preferably even in the face of some threshold of random variation). There's probably interesting work to be done even here, with (e.g.) constraints and regularization parameters that aren't affected by the output.

But this type of system doesn't have any self-reflection; it can't foresee that changing the criterion can have big effects on making it easier to optimize that criterion, it can only locally drift toward ways to make its score higher. So at the nex

...
Rationality Quotes Thread March 2015
by Vaniver — 2015-03-02T23:38:48.068Z

Another month, another rationality quotes thread. The rules are:

Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.

No more than 5 quotes per person per monthly thread, please.

Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
Announcing the Complice Less Wrong Study Hall
by MalcolmOcean — 2015-03-02T23:37:24.563Z

(If you're familiar with the backstory of the LWSH, you can skip to paragraph 5. If you just want the link to the chat, click here: LWSH on Complice)

The Less Wrong Study Hall was created as a tinychat room in March 2013, following Mqrius and ShannonFriedman's desire to create a virtual context for productivity. In retrospect, I think it's hilarious that a bunch of the comments ended up being a discussion of whether LW had the numbers to get a room that consistently had someone in it. The funny part is that they were based around the assumption that people would spend about 1h/day in it.

Once it was created, it was so effective that people started spending their entire day doing pomodoros (with 32minsWork+8minsBreak) in the LWSH and now often even stay logged in while doing chores away from their computers, just for cadence of focus and the sense of company. So there's almost always someone there, and often 5-10 people.

A week in, a call was put out for volunteers to program a replacement for the much-maligned tinychat. As it turns out though, video chat is a hard problem.

So nearly 2 years later, people are still using the tinychat.

But a few weeks ago, I discovered that you can embed the tinychat applet into an arbitrary page. I immediately set out to integrate LWSH into Complice, the productivity app I've been building for over a year, which counts many rationalists among its alpha & beta users.

The focal point of Complice is its today page, which consists of a list of everything you're planning to accomplish that day, colorized by goal. Plus a pomodoro timer. My habit for a long time has been to have this open next to LWSH. So what I basically did was integrate these two pages. On the left, you have a list of your own tasks. On the right, a list of other users in the room, with whatever task they're doing next. Then below all of that, the chatroom.

(Something important to note: I'm not planning to point existing Complice users, who may not be LWers, at the LW Study Hall. Any Complice user can create their own coworking room by going to complice.co/createroom)

With this integration, I've solved many of the core problems that people wanted addressed for the study hall:

an actual ding sound beyond people typing in the chat

synchronized pomodoro time visibility

pomos that automatically start, so breaks don't run over

Intentions — what am I working on this

...
March 2015 Media Thread
by ArisKatsaris — 2015-03-02T18:51:38.224Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Open thread, Mar. 2 - Mar. 8, 2015
by MrMind — 2015-03-02T08:19:47.940Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Meetup : HPMOR/Other media discussion
by rocurley — 2015-03-02T05:35:33.341Z

Discussion article for the meetup : HPMOR/Other media discussion

WHEN:
02 March 2015 06:15:00PM (-0800)

WHERE:
1061 Market St #4, San Francisco, CA 94103

This is the first meetup of the month and is therefore the Schelling meetup: If you want to come once a month, now's the time.
We'll be meeting to talk about methods (in particular, the recent puzzle).
Probably not everyone's read methods, so we'll be talking about other media as well.
As always for discussion topic meetups, this is just the schelling topic, and you're free to talk about whatever you want.

Discussion article for the meetup : HPMOR/Other media discussion
Meetup : App Academy Discussion
by Danny_Hintze — 2015-03-02T04:52:02.937Z

Discussion article for the meetup : App Academy Discussion

WHEN:
08 March 2015 02:00:00PM (-0700)

WHERE:
300 East Orange Mall, Tempe, AZ 85281

Jay is going to fill us in on his recent App Academy experiences.

Text or call 602 501 9420. We usually meet up in one of the meeting rooms in the sub-basement (yes, there is a basement to the basement) of Hayden library, so it can be hard to find the first time you try and meet up with us.

Discussion article for the meetup : App Academy Discussion
Imagining Scarcity
by  — 2015-03-02T01:43:17.688Z

Sequence summary: This is a series of 18 articles on the most fundamental concepts of economics: scarcity, opportunity cost, marginalism, and self-interest. These are the atoms, molecules, cells, the core things you need to have a grip on to move on with the science, and, if all goes well, we will move on, but first it is absolutely vital to get a strong grasp of the fundamentals. Though these are basic concepts, they are not easy to understand. If you want evidence of this, open a newspaper....

Thank goodness this wasn't a restaurant where you had to order only one thing and you never found out what all the other things on the menu tasted like. Harry hated that, it was like a torture chamber for anyone with a spark of curiosity: Find out about only one of the mysteries on this list, ha ha ha!

-HPMOR

A simple way to understand scarcity is to imagine you're trying to fit all your sand into a hole, but the hole is too small for all the sand to fit into.

It is, of course, possible to make the hole deeper or wider. It's also possible to compress the sand. However, either task can only be accomplished with the help of a mysterious element called "technology." The thing is, economists don't know what this element looks like or how to find it. Sometimes we look at all the people putting sand into holes and notice that the hole is bigger or the sand is more compressed, and we conclude "technology" must have happened. But it's not something we can predict or count on. So how are you going to get all of your sand into this hole?

You're not. Look, I don't know what's so special about this sand, and I don't know why you have to get it into this hole, but I know not all of it's going in. And that means if you want any of it to go in, you must leave some of it out.

That's scarcity: you must give up something to get anything.

It won't fit. Don't try to force it - it won't fit. And that means you're going to have to make a choice.

"Hold on," you say. "I don't really care which sand goes into the hole and which stays out here."

"Okay, okay, but this is economic sand. It's representative."

"Of what?"

"Take a closer look."

You give the sky (being uncertain of where this voice is coming from) a skeptical look, but you grudgingly crouch and inspect the sand (which stretches for mil...
Meetup : Effective Altruism Meetup Vienna
by AnnaLeptikon — 2015-03-01T21:33:32.735Z

Discussion article for the meetup : Effective Altruism Meetup Vienna

WHEN:
19 March 2015 07:00:00PM (+0100)

WHERE:
Vienna

Meetup.com event for further information (like location)
http://www.meetup.com/Effective-Altruism-Vienna/events/220706342

Discussion article for the meetup : Effective Altruism Meetup Vienna
Meetup : Rationality Meetup Vienna
by AnnaLeptikon — 2015-03-01T21:29:33.203Z

Discussion article for the meetup : Rationality Meetup Vienna

WHEN:
18 April 2015 02:00:00PM (+0100)

WHERE:
Kaisermühlenstraße 24/2, 1120 Vienna

directions:
http://web.student.tuwien.ac.at/~e0326238/rationality_meetup/directions.html

FB-Event for more details and discussions:
https://www.facebook.com/events/959818560709439
(You need to become member of "Rationality Vienna" to see it)

Discussion article for the meetup : Rationality Meetup Vienna
Meetup : Rationality Meetup Vienna
by AnnaLeptikon — 2015-03-01T21:27:59.567Z

Discussion article for the meetup : Rationality Meetup Vienna

WHEN:
14 March 2015 03:00:00PM (+0100)

WHERE:
Kaisermühlenstraße 24, 1120 Wien

directions:
http://web.student.tuwien.ac.at/~e0326238/rationality_meetup/directions.html

FB-Event for more details and discussions:
https://www.facebook.com/events/959818560709439
(You need to become member of "Rationality Vienna" to see it)

Discussion article for the meetup : Rationality Meetup Vienna
Meetup : Sydney Rationality Dojo - Skill Training Part 2
by luminosity — 2015-03-01T21:08:03.401Z

Discussion article for the meetup : Sydney Rationality Dojo - Skill Training Part 2

WHEN:
12 April 2015 04:00:00PM (+1000)

WHERE:
272 Wyndham St, Alexandria

Since we didn't cover everything we could have, this month we'll be following up with a further look at skill training. This has been shifted from the first Sunday to the second due to Easter.

Discussion article for the meetup : Sydney Rationality Dojo - Skill Training Part 2