trentmkelly/LessWrong-43k huggingface dataset:
Principled vs. Pragmatic Morality
by Chris_Leong — 2018-05-29T04:31:04.620Z

One reason why conversations about morality are often unproductive is that there are two seperate conversations that often get mixed together. The first is principled morality: How ought perfect, incorruptible beings with no false beliefs act, further assuming the absence of any side-effects? The second is practical morality: How ought flawed, corruptible beings with false beliefs act, accounting for the possibility of side-effects?

I believe that both of these questions are ultimately valuable: the pragmatic because it tells us how to ultimately act; the principled because it is instructive. This post will focus on defending the value of discussing principled morality as people on Less Wrong generally don't object to discussing pragmatic morality. However, I grant that there might exist some very strong deontologists who would require the opposite message. More commonly though, I suspect that people who ignore pragmatic morality have just accidentally skipped a step and forgotten to account for practical considerations.

I already explained in my post on the Direct Application Fallacy that even if a hypothetical situation never occurs in real life, it can still provide opportunities for learning, be tied back to the real through further argument or make us aware of the limitations of our model. Eliezer expresses a similar sentiment in a recent Facebook post, where he identifies a particular kind of argument as stemming from, "one of those awful durable forms of misunderstanding where Msr. Toolbox doesn't see what you could possibly be telling somebody to do with a "good" or "ideal" algorithm besides executing that algorithm."

Further, separating the discussion of principled and pragmatic issues is important for the same reason that local validity is important. If you have to clearly specify your stance on principled morality and then your stance on how these considerations play out practically, you have less scope to fudge that if you refuse to seperate them.

Objections

Undoubtedly, contextualisers would object that insisting on this kind of discussion is often harmful. For example, someone might oppose torture because they believe that giving a government that much power is too risky. They might want to refuse to discuss the principled question where this consideration is excluded because they might be worried that someone...
Bounded Rationality: Two Cultures
by Ben Pace — 2018-05-29T03:41:49.527Z
Link: https://www.tandfonline.com/doi/pdf/10.1080/1350178X.2014.965908?needAccess=true

Have not read this in any detail, but looked like interesting food for thought. Here's the first section of the paper.

Introduction and Outline

Bounded rationality does not speak with one voice. This is not only because bounded rationality is researched in various fields such as economics, psychology, engineering and management. Even within a single field such as economics, there are clear differences. For example, Selten (2001) rejects the optimization of a utility function as an expression of bounded rationality, contrary to the standard approach of behavioral economics as in bargaining games by Fehr and Schmidt (1999). There are multiple views of bounded rationality, as many authors including Rubinstein (1998) have pointed out.

The first contribution of this article is to analyze the formal modeling used to describe people’s bounded rationality. At the risk of oversimplifying, I distinguish between two cultures, which I call ‘idealistic’ and ‘pragmatic’. At a first approximation, the idealistic culture pursues a minimum departure from the neoclassical-economics framework of unbounded rationality, which assumes the ideals of omniscience and optimization of a utility function and adds factors such as inequity aversion or probability weighting to the utility function. On the other hand, the pragmatic culture holds that people sometimes ignore information and use simple rules of thumb in order to achieve satisfactory outcomes. A detailed discussion of the differences in modeling between the two cultures is provided in Section 2. The reality of the cultures and their differences is demonstrated by examples drawn from the literatures on risky choice and bargaining games. Note that it does not make sense to try to perfectly map specific researchers or programs of research to one or the other culture; for example, Amos Tversky worked on both cultures, with prospect theory being an idealistic model and elimination by aspects being a pragmatic model.

Although the distinction between the idealistic and pragmatic cultures of bounded rationality can be criticized, as all binary distinctions can be, it provides food for thought and new insights. I aim at emulating Breiman’s (2001) analysis of two cultures in statistics. Breiman argued that there exist two cultures that lead to two very different kinds of statistical theory and practice, proof-based and data-driven. A

...
Meta-Honesty: Firming Up Honesty Around Its Edge-Cases
by Eliezer Yudkowsky — 2018-05-29T00:59:22.084Z

(Cross-posted from Facebook.)

0: Tl;dr.

A problem with the obvious-seeming "wizard's code of honesty" aka "never say things that are false" is that it draws on high verbal intelligence and unusually permissive social embeddings. I.e., you can't always say "Fine" to "How are you?" This has always made me feel very uncomfortable about the privilege implicit in recommending that anyone else be more honest.
Genuinely consistent Glomarization (i.e., consistently saying "I cannot confirm or deny" whether or not there's anything to conceal) does not work in principle because there are too many counterfactual selves who might want to conceal something.
Glomarization also doesn't work in practice if the Nazis show up at your door asking if you have fugitive Jews in your attic.
If you would lie to Nazis about fugitive Jews, then absolute truthsaying can't be the whole story, which makes "never say things that are false" feel to me like a shaky foundation in that it is literally false, and something less shaky would be nice.
Robin Hanson's "automatic norms" problem suggests different people might have very different ideas about what constitutes a good person's normal honesty, without realizing that they have very different ideas. Perceived violations of an honesty norm can blow up and cause interpersonal conflict. It seems to me that this is something that doesn't always work well when people leave it alone.

A rule which seems to me more "normal" than the wizard's literal-truth rule, more like a version of standard human honesty reinforced around the edges, would be as follows:

"Don't lie when a normal highly honest person wouldn't, and furthermore, be honest when somebody asks you which hypothetical circumstances would cause you to lie or mislead—absolutely honest, if they ask under this code. However, questions about meta-honesty should be careful not to probe object-level information."

I've been tentatively calling this "meta-honesty", but better terminology is solicited.

1: Glomarization can't practically cover many cases.

Suppose that last night I helped hide a fugitive marijuana seller from the Feds. You ask me what I was doing last night, and I, preferring not to emit false statements, reply, "I can't confirm or deny what I was doing last night."

We now have two major problems here:

Even on an ordinary day, if you casually ask me what I was doing last night, I theoret

...
*Another* Double Crux Framework
by Raemon — 2018-05-28T23:30:32.891Z

Epistemic Status: Untested hypothesis. Looking to operationalize better.

Followup to:

A Concrete Multi Step Variant of Doublecrux I Have Used (deluks917)
Musings on Double Crux (me)

A problem with resolving deep disagreements is that there's a lot of finnicky moving parts to the dispute. It's easy to bring up 3 examples to illustrate a point, then you get fixated on one of the examples, and then bring up some adjaecent issues to those examples, and then hours (or months) later you've managed to go in circles three times and still not really resolve anything.

I just had an idea, which may or may not work but I'd like to try (and I have something like 3-5 outstanding deep disagreements with 3-4 different people so it seemed like an opportune time to flesh it out), which is to manage the deep disagreement via a google doc, divided into sections:

1. Common beliefs – What do we actually agree on?

2. Empirical Checks / Bets – What things you can go look at in the real world that'd resolve common cruxes?

3. Common cruxes – What do we agree that we disagree on? (that we have operationalized, such that we are confident we understand each other, and can think productively about how to empirically resolve them)

4. Passing Each Other's ITT – Each person attempts to summarize each other's position as best they can, in succinct bullet point form. (Contains a subsection for each participant)

5. Amorphous exploratory discussion / ranting – Two large meandering sections that attempt to suss out the parts of the disagreement where you aren't actually sure what each other are saying. Still try to keep points to individual bullets.

This can be done in tandem with an in person discussion. But the main idea here is have an explicit goal that bullet points slowly get refactored into clearer versions of themselves, and move up the hierarchy. Amorphous rants should turn into ITT-passing arguments should turn into clearly identified common cruxes should turn into empiricism should turn into shared beliefs. (And, hopefully, a lot of disagreements dissolve under scrutiny such that empiricism isn't even necessary)

This would give you a clear sense of whether you're accomplishing anything – if the bottom sections keep getting bigger and don't eventually move to the upper sections, you're probably talking in circles, or making some kind of ...
Using the LessWrong API to query for events
by habryka — 2018-05-28T22:41:52.649Z

Taymon Beal was pinging us about some help with using the LessWrong API to construct a sidebar for SSC with upcoming meetups. I figured I would write up my response publicly so that other people might be able to learn from it.

As I understand the thing correctly (though Taymon can correct me in the comments) he wanted a way to query the API to get a list of all upcoming SSC meetups. This is the the GraphQL query that I would construct to make that happen (note to myself to fix the code embedding editor component):

{PostsList(terms: {view: "nearbyEvents", filter: "SSC", lat: 0, lng: 0}) {

_id

createdAt

title

mongoLocation

location

}}

You can also play around with this interactively on our GraphiQL page.

The "nearbyEvents" view right now requires you to specify a location, and returns all upcoming meetups (to any point in the future) ordered by their distance from that point. You can find the full definition of that view here.
Shared interests vs. collective interests
by Said Achmiz — 2018-05-28T22:06:50.911Z
Link: https://blog.obormot.net/Shared-interests-vs-collective-interests

Suppose that I, a college student, found a student organization—a chapter of Students Against a Democratic Society, perhaps. At the first meeting of SADS, we get to talking, and discover, to everyone’s delight, that all ten of us are fans of Star Trek.

This is a shared interest.

Shared interests

A shared interest—in the way I am using the term—is nothing more than what it sounds like: an interest (in the broad sense of the word) that happens, for whatever reason, to be shared among all members of a group.

The distinction I want to draw is between a shared interest (of a group) and a collective interest (of a group). The former is a superset of the latter; all collective interests are, by definition, shared interests; but not all shared interests are collective interests.

Collective interests

What is a collective interest?

Well, suppose that I found another student organization (extracurricular activities look great on a résumé). This one is a Star Trek fan club. At the first meeting of Campus Trekkies United, we discover, to no one’s surprise, that all fifteen of us are fans of Star Trek.

… well, of course we’re all fans of Star Trek. That’s why we’re in the fan club in the first place! Anyone who’s not a fan, has no reason to join the fan club. And so: Star Trek fandom is a collective interest of Campus Trekkies United.

A collective interest is an interest that is shared by every member of a group in virtue of being a member of that group. Anyone who does not share that interest, will not be a group member.[1] And thus, by modus tollens: anyone who is a member of the group, will share that interest. It is guaranteed that every member of the group will share that interest.

Details & implications

Several important consequences follow from this.

Preservation of interests

Unlike a collective interest, a shared interest is not at all guaranteed to stay shared among all group members. Nothing stops someone from joining the Students Against a Democratic Society, who does not like Star Trek. At that point, Star Trek fandom ceases to be a shared interest of SADS. (Which may lead to some awkward consequences if, for instance, we had decided to start wearing colorful jumpsuits to our political rallies.)

Infiltration

I said earlier that “[i]t is guaranteed that every member of the group will share [a collective] interest”. But is this really true? Well, it’s true if the c...
The Alignment Newsletter #8: 05/28/18
by Rohin Shah — 2018-05-28T16:00:47.422Z

Highlights

Solving the Rubik's Cube Without Human Knowledge

(Stephen McAleer, Forest Agostinelli, Alexander Shmakov et al): This paper proposes Autodidactic Iteration (ADI), which is a technique that can be combined with the techniques in AlphaGo and expert iteration to solve problems with only one goal state, such as the Rubik's cube. MCTS with value and policy networks will not suffice, because when starting from a randomly scrambled cube, MCTS will never find a path to the goal state, and so there will never be any reward signal. (Whereas with Go, even if you play randomly the game will end relatively quickly, giving you some reward signal.) To get around this, they start from the goal state and generate states that are near the goal state. This gives them a training dataset of states for which they know (a good approximation to) the value and the best action, which they can use to train a value and policy network. They then use this with MCTS to solve the full problem, as in AlphaGo.

My opinion: This general idea has been proposed in robotics as well, in

Reverse Curriculum Generation for Reinforcement Learning

, where there is a single goal state. However, in this setting we have the added benefit of perfect inverse dynamics, that is, for any action a that moves us from state s to s', we can find the inverse action a' that moves us from state s' to s. This allows the authors to start from the goal state, generate nearby states, and automatically know the value of those states (or at least a very good approximation to it).

Hindsight Experience Replay

also tackles similar issues -- I'd be interested to see if it could solve the Rubik's cube. Overall, the problem of sparse rewards is very difficult, and it seems like we now have another solution in the case where we have a single goal state and perfect (or perhaps just sufficiently good?) inverse dynamics.

Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior

(Siddharth Reddy et al): Inverse reinforcement learning algorithms typically assume that the demonstrations come from an expert who is approximately optimal. However, this is often not the case, at least when the experts are fallible humans. This paper considers the case where the expert has an incorrect model of the dynamics (transition function) of the environment, and proposes learning t...
Understanding is translation
by cousin_it — 2018-05-28T13:56:11.903Z

Does this feel familiar: "I thought I understood thing X, but then I learned something new and realized that I'd never really understood X?"

For example, consider a loop in some programming language:

var i = 0;
while (i < n) {
i = i + 1;
}

If you're a programmer, you probably understand it just fine. How it works, in what order the lines are executed, how the variable changes over time... But have you ever noticed that the simplest way to compile such a loop to machine code involves two jump instructions - one conditional and one unconditional? (Try doing that with only one jump, it won't work.)

Now you might feel that your "understanding" of loops has become slightly closer to "proper understanding".

Or not!

An alternative view is that understanding is translation. It's a two-place word. Your understanding of loops, in the sense of translating them to execution histories, was perfectly fine. But your understanding of loops, in the sense of translating them to machine code, was slightly lacking.

When you see that pattern once, you notice it everywhere. A middle-schooler can understand numbers, in the sense of translating them to amounts of apples and such, but doesn't immediately translate the expression "x > 5" to a half-open ray on the number line. A self-taught singer can translate from heard notes to sung notes, but can't translate either to notes on a staff; a self-taught guitarist is missing a different subset of those skills. A bilingual person can translate a Japanese sentence with the word "integral" to English, without knowing what integral means. You can be good at translating other people's facial expressions to emotional states, but lousy at translating them to pencil sketches; your friend is the opposite; which of you "understands" human faces better? There's no answer, or many answers. Don't ask whether someone understands X. Instead, ask if they can translate X <-> Y.

That has implications for teaching. If you walk into a classroom intending to make students "understand" X, you'll fail at teaching. (I'm speaking from experience here.) But if you find some Y, already understood by the students, that can be translated to X - and drill them repeatedly on both directions of translation - then they will begin to "understand" X.
GreaterWrong—even more new features & enhancements
by Said Achmiz — 2018-05-28T05:08:31.236Z

(Previous posts: [1], [2], [3])

GreaterWrong.com has recently added a number of new features and UI enhancements, especially to the mobile version of the site:

Private messaging

You can now send and receive private messages.

To send a PM, click on a user’s name, then click “Send private message” (at the top-right):

You can view PMs that you received, send replies, and view the entire back-and-forth conversation, by going to your user page, and clicking on “Conversations”:

Received PMs also show up in your Inbox. (Any new items in your inbox make the envelope icon next to your name in the nav bar turn red.)

Sort user’s posts/comments by karma rating

You can now view a user’s top-rated posts/comments, by going to their user page and switching the sort order to “Top”:

Individual comment threads

You can now browse an individual comment thread on a separate page. The “anchor” icon at the top of a comment is the permalink to that comment thread’s page:

Classic Less Wrong theme

There is now a new theme (bringing the total to eight themes to choose from): “Classic Less Wrong”. This theme replicates, as much as possible, the styling of the old Less Wrong website (a.k.a. “Less Wrong 1.0”).

Mobile theme selection

Users on mobile devices can now switch between the available themes, just like users on desktops.

New editing UI for smartphones

For smartphone users, a new and greatly improved post/comment editor UI is live.

Faster loading speed

Changes to the server code have brought substantial speed improvements, making pages load much faster.

Minor enhancements

There are many minor UI enhancements, including:

comment-collapse buttons on every comment thread

upvote/downvote buttons at the bottoms of comments (as well as at the top)

… and other minor fixes and improvements.
Ineffective entrepreneurship: post-mortem of Hippo, the happiness app that never quite was
by Elizabeth — 2018-05-27T23:03:37.333Z
Link: http://effective-altruism.com/ea/1oq/ineffective_entrepreneurship_postmortem_of_hippo/

" I spent two and half years trying to start a startup I thought might do lots of good. It failed. I explain what happened, how it went wrong and try to set out some relevant lessons for others. Main lesson: be prepared for the fact you might find the experience mostly stressful and have nothing particularly useful come of it."
The simple picture on AI safety
by Alex Flint — 2018-05-27T19:43:27.025Z

At every company I've ever worked at, I've had to distill whatever problem I'm working on to something incredibly simple. Only then have I been able to make real progress. In my current role, at an autonomous driving company, I'm working in the context of a rapidly growing group that is attempting to solve at least 15 different large engineering problems, each of which is divided into many different teams with rapidly shifting priorities and understandings of the problem. The mission of my team is to make end-to-end regression testing rock solid so that the whole company can deploy code updates to cars without killing anyone. But that wasn't the mission from the start: at the start it was a collection of people with a mandate to work on a bunch of different infrastructural and productivity issues. As we delved into mishmash after mishmash of complicated existing technical pieces, the problem of fixing it all became ever more abstract. We built long lists of ideas for fixing pieces, wrote extravagant proposals, and drew up vast and complicated architecture diagrams. It all made sense, but none of it moved us closer to solving anything.

At some point, we distilled the problem down to a core that actually resonated with us and others in the company. It was not some pithy marketing-language mission statement; it was not a sentence at all -- we expressed it a little differently every time. It represented an actual comprehension of the core of the problem. We got two kinds of reactions: to folks who thought the problem was supposed to be complicated, out distillation sounded childishly naive. They told us the problem was much more complicated. We told them it was not. To folks who really wanted to solve the problem with their own hands, the distillation was energizing. They said that yes, this is the kind of problem that can be solved.

I have never encountered a real problem without a simple distillation at its core. Some problems have complex solutions. Some problems are difficult or impossible to solve. But I have never encountered a problem that did not have a deeply simple core to it, and I have always found that enunciating this core is helpful.

So here is my distillation of the AI safety problem.

First, there is the technical engineering problem of building a safe superintelligence. This is a technical engineering problem, and nothing more. Yes, it is very diffi...