trentmkelly/LessWrong-43k huggingface dataset:
The Alignment Newsletter #9: 06/04/18
by Rohin Shah — 2018-06-04T16:00:42.161Z

Highlights

Playing hard exploration games by watching YouTube

(Yusuf Aytar, Tobias Pfaff et al): There are many YouTube videos demonstrating how to play levels of eg. Montezuma's Revenge. Can we use these demonstrations to solve the hard exploration tasks in Atari? One challenge is that the videos have slightly different visual properties (like color and resolution). They propose to learn a shared feature space by using an auxiliary loss where the network must predict the number of timesteps between two frames of a video, or to predict the delay between a video and audio clip from the same trajectory. Using this shared feature space, they can define a reward function that encourages the agent to take trajectories whose features match those of the demonstrations. In experiments they exceed human performance on Atari games with hard exploration problems.

My opinion: It seems to me that this is how we'll have to solve exploration in practice if we don't want to have a huge sample complexity, though I know other researchers are optimistic about solving exploration using curiosity or diversity. It's pretty exciting that they could use a source of data that was already present in the real world.

Technical AI alignment

Problems

The simple picture on AI safety

(alexflint): Argues that we should distill the problem of AI safety into a simple core. The author proposes it be distilled into two simple (but not easy) problems -- the technical engineering problem of how to build a safe superintelligence, and the coordination problem of how to prevent an unaligned superintelligence from being built first.

Iterated distillation and amplification

Amplification Discussion Notes

(William_S)

Learning human intent

Learning Safe Policies with Expert Guidance

(Jessie Huang et al): Expert demonstrations can be consistent with many possible reward functions. Instead of simply trying to mimic the demonstration, the authors consider all possible rewards that are consistent with the demonstration, and then maximize the worst reward, leading to safe behavior.

My opinion: This is very related to

Inverse Reward Design

, where instead of maxmin planning we use risk-averse planning, and instead of considering all rewards compatible with an expert demonstration we consider all reward functions that are probable based on which reward function the designer wrote down.

Handling groups o

...
Simplified Poker
by Zvi — 2018-06-04T15:50:00.299Z

This is intended as a three-part sequence. Part two will go over my strategy. Part three will reveal the results and discuss some implications.

In the same class in which we later played The Darwin Game, we played a less complex game called Simplified Poker. As in The Darwin Game, we were given the rules and asked to submit instructions for a computer program that would play the game, and the professor would then code our programs for us.

The rules of Simplified Poker are as follows:

Game is played with a 3-card deck, with the cards labeled 1, 2 and 3.

Each hand, the players alternate who goes first, each player antes one chip and is dealt one card.

The first player can bet one chip, or check.

If the first player bets, the second player can either call the one chip bet, or fold.

If the first player checks, the second player can either also check, or can bet. If the second player bets, the first player can either call the one chip bet, or fold.

There is at most one bet per hand, as neither player is allowed to raise.

If either player folds, the other wins the pot of 2 chips and takes back their 1 chip bet. Neither card is shown. If neither player folds – either both players check, or there is a bet and a call – then both cards are revealed and the player with the higher card takes all 4 chips.

In the class, all programs would play a round robin with all other programs, with 50 hands per match. Your goal is to maximize the average number of chips won over all rounds – note that how many opponents you beat does not matter, only the number of chips won.

The game is simple. A lot, but far from all, of your decisions are forced. There’s no weird trick, but optimal play still isn’t obvious. I’ll pause here to allow and encourage thinking about what strategy you’d submit.
Teaching Methodologies & Techniques
by ChaosHufflepuff — 2018-06-04T11:33:15.436Z

This is my first post so I'm not too sure of whether I am posting this in the appropriate format, or whether this post is appropriate or not - please let me know if it's not.

I am a first year university student who is intending to become a high school teacher after I finish my undergraduate studies. I am wondering if there are many other school teachers in this community and what particular methodologies you implement. I am seeking out discussions on teaching methodologies that people use, why they use them, and what is successful about them. In the lead-up to my career as a teacher, as of a month or so ago, I have begun compiling a document of methodologies and techniques I intend to use to maximise my effectiveness of a teacher.

A particular thing I would love to discuss is philosophies on teaching science. I strongly believe that it is *very* important for me to include a component on rationality in the curriculum, because too often students are taught "science" without first being taught how to think about science.
The lesswrong slack - an introduction to our regulars
by Elo — 2018-06-04T06:29:18.247Z

About us

The lesswrong slack has been active for 2-3 years now. It’s a garden in as much as I try to keep it peaceful.

A new member joined the slack recently. He asked: What are the moderation rules here?

I wrote the brief rules, but then I also was feeling particularly chatty so I also introduced several of our regular characters. Below is a log of that conversation.

There are few rules. I am in charge. Only ever banned three guys. This is a private community. It's rare that we have any problems. And if we do we talk about it.

We know each other quite well. These are all my friends, also practically my family. I can introduce the regulars.

We have about 40 active users from the pool of 577 or so that have signed up to try out if we suit.

We are generally quiet ~3000 messages a week, Spread over several channels. A feature of being quiet is that we all tend to read all the messages, no matter what channel they appear in. That keeps us close together and good friends!

Common channels are:

AI - Artificial intelligence
Art_and_media - sharing pictures, books, videos, music
Business_and_startups - talking about business ideas
Campfire - talking about our past and personal stories
Cognitive_enhancement - talking about drugs and other relevant enhancers
Dailychallenge - occasional critical life questions to everyone
Dieting_and_food - diet, health, cooking hedons
Enlightenment - talking about enlightenment states and related ideas
Fit - do you even lift? We do. We all do.
Goals_of_lesswrong - about lesswrong
Human_relationships - humans are hard.
Mental_health - day to day mental health check ins and following each other’s journeys
Open - anything
Parenting - yep.
Personal_finance - approaching financial independence and how to think about things
Philosophy - yep.
Political_talk - about politics, not about agendas.
Programming - prog.
Projects - people report and share what they are working on
Rational_hell - where all the good rats go to be silly
Rationalfic - fiction
Rationality - craft shaping methods and strategies or questions
Real_life_advice - ask the brains trust
Rss_lesswrong - rss feed
Rss_lesswrong_comments - rss feed
Rss_general - rss feed of our friends
Science_and_technology - yep.
Sexual_preferences - yep. Although we do a lot of talking about talking about sex.
Today_I_did_this - report mini victories
Vulnerability - talking about our present vulnerabilities. A place to report exp

...
Against accusing people of motte and bailey
by Kaj_Sotala — 2018-06-03T21:31:24.591Z

Motte-and-bailey is a pretty famous term in rationalist circles. To quote the canonical SSC post on the topic:

So the motte-and-bailey doctrine is when you make a bold, controversial statement. Then when somebody challenges you, you claim you were just making an obvious, uncontroversial statement, so you are clearly right and they are silly for challenging you. Then when the argument is over you go back to making the bold, controversial statement.

I think that this is a real thing that very definitely happens.

I also can't remember having been in a single discussion that would have been improved by someone using this concept, and several that were worsened by it.

What's wrong with pointing out a motte and bailey if you see one?

Well, consider this conversation, whose content is fictional, but whose structure is similar to several very frustrating conversations that I've had:

Person: LW rationality isn't worth looking at. It's involves basing all your thinking on explicit calculations, which never works.

Me: Actually, that's not the case! People in the rationalsphere have explicitly said that that you shouldn't do that, and that emotion and intuition are actually important inputs to your decision-making process as well. For instance, here's a link to Julia Galef on straw vulcans, and here's an article saying that you shouldn't always use probabilities, and...

Person: Right, but that's just the motte. What they actually recommend in practice is just doing an explicit calculation about everything.

I am now faced with an impossible situation. Given that this person has decided that all the explicit declarations of "use emotion and intuition too" are just the motte, there's practically nothing that I could do to change their mind. Any links to articles saying that will be dismissed as just being parts of the motte. I could try to challenge their claim by asking for links that would support their case - but no doubt that the person can dig up some links which on an uncharitable reading at least sound like the thing that the person is saying. And possibly the person's actual crux is that they've actually run into some people who did misinterpret LW rationality in that way, and now this person has decided that those people were actually representative of all LW rationality.

Now, in this particul...
Excessive EDA Effortposting
by abstractapplic — 2018-06-03T19:17:22.595Z

Introduction and motivation

Science! It’s important. It’s also difficult, largely because of how easily people fool themselves. I’ve seen plenty of people (my younger self included!) lose the plot as they attempt their first few investigations, letting subtle methodological problems render their entire chain of reasoning worthless.

So I’ve decided to pick a small, well-behaved dataset nobody’s had a thorough look at yet, and perform an egregiously thorough and paranoid exploration. My aim is both to showcase Good Science for aspiring Data Analysts, and to give others a chance to criticise my approach: it’s entirely possible that I’m still making some blunders, and if I am then I want to know.

Below is a summary of my methodology, with emphasis on the mistakes I managed to dodge. You can see the full R kernel here.

The dataset

The data was collected by renowned gamedev/educator/all-round-smart-person Nicky Case, in an attempt to decide which of six projects (‘Crowds’, ‘Learn/Teach’, ‘Win/Win’, ‘Nonviolence’, ‘Understand’, and ‘Mindfulness’) they should work on next. They described each possible project, and asked fans to rank them from 1-5 stars. The poll, with full descriptions of each project, is still up here, though the project names are indicative enough that you shouldn’t need it.

‘Crowds’ won, handily, and Case built it. At time of writing, they are between major projects; my pretext for performing this analysis is helping them decide which of the runners-up (‘Understand’ and ‘Learn/Teach’) to prioritise.

Exploration

I began by loading and cleaning the data. For most data analysis jobs, handling irregularities in a dataset can take up as much time as actually analysing it: I picked an unusually tidy dataset so I’d be able to skip to the parts where people make interesting mistakes.

The most I had to deal with was a few responses with missing values, caused by people not filling in every part of the poll. I considered handling this by imputation – deciding to interpret blanks as three-star ratings, say – but eventually decided to limit subjectivity and drop all incomplete responses.

[Normally, I’d look into missing values in more detail, but because I knew the causal mechanism and they were a small enough proportion of the dataset and I knew there aren’t going to be outliers (there’s no way a missing value could be hiding a rating of 100/5 stars) and these seeme...
Swimming Upstream: A Case Study in Instrumental Rationality
by TurnTrout — 2018-06-03T03:16:21.613Z

One data point for careful planning, the unapologetic pursuit of fulfillment, and success. Of particular interest to up-and-coming AI safety researchers, this post chronicles how I made a change in my PhD program to work more directly on AI safety, overcoming significant institutional pressure in the process.

It's hard to believe how much I've grown and grown up in these last few months, and how nearly every change was borne of deliberate application of the Sequences.

I left a relationship that wasn't right.
I met reality without flinching: the specter of an impossible, unfair challenge; the idea that everything and everyone I care about could actually be in serious trouble should no one act; the realization that people should do something [1], and that I am one of those people (are you?).
I attended a CFAR workshop and experienced incredible new ways of interfacing with myself and others. This granted me such superpowers as (in ascending order): permanent insecurity resolution, figuring out what I want from major parts of my life and finding a way to reap the benefits with minimal downside, and having awesome CFAR friends.
I ventured into the depths of my discomfort zone, returning with the bounty of a new love: a new career.
I followed that love, even at risk of my graduate career and tens of thousands of dollars of loans. Although the decision was calculated, you better believe it was still scary.

I didn't sacrifice my grades, work performance, physical health, or my social life to do this. I sacrificed something else.

CHAI For At Least Five Minutes

January-Trout had finished the Sequences and was curious about getting involved with AI safety. Not soon, of course - at the time, I had a narrative in which I had to labor and study for long years before becoming worthy. To be sure, I would never endorse such a narrative - Something to Protect, after all - but I had it.

I came across several openings, including a summer internship at Berkeley's Center for Human-Compatible AI. Unfortunately, the posting indicated that applicants should have a strong mathematical background (uh) and that a research proposal would be required (having come to terms with the problem mere weeks before, I had yet to read a single result in AI safety).

OK, I'm really skeptical that I can plausibly compete this year, but applying would be a valuable information-gath

...
Trajectory
by Logan Riggs — 2018-06-02T18:29:06.023Z

Position:

What's your goal? What do you want? What do you want different in this world? What should be different in this world?

Where are you now?

Velocity:

Where are you headed? Where are your actions pointing towards? Which actions are pushing towards your goal? Which actions are pulling away?

What's your trajectory?

Acceleration:

What will it take to move faster? What are you doing right now to move faster, to be more, to take on more in order to get what you want sooner, or even to get it at all?

What bounds are you pushing? What limits are you neglecting to push?

Jerk:

How fast are you improving? How often do you up your game? Are you pursuing growth more and more? Are you embracing the growing pains that come with it?

Have you even considered this question at all?

Since starting my 80 day productivity sprint, I've put a lot of thought into my trajectory, my path my actions are pointing towards and how fast I'm headed there. If I was to condense the above poetic-text, I would say:

Position: Your goal and current state

Velocity: Speed towards goal

Acceleration: Speeding up

Jerk: Speeding up more often

On Velocity:

Trajectory is the combination of all relevant velocities. There are actions that add to your velocity and those that take away the opportunity to choose a positive velocity. For me, reading textbooks, work, and connections add to my current pace. Pica such as junk food, fiction, and social events take away time & energy that could be spent elsewhere.

A common heuristic I asking myself is, "Is this action helping my trajectory?"

On Acceleration:

Acceleration is upping my trajectory to achieve my goal sooner by pushing my bounds at a certain rate. In order to do that, I need to have a better understanding of my bounds, what it feels like to push those limits, and to actually do it without burning out. I'll separate those bounds into quantity and quality bounds.

Quantity:

How can I gain more time & energy? One already stated is getting rid of pica in my life. Another is changing my environment to be more conducive to deep work such as no people, phone notifications, or visible tabs on my browser. The last one is saving money that I can trade for time.

Quality:

How can I more effectively use my time & energy? To quote Raemon quoting Critch:

There are increasing returns to deep, uninterrupted work. A half hour here and the

...
Why kids stop asking why
by Expipiplusone — 2018-06-02T17:03:51.859Z

I've recently written a post in my own blog, asking a question: unfortunately it's in Italian, so I'll try to summarize it in English here.

Kids are born scientists: they are curious, ask questions, are eager to seek an understanding on the world around them; but then they turn into adults: most of them stop asking questions, even to themselves, stop being permeable to new knowledge, and appear to have granitic certainties which no experiment could ever shatter. Those who later turn to become scientists, will have to learn again the virtue of humble curiosity.

I wonder why. Is it a natural phenomenon of human development, because you must be certain of your arguments if you don't want to perish in front of your opponents? Or is it the consequence of a wrong educational system, which trumps their curiosity saying "stop asking questions, learn what I tell you, unconditionally" and penalizes them when the do not brag parade enough what they have memorized?

I guess both are true, but my question is: to what extent is it a biological fact, and to what extent is it a cultural one? To what extent should we fight it teaching people evolutionary biology facts and biases, and to what extent should we fight it changing the educational system in ?
Sleeping Beauty Resolved (?) Pt. 2: Identity and Betting
by ksvanhorn — 2018-06-02T02:43:24.220Z

Introduction

This is a followup to my previous article, Sleeping Beauty Resolved? Some objected to the solution I presented (building on Radford Neal's analysis) on the grounds that it runs afoul of a Thirder betting argument:

If Beauty uses anything other than a probability of exactly 1/3 for Heads, she will accept certain bets she should not, and reject others she should accept.

Alas, there is an alternative Halfer betting argument that makes the same claim, but replacing 1/3 with 1/2.

I'll show that both arguments are wrong, as they get the effective payoffs wrong; but if Beauty uses the correct effective payoffs, together with a probability of 1/(3−q(y)) for Heads, she makes the right betting decisions.

To get there requires addressing some questions of unique identity, so that's where I'll start.

Indexicals and Identity

Thirder arguments often make use of statements such as

today is Monday

and

today is Tuesday,

treating them as mutually exclusive propositions. Probability theory is based on the classical propositional logic, and deals exclusively with classical propositions; are the above legitimate classical propositions?

I argue that they are not. The word "today" is problematic; it is an indexical, which the article "Demonstratives and Indicatives" in the Internet Encyclopedia of Philosophy defines as

...any expression whose content varies from one context of use to another. The standard list of indexicals includes adverbs such as “now”, “then”, “today”, “yesterday”, “here”, and “actually”.

The article furthermore remarks,

Indexicals and demonstratives raise interesting technical challenges for logicians seeking to provide formal models of correct reasoning in natural language...

and goes on to discuss various efforts to construct logics appropriate for reasoning with indexicals.

Clearly indexicals pose a problem for classical logic, else there would be no interest in constructing alternative logics to deal with them. As Richard Epstein writes in Classical Mathematical Logic: The Semantic Foundations of Logic,

...we cannot take sentence types as propositions if we allow the use of indexicals in our reasoning

(p. 4). This is because the meaning of a classical proposition must be definite and stable:

When we reason together, we assume that words will continue to be used in the same way... We will assume that throughout any particula

... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Three types of "should"
by Sniffnoy — 2018-06-02T00:54:49.171Z

Note: This post is about "should" in how people think, in human psychology -- not about "should" in some deeper/broader philosophical sense that might apply to general agents.

I think when people talk about how they "should" do something, there's basically three different types of motivation/shouldness that are meant, and I think noticing this helps make sense of the idea of "getting rid of 'should'" that some people talk about. I'll call the three "external", "internal", and "interalized". (I don't really like this terminology but it's what I've come up with. If you have a suggestion for something better, let me know.)

External here refers to things that are motivated entirely externally -- I don't particularly want to do X, but things will not go well for me if I don't. External "should" has no real moral component to it -- here by "moral" I mean that not in the broad consequentialist sense of what one should do (here by "should" I do mean that in a broader sense rather than a human sense!) but rather that thing that people think of as morality (like for example things that deal with other people's welfare and not one's own, or things being forbidden-or-allowed-or-mandatory).

Internal is the opposite, purely internal -- I feel that X is something that needs to be done and so I am internally motivated to do it. Well, OK -- that description has the problem that it doesn't do enough to distinguish it from the third one, "internalized". I'm hoping the distinction will become clear in a moment when I discuss the third.

Internalized is the nasty one that you want to avoid, the source of scrupulosity, the reason that people talk about "getting rid of 'should'". It's when you take someone else's morality, that you are not allowed to question, and internalize it as binding on yourself. As I said -- this is where scrupulosity comes from; it's not a good thing.

The thing is that if you're not already aware of the distinction it can be hard to describe internal or internalized in a way that couldn't also describe the other. Like, internalized masks itself as internal. Above I said that internal is things that you want to do, that you feel need to be done -- but the person in the grip of scrupulosity would just say, yes, I want to do these things...