trentmkelly/LessWrong-43k huggingface dataset:
Critch on career advice for junior AI-x-risk-concerned researchers
by Rob Bensinger — 2018-05-12T02:13:28.743Z

In a recent e-mail thread, Andrew Critch sent me the following "subtle problem with sending junior AI-x-risk-concerned researchers into AI capabilities research". Here's the explanation he wrote of his view, shared with his permission:

I'm fairly concerned with the practice of telling people who "really care about AI safety" to go into AI capabilities research, unless they are very junior researchers who are using general AI research as a place to improve their skills until they're able to contribute to AI safety later. (See

Leveraging Academia

).

The reason is not a fear that they will contribute to AI capabilities advancement in some manner that will be marginally detrimental to the future. It's also not a fear that they'll fail to change the company's culture in the ways they'd hope, and end up feeling discouraged. What I'm afraid of is that they'll feel pressure to start pretending to themselves, or to others, that their work is "relevant to safety". Then what we end up with are companies and departments filled with people who are "concerned about safety", creating a false sense of security that something relevant is being done, when all we have are a bunch of simmering concerns and concomitant rationalizations.

This fear of mine requires some context from my background as a researcher. I see this problem with environmentalists who "really care about climate change", who tell themselves they're "working on it" by studying the roots of a fairly arbitrary species of tree in a fairly arbitrary ecosystem that won't generalize to anything likely to help with climate change.

My assessment that their work won't generalize is mostly not from my own outside view; it comes from asking the researcher about how their work is likely to have an impact, and getting a response that either says nothing more than "I'm not sure, but it seems relevant somehow", or an argument with a lot of caveats like "X might help with Y, which might help with Z, which might help with climate change, but we really can't be sure, and it's not my job to defend the relevance of my work. It's intrinsically interesting to me, and you never know if something could turn out to be useful that seemed useless at first."

At the same time...
A Self-Respect Feedback Loop
by ialdabaoth — 2018-05-11T23:09:02.086Z

This is a followup to Affordance Widths.

Epistemic Status: It’s only a model

Okay! This is something I’ve been trying to explain for awhile, but I think I have a handy chart for it now.

Here’s how it works:

A person can actually regulate how much self-respect they feel, and show. Other people will reward them for having more self-respect, up to a point.

Then they start pushing back.

BUT, each of these “pushbacks” is a temporary dip in the “self-respect to positive feedback” curve. You just have to have enough self-control, or willpower, or “grit”, or “spoons”, or whatever, to keep pushing through and powering more and more self-respect while people attack you for having it, until you break through into the next upswing of the curve.

The thing is, a lot of self-control/willpower/grit/spoons/etc. is powered by people not treating you like shit.

It seems like there are actually three different dips that occur, each with a wider gap than the last.

Some people try to push up into a gap, discover they don’t have enough willpower to escape to the far side of the dip, give up, and fall back into the previous sustainable peak.

Those that can’t even make it past the first peak are losers - people that everyone can tell can’t even get their basic needs met. They make it obvious that they have needs when they’re in the “needy” dip, but never manage to show enough self-respect for anyone else to feel like their needs matter.

Those that can’t make it past the second peak are doormats - people who can’t enforce their boundaries or reasonably request basic fairness. They make it obvious that that they object to the situation they’re in when they push themselves into the “entitled” dip, but never manage to show enough self-respect for anyone else to feel like respecting those boundaries or requests.

Those that can’t make it past the third peak are the vast majority of the human population - people who can’t pull off the Steve Jobs level of demanding other people’s resources and time and just getting it. They make it obvious that that they want more - or even think they deserve more - when they push themselves into the “arrogant” dip, but never manage to show enough self-respect for anyone else to feel like following them into the breach.

There are a few people have their goal and identity set on being in a particular peak, higher than the one they’re on, and keep pushing and pushing and...
Welcome to the Vancouver Rationality Community 
by Evan_Gaensbauer — 2018-05-11T06:37:34.491Z

We're the rationality community in Vancouver, British Columbia, Canada. We organize meetups on all kinds of topics, like LessWrong, Slate Star Codex, and effective altruism. We're aspiring to launch a MIRIx workshop. We also host events focusing on other intellectual activities like lightning talks, reading groups and debate clubs; and social events like parties, game nights and hang outs. We also occasionally co-host events with Burners, life extensionists, or effective altruists in the Vancouver area. We usually end up having a couple meetups a month. A lot of us lived clusters in some of the same neighbourhoods, so we most often meet in Burnaby, Kitsilano, UBC, or Downtown.

We're friends of local organizations and communities like:

Rethink Charity, an educational and evidence-based research organization in the effective altruism community.

The Lifespan Society of BC, the province's life extension project and advocacy society.

DCTRL, the local, (both figuratively and literally) underground cryptocurrency society.

The University of British Columbia Effective Altruism Club.

We intend to start reading groups for Slate Star Codex and Rationality: From AI to Zombies in the near future, so stay tuned.
Affordance Widths
by ialdabaoth — 2018-05-10T23:37:43.037Z

This article was originally a post on my tumblr. I'm in the process of moving most of these kinds of thoughts and discussions here.

Okay. There’s a social interaction concept that I’ve tried to convey multiple times in multiple conversations, so I’m going to just go ahead and make a graph.

I’m calling this concept “Affordance Widths”.

Let’s say there’s some behavior {B} that people can do more of, or less of. And everyone agrees that if you don’t do enough of the behavior, bad thing {X} happens; but if you do too much of the behavior, bad thing {Y} happens.

Now, let’s say we have five different people: Adam, Bob, Charles, David, and Edgar. Each of them can do more or less {B}. And once they do too little, {X} happens. But once they do too much, {Y} happens. But where {X} and {Y} starts happening is a little fuzzy, and is different for each of them. Let’s say we can magically graph it, and we get something like this:

Now, let’s look at these five men’s experiences.

Adam doesn’t understand what the big deal about {B} is. He feels like this is a behavior that people can generally choose how much they do, and yeah if they don’t do the *bare minimum* shit goes all dumb, and if they do a *ridiculous* amount then shit goes dumb a different way, but otherwise do what you want, you know?

Bob understands that {B} can be an important behavior, and that there’s a minimum acceptable level of {B} that you need to do to not suffer {X}, and a maximum amount you can get away with before you suffer {Y}. And Bob feels like {X} is probably more important a deal than {Y} is. But generally, he and Adam are going to agree quite a bit about what’s an appropriate amount of {B}ing for people to do. (Bob’s heuristic about how much {B} to do is the thin cyan line.)

Charles isn’t so lucky, by comparison. He’s got a *very* narrow band between {X} and {Y}, and he has to constantly monitor his behavior to not fall into either of them. He probably has to deal with {X} and {Y} happening a lot. If he’s lucky, he does less {B} than average; if he’s not so lucky, then he tries to copy Bob’s strategy and winds up getting smacked with {Y} way more often than Bob does.

Poor David’s in a situation called a “double bind”. There is NO POSSIBLE AMOUNT of {B} he can do to prevent both {X} and {Y} from happening; he simply has to choose his poison. If he tries Bob’s strategy, he’ll get hit hard with {X} *A

...
Tech economics pattern: "Commoditize Your Complement"
by gwern — 2018-05-10T18:54:42.191Z
Link: https://www.gwern.net/Complement

Joel Spolsky in 2002 identified a major pattern in technology business & economics: the pattern of "commoditizing your complement", an alternative to vertical integration, where companies seek to secure a chokepoint or quasi-monopoly in products composed of many necessary & sufficient layers by dominating one layer while fostering so much competition in another layer above or below its layer that no competing monopolist can emerge, prices are driven down to marginal costs elsewhere in the stack, total price drops & increases demand, and the majority of the consumer surplus of the final product can be diverted to the quasi-monopolist.

A classic example is the commodification of PC hardware by the Microsoft OS monopoly, to the detriment of IBM & benefit of MS.

This pattern explains many otherwise odd or apparently self-sabotaging ventures by large tech companies into apparently irrelevant fields, such as the high rate of releasing open-source contributions by many Internet companies or the intrusion of advertising companies into smartphone manufacturing & web browser development & statistical software & fiber-optic networks & municipal WiFi & radio spectrum auctions (Google): they are pre-emptive attempts to commodify another company elsewhere in the stack, or defenses against it being done to them.
Dissolving Scotsmen
by Paperclip Minimizer — 2018-05-10T16:08:11.862Z

[Epistemic status: I don't expect this to break new ground not already in core reading (such as Eliezer Yudkowsky's A Human's Guide to Words or Scott Alexander's Categorisation and Concepts), but I am writing this as a pointer for future reference]

[Trigger warning: analogical discussion of rape, kinda treating Scotsmen as if they're not there]

In Traditional Rationality, the No True Scotsman fallacy is a logical fallacy in which a person saying all X are Y, faced with an alleged counterexample Z where Z is X but not Y, deny the alleged counterexample Z is X, thus saving their statement from falsification.

Scotsmen

The archetypal example of the No True Scotsman fallacy, which gives it its name, is this one:

Imagine Hamish McDonald, a Scotsman, sitting down with his Glasgow Morning Herald and seeing an article about how the "Brighton Sex Maniac Strikes Again". Hamish is shocked and declares that "No Scotsman would do such a thing". The next day he sits down to read his Glasgow Morning Herald again; and, this time, finds an article about an Aberdeen man whose brutal actions make the Brighton sex maniac seem almost gentlemanly. This fact shows that Hamish was wrong in his opinion, but is he going to admit this? Not likely. This time he says: "No true Scotsman would do such a thing".

Flew, Antony (1975), Thinking About Thinking: Do I Sincerely Want to Be Right?, London: Collins Fontana, p. 47, ISBN 978-0-00-633580-1

Reading this, one is tempted to conclude that any argument of this form is wrong and fallacious and should not be used. But consider the following dialogue:

ALICE: All Scotsmen are UK citizens

BOB: But Xi Jinping is a Scotsman, yet he isn't an UK citizen.

ALICE: He is not a true Scotsman.

BOB: You're using the No True Scotsman fallacy !

It's self-evident that Alice is right and Bob is wrong. One might be tempted to call No True No True Scotsman Fallacy, but it fits the definition of the No True Scotsman fallacy we gave earlier: Alice is saying that all Scotsmen are UK citizens and, when faced with the alleged counterexample that Xi Jinping is a Scotsman but not an UK citizen, deny that Xi Jinping is a Scotsman, thus saving her statement from falsification. Either we should abandon all statements of the form "All X are Y" when faced with one of the Bobs of this world, either we should stop calling No True Scotsman a f...
Upvotes, knowledge, stocks, and flows
by Jameson Quinn — 2018-05-10T14:18:15.087Z

This is a short post inspired by "EXEMPLIFYING EQUITABLE GROWTH: MR. GOOGLE SERVES ME A BAKER'S HALF-DOZEN FROM THE WCEG WEBSITE, AND WHAT I LEARN THEREBY...". Delong looks at google results for his blog archive, and finds they are very weakly predictive of what he thinks is useful about his blog. He thinks about that in terms of the "flow" of intellectual discussion — what people are talking about — versus the "stock" — what is known.

I think that's a productive analytical frame. Websites live and die off of their salience to the flow, but their long-term value is what they contribute to the stock. Twitter is optimized for flow to the point where it's nearly useless as stock; but high-quality work that never gets into the flow ends up so unread and hard to find that it has little value as stock.

For sites like Less Wrong, I think that thinking about stock and flow could help to improve how upvotes work. To some extent, upvotes help promote a healthy flow, which is an important and useful goal. But that is essentially a short-term value, and so upvotes like that should decay. On the other hand, upvotes also serve as long-term markers of valuable stock. That kind of upvote shouldn't decay.

One place this kind of thinking could lead would be to having different kinds of upvotes. "Thanks" and "good question" would be more flow-y and thus decay faster; "well said" and "good link" would be more stock-y and last longer.
Thoughts on "AI safety via debate"
by Gordon Seidoh Worley — 2018-05-10T00:44:09.335Z

Geoffrey Irving, Paul Christiano, and Dario Amodei of OpenAI have recently published "AI safety via debate" (blog post, paper). As I read the paper I found myself wanting to give commentary on it, and LW seems like as good a place as any to do that. What follows are my thoughts taken section-by-section.

1 Introduction

This seems like a good time to confess that I'm interested in safety via debate because I thought about it prior to the publication of this paper and don't think it will work. I like the general direction and think it is of the kind of thing that is likely to work because it puts pressure on AI decision processes to be both value aligned and believably aligned, but I think debate as a form has limitations that inherently make it unlikely to produce aligned AI. I instead prefer the idea of safety via dialectic and have been working on a not-yet-ready-for-publication AI alignment proposal I call "dialectical alignment".

I point this out to give some context to my thoughts and lay my biases on the table: I'm already thinking about something pretty similar to what we might call "safety via debate" but am skeptical of debate itself.

2 The debate game

Debate is here described as a specific game with mathematical formalism rather than the fuzzy human process we often call debate. I appreciate this because it lets us compare the game precisely to the dialectic process I prefer.

In the debate game we have two agents. Each is shown the question and asked for an answer independently and with no knowledge of the other agent's answer. Unlike in most human debate the two agents are allowed to give the same answer if they like. The two agents then go back and forth making statements in support of their answers and each agent has knowledge at this point of the other agent's answer and the previous statements made. The game is decided by a (human) judge who knows the question, the answers, and the statements. Critically the debate game is zero-sum and hinges on the claim that it is harder to lie than to refute a lie within the context of the game.

In contrast I think of the dialectic process as one with a single agent who starts with a question and proposes a thesis. The agent presents statements supporting the thesis up to some bound (e.g. number of statements, time spent producing support statements, etc.), then...
Saving the world in 80 days: Prologue
by Logan Riggs — 2018-05-09T21:16:03.875Z

What is this?

This is an intro to an intensive productivity sprint for becoming useful in the field of AI safety. I'll be posting every week or 4 on the good/bad/ugly of it all. Definitely inspired by Nate Soares' productivity sprint.

Who are you?

Recent graduate in Computer Engineering who has someone to protect

Why?

Ultimately to further the field of AI safety. In the process of doing this and writing these posts, I'll:

1. Create a more realistic model of what I can do/ of my limits

2. Receive valuable feedback from other rationalists (that's you!)

3. Radically improve my knowledge of the field & various maths

So what are you going to be doing exactly?

16 hours in a day:

Food/hygiene/slack - 4 1/2 hr
Current mathy book - 3 hr (How to Prove It, atm)
Work - 4 hr (Might change depending on work needs)
Current Application - 2 hr (tensorFlow)
AI-Safety reading - 1 hr (Miri's research guide)
Living Maintenance - 1/2 hr
Meditation ~ 1 hr (Doing squirrelInHell's site, specifically this part)

Aren't you going to flounder/fail/burn-out?

I've done 3 other intensive sessions like this in the past. One was a flop and two were quite successful, so I feel prepared for this one. I'm definitely shooting for sustainability here, so I'm trying to figure out my limits without burning out. This could include getting more work done per hour, getting more efficient rest, or varying the amount of slack time given.

My current Murphyjitsu results:

1. I could waste a lot of time on pica. To prevent that, I'm going to notice when I want to leave my current situation and pursue that and just watch that thought/sensation without fueling it. I've developed this skill while meditating, and have had some success already, though I feel like there's more that's missing. Something like I'm quickly making deals with myself such as "You only have to deal with this for a few more minutes, then you're free".

2. I could get uber mentally tired. If so, I'll move on to living maintenance or meditation which I consider mentally relaxing.

3. Trips & friends taking a lot of time. I'm not going to go out of my way to spend time with people, but important trips (weddings, family) I'm going to for sure, and I'm okay with less productivity; I'm just gonna' enjoy the heck out of it.

[Note: I've a...
Thoughts on AI Safety via Debate
by Vaniver — 2018-05-09T19:46:00.417Z

Geoffrey Irving et al. at OpenAI have a paper out on AI safety via debate; the basic idea is that you can model debates as a two-player game (and thus apply standard insights about how to play such games well) and one can hope that debates asymmetrically favor the party who's arguing for a true position over a false position.

If so, then we can use debates between AI advisors for alignment; suppose an AI develops a new drug that we could give to patients. One advisor argues the case that this will be beneficial; another argues the case that it will be detrimental, and human judges decide who is more convincing. If both advisors are equally smart (perhaps they're mirrored systems, just with different targets), then any superhuman persuasiveness the advisors have should be cancelled out, and we'll be left with just the edge for truthfulness.

There are more subtleties to the proposal; in particular, you want the AIs to not overwhelm the human with data, and so in current examples each advisor can reveal a single pixel of an image to the judge. This moves the contradictions up to the level of interpretation--any pixel where the two advisors agree on its meaning can be ignored, and the attention focused to pixels where the advisors disagree. And, so long as advisors are incentivized to tell the truth when they're arguing for the right thing, any such disagreement where an advisor has been exposed to be lying about a tiny fact can settle the debate.

This is also something we can test and get some evidence on now. There's a debate game where three people can play the three roles--red arguer, blue arguer, and judge. The typical game is that you choose a photo of a cat or a dog, then flip a coin to determine whether blue or red will be honest (with the other one arguing for the other variety). Each arguer can draw boxes on the image to communicate with the judge, and can choose to reveal a single pixel over the course of the argument.

I've played the game several times (and won a bunch), and thought it might be helpful to write up some of my experiences.

1. I know a lot about cats and dogs, and I think this gives me a pretty significant edge. In one game, I was honestly arguing that the image was of a cat; it happened to be a tortoiseshell cat, which is a coloration dogs can't have (brindle is a different thing), and I was able to identi...
Hypotheticals: The Direct Application Fallacy
by Chris_Leong — 2018-05-09T14:23:14.808Z

A few years ago, I tried convincing people some commenters that hypotheticals were important even when they weren't realistic. That failed, but I think I've spend enough time reflecting to give this another go. This time, my focus will be on challenging the following common assumption:

The Direct Application Fallacy: If a hypothetical situation can't conceivably occur, then the hypothetical situation doesn't matter

I chose this name because it assumes that the only purpose of discussing a hypothetical is to know what would happen or what we should do in such a situation. It ignores the other lessons that such a discussion may teach us and how it might have logical consequences for situations that actually do occur.

(Note: This post was renamed from: Unrealistic Hypotheticals Still Contain Lessons)

Exploiting Opportunities for Learning

In The Least Convenient Possible World, Scott Alexander considers the classic objection to utilitarianism that it implies that a surgeon should be prepared to harvest the organs of a random traveller if it would allow them to save five other patients. Scott argues that pointing out that the random traveller's organs probably be genetic mismatches, while "technically correct", also "completely misses the point and loses a valuable opportunity to examine the nature of morality". He also notes that responding in this manner leaves too much "wiggle room". Even if we aren't consciously aware of it, we often construct arguments to avoid believing things that we don't want to, so we can improve our rationality by limiting our ability to avoid understanding the other person's perspective. While Scott is referring to people who completely miss the point of the hypothetical, I think that dismissing a hypothetical as unrealistic often also sacrifices opportunities for learning as we'll see below.

Practise Exercises Don't Need to be Real

Imagine that you are an instructor setting problems for your students so that they can learn an area like economics, physics or applied maths. How strongly do you care about these exercises being realistic? I would argue that this isn't very important and that this further applies to philosophy:

1. Simplification: Students may be at a point where a realistic exercise would be quite beyond their abilities. Imagine that you are tryi...