trentmkelly/LessWrong-43k huggingface dataset:
Global Catastrophe Prevention Plan Comprehensive Working Outline (wip)
by Sailor_Vulcan — 2017-10-18T03:05:30.130Z
Link: https://docs.google.com/document/d/1b_FHAypOIB2VcrC93Im1_09qpo7vyQ84b3WCOamCdao/edit?usp=sharing

Hi. It occurred to me to wonder if there was any unified practical step by step plan for getting our species to actually survive. I figured this is the kind of thing that would benefit from being organized into a collaborative to-do list or goal/subgoal list/map of sorts. I looked online and the closest things I was able to find to this were the Immortality Road Map website's "Existential Risks Prevention Roadmap" and a research paper by the Global Catastrophic Risk Institute titled "Towards an Integrated Assessment of Global Catastrophic Risk".

As far as I can tell, the "Existential Risks Prevention Roadmap" does not lay out a specific unified practical step by step plan for saving the world, it is instead a very general map of a variety of different endeavors that can be done to help at different points in our technological and sociocultural development, without breaking it down into a unified practical step by step plan. And GCRI's assessment that I just mentioned does not appear to lay out a unified practical step by step plan either.

I've read a lot about the subject over the past few years though, and I decided to try writing a rudimentary first draft of such a plan. I think I might have done a better job of it than I expected, though it is still far from complete. Could you guys perhaps look over what I have written so far and give some feedback?

Thanks!

https://docs.google.com/document/d/1b_FHAypOIB2VcrC93Im1_09qpo7vyQ84b3WCOamCdao/edit?usp=sharing
Seek Fair Expectations of Others’ Models
by Zvi — 2017-10-17T14:30:00.989Z

Epistemic Status: Especially about the future.

Response To (Eliezer Yudkowsky): There’s No Fire Alarm for Artificial General Intelligence

It’s long, but read the whole thing. Eliezer makes classic Eliezer points in classic Eliezer style. Even if you mostly know this already, there’s new points and it’s worth a refresher. I fully endorse his central point, and most of his supporting arguments.

What Eliezer has rarely been, is fair. That’s part of what makes The Sequences work. I want to dive in where he says he’s going to be blunt – as if he’s ever not been – so you know it’s gonna be good:

Okay, let’s be blunt here. I don’t think most of the discourse about AGI being far away (or that it’s near) is being generated by models of future progress in machine learning. I don’t think we’re looking at wrong models; I think we’re looking at no models.

I was once at a conference where there was a panel full of famous AI luminaries, and most of the luminaries were nodding and agreeing with each other that of course AGI was very far off, except for two famous AI luminaries who stayed quiet and let others take the microphone.

I got up in Q&A and said, “Okay, you’ve all told us that progress won’t be all that fast. But let’s be more concrete and specific. I’d like to know what’s the least impressive accomplishment that you are very confident cannot be done in the next two years.”

There was a silence.

Eventually, two people on the panel ventured replies, spoken in a rather more tentative tone than they’d been using to pronounce that AGI was decades out. They named “A robot puts away the dishes from a dishwasher without breaking them”, and Winograd schemas. Specifically, “I feel quite confident that the Winograd schemas—where we recently had a result that was in the 50, 60% range—in the next two years, we will not get 80, 90% on that regardless of the techniques people use.”

A few months after that panel, there was unexpectedly a big breakthrough on Winograd schemas. The breakthrough didn’t crack 80%, so three cheers for wide credibility intervals with error margin, but I expect the predictor might be feeling slightly more nervous now with one year left to go. (I don’t think it was the breakthrough I remember reading about, but Rob turned up this paper as an example of one that could have been submitted at most 44 day

...
Anti-tribalism and positive mental health as high-value cause areas
by Kaj_Sotala — 2017-10-17T10:20:03.359Z

I think that tribalism is one of the biggest problems with humanity today, and that even small reductions of it could cause a massive boost to well-being.

By tribalism, I basically mean the phenomenon where arguments and actions are primarily evaluated based on who makes them and which group they seem to support, not anything else. E.g. if a group thinks that X is bad, then it’s often seen as outright immoral to make an argument which would imply that X isn’t quite as bad, or that some things which are classified as X would be more correctly classified as non-X instead. I don’t want to give any specific examples so as to not derail the discussion, but hopefully everyone can think of some; the article “Can Democracy Survive Tribalism” lists lot of them, picked from various sides of the political spectrum.

Joshua Greene (among others) makes the argument, in his book Moral Tribes, that tribalism exists for the purpose of coordinating aggression and alliances against other groups (so that you can kill them and take their stuff, basically). It specifically exists for the purpose of making you hurt others, as well as defend yourself against people who would hurt you. And while defending yourself against people who would hurt you is clearly good, attacking others is clearly not. And everything being viewed in tribal terms means that we can’t make much progress on things that actually matter: as someone commented, “people are fine with randomized controlled trials in policy, as long as the trials are on things that nobody cares about”.

Given how deep tribalism sits in the human psyche, it seems unlikely that we’ll be getting rid of it anytime soon. That said, there do seem to be a number of things that affect the amount of tribalism we have:

* As Steven Pinker argues in The Better Angels of Our Nature, violence in general has declined over historical time, replaced by more cooperation and an assumption of human rights; Democrats and Republicans may still hate each other, but they generally agree that they still shouldn’t be killing each other.

* As a purely anecdotal observation, I seem to get the feeling that people on the autism spectrum tend to be less tribal, up to the point of not being able to perceive tribes at all. (this suggests, somewhat oddly, that the world would actually be a better place if everyo...
Defense against discourse
by Benquo — 2017-10-17T09:10:01.023Z

So, some writer named Cathy O’Neil wrote about futurists’ opinions about AI risk. This piece focused on futurists as social groups with different incentives, and didn’t really engage with the content of their arguments. Instead, she points out considerations like this:

First up: the people who believe in the singularity and are not worried about it. […] These futurists are ready and willing to install hardware in their brains because, as they are mostly young or middle-age white men, they have never been oppressed.

She doesn’t engage with the content of their arguments about the future. I used to find this sort of thing inexplicable and annoying. Now I just find it sad but reasonable.

O’Neil is operating under the assumption that the denotative content of the futurists’ arguments is not relevant, except insofar as it affects the enactive content of their speech. In other words, their ideology is part of a process of coalition formation, and taking it seriously is for suckers.

AI and ad hominem

Scott Alexander of Slate Star Codex recently complained about O’Neil’s writing:

It purports to explain what we should think about the future, but never makes a real argument for it. It starts by suggesting there are two important axes on which futurists can differ: optimism vs. pessimism, and belief in a singularity. So you can end up with utopian singularitarians, dystopian singularitarians, utopian incrementalists, and dystopian incrementalists. We know the first three groups are wrong, because many of their members are “young or middle-age white men” who “have never been oppressed”. On the other hand, the last group contains “majority women, gay men, and people of color”. Therefore, the last group is right, there will be no singularity, and the future will be bad.

[…]

The author never even begins to give any argument about why the future will be good or bad, or why a singularity might or might not happen. I’m not sure she even realizes this is an option, or the sort of thing some people might think relevant.

Scott doesn’t have a solution to the problem, but he’s taking the right first step - trying to create common knowledge about the problem, and calling for others to do the same:

I wish ignoring this kind of thing was an option, but this is how our culture relates to things now. It seems important to mention that, to have it out in the open, so that people who turn out thei

...
The Balance Between Hard Work and Exhaustion
by Rossin — 2017-10-17T04:06:39.572Z

Rationalists often find difficult, important challenges to work on and they become very excited and passionate about their causes. I expect it is common (because it happened to me and I have heard references to similar episodes by others) that such causes seem so important that aspiring rationalists set unreasonably high standards of dedication for themselves.

I think the concept of giving an extraordinary effort is a very important one, your brain wants to be lazy and if you are trying to do something challenging that you think is vitally important you want to push yourself to actually work hard, rather than merely “work hard”. A significant portion of this effort is exploring how to make raw effort more effective, but I what I want to address with this essay is how rest factors into attempting an extraordinary effort.

There is a certain kind of person who really needs to be warned that making an extraordinary effort does not mean one should try to work well past the point of exhaustion all the time. Part of giving an extraordinary effort is listening to your body, learning what you can do short of exhaustion, and maintaining that over time. Trying as hard as you possibly can until you burn out from overwhelming exhaustion is a merely desperate effort, and while it may be better than no effort at all, one can do much better by thinking in longer strategic terms.

It is easy to imagine an unattainably high standard of dedication you ought to have to your cause (it is so high because solving the problem is so tremendously important—there is little room for unimportant considerations like comfort). An ideal agent probably would work that much. However, it is critical to remember that we are humans rather than ideal agents. That usually means we cannot consistently do as much work as the most important problems seem to deserve. If we try, our brains will slowly give us worse and worse performances.

To someone who thinks they ought to be working that hard, this exhaustion is very distressing. They may not think to stop and revaluate, but believe the virtuous path is to continue striving onward through exhaustion. They have entered what I like to call a Humanistic Fervor. They have found a really important cause which they are wonderfully excited about working on it and they will do so will an unhealthy zeal. This approach, of forcing willpower to battle exhaustio

...
That's a Thing!
by abramdemski — 2017-10-16T22:25:41.899Z

Lauren Lee responded on facebook to my post about locating yourself as an instance of a class:

Riffing on this LW post by Abram Demski (https://www.lesserwrong.com/…/placing-yourself-as-an-instan…)

In particular this paragraph:

<< For example, if a person is trying to save money but sees a doodad they'd like to buy, the fool reason as follows: "It's just this one purchase. The amount of money isn't very consequential to my overall budget. I can just save a little more in other ways and I'll meet my target." The wise person reasons as follows: "If I make this purchase now, I will similarly allow myself to make exceptions to my money-saving rule later, until the exception becomes the rule and I spend all my money. So, even though the amount of money here isn't so large, I prefer to follow a general policy of saving, which implies saving in this particular case." A very wise person may reason a bit more cleverly: "I can make impulse purchases if they pass a high bar, such that I actually only let a few dollars of unplanned spending past the bar every week on average. How rare is it that a purchase opportunity costing this much is at least this appealing?" **does a quick check and usually doesn't buy the thing, but sometimes does, when it is worth it** >>

The above is a rough outline for one set of moves you can make, but there are ways you can enhance the decision process even further.

One way you can generally increase your "wisdom" is to notice what specific environmental cues are /relevant/ to WHY the impulse occurred in the first place. I think these cues are initially not obvious, at least in my experience.

To illustrate, say I walk into a random store.

What factors determine the likelihood that I'm going to want to impulsively buy a thing?

There are times when I walk into a store without a plan, and I can be confident I'm going to walk out of it without having purchased anything. Other times, I feel more like I will have ended up purchasing something. What is giving me these clues?

The situation of me being in a grocery store while I'm hungry is a different /class/ of situation from me being in a grocery store while I'm full. And so I want to treat these classes as somewhat separate when making decisions.

Somehow, I figured out at some point that this was a differe

...
Outside View as the Main Debiasing Technique
by abramdemski — 2017-10-16T21:53:10.722Z

The inside view refers to what your explicit models tell you. For example, if you write down a plan with explicit contingencies for things going wrong and time estimates for each step, or write down a careful argument for your point of view, or solve some equations. The outside view refers to what "an independent observer" might think; the proverbial man-on-the-street; a perspective which doesn't pay attention to all the details of the situation, but places it as an instance of a class. For a plan, you have some outside-view probability that it will fail or take longer than you expect, based on unforeseen complications rather than explicitly-listed contingencies. For an argument, although the argument may conclude a belief with some high probability, your outside-view probability should usually be lower to account for your uncertainty that the argument is correct. For the equations, the outside view accounts for the probability that you've made a mathematical error.

One way of thinking about inside view vs outside view is that they are human-friendly ways of thinking about the likelihood function and the prior. Somehow, humans are frequently tempted to commit base-rate neglect: the belief in a hypothesis is just the extent to which it matches the data. For example, if a test for a rare disease comes up positive. People are not inclined by default to multiply the evidence for something by the prior odds against, except in everyday circumstances which they've experienced a lot. (I think this mistake may also account for the conjunction fallacy.) "Outside view" helps you imagine base rates by envisioning a perspective with more distance from the issue. So, you come up with an inside view and an outside view and "multiply them together" to become a proper Bayesian. This gets rid of base-rate neglect, and related biases such as the availability heuristic (where you implicitly use "number of times I've heard about this" or similar things as a substitute for base-rates).

However, I think this underestimates the usefulness of outside view. Robin Hanson discusses it almost as if it is equivalent to eliminating bias. While I do think a strong outside view needs to be combined with a strong inside view, I think there's a sense in which outside view is the first thing you reach for to debias almost anything,

...
Open thread, October 16 - October 22, 2017
by root — 2017-10-16T18:53:16.852Z

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top-level comments on this article" and "Make this post available under..." before submitting.
On the construction of beacons
by Benquo — 2017-10-16T09:10:00.866Z

I am afraid of the anglerfish. Maybe this is why the comments on my blog tend to be so consistently good.

Recently, a friend was telling me about the marketing strategy for a project of theirs. They favored growth, in a way that I was worried would destroy value. I struggled to articulate my threat model, until I hit upon the metaphor of that old haunter of my dreamscape, the anglerfish.

I am sure that something like the general process I'm describing is common and important. I am not sure at all of the details, but I am going to try to state this strongly and vividly, with little hedging, for the sake of clarity.

The Anglerfish

The anglerfish lives in waters too far beneath the surface of the sea for sunlight to reach. It dangles a luminescent lure in front of itself. This resembles a fishing angle, whence comes its name. This lure attracts animals of the deep sea, which approach the anglerfish, and are consequently eaten by it.

Why - in the deep sea where no sunlight can reach - would evolution favor animals that are attracted to light? I do not know exactly what strategy the Anglerfish's prey are following, but we do know some general things about why an animal might be attracted to light.

The secondary uses of such a strategy are clear enough. Once some deep-sea-dwellers emit light, larger animals that predate on them might do better if attracted to light sources. But that presupposes the existence of other animals that already emit light, for other reasons.

What are the primary uses of light? In a region where no other creatures emit light, here are some reasons why would might begin to do so:

To illuminate potential prey.
As a ward, to warn potential competitors that one is prepared to defend territory.
To attract complementary animals, either as symbiotes, or as mates.

In all these cases, the purpose of the light is to reveal information. In all but the first case, it is to share information with others, in order to enable cooperation. Perhaps the purest version of this is the mating display. We can see this in the firefly, which uses its distinctive patterns of luminescent flashes to find mates.

The firefly has some information. It activates a beacon, in order to find someone with complementary information, in order to engage in productive exchange. Likewise for deep-sea fish who mate or find symbiotes by means of a light display.

Some fireflies mimic t...
Multidimensional signaling
by KatjaGrace — 2017-10-16T07:00:00.176Z

What do you infer about a person who has ugly clothing? Probably that they have poor taste (in clothes, or subcultures). But it could also be that they are too poor to improve their wardrobe. Or can’t be bothered.

What about someone with poor grades? The obvious inference is that they aren’t so capable at the subject, but it may again be that they can’t be bothered, or that they have more urgent things to do with their time.

And someone who makes clever jokes? Probably that they are smart and naturally funny, but if they had more time or effort to spend on this, it probably helped.

For all kinds of traits that people might try to signal with their behavior, someone can send a better signal if they have more money or time or self-control. Even when the main signal being sent is not usually thought to be about any of those things.

The reason this interests me: if signals often divide the population into ‘better or richer’ vs. ‘worse or poorer’, I wonder if this would cause us to imagine that being rich is associated with being better, even if the two were entirely independent. (And similarly for wealth in other general-use resources, like self-control and time).

In a simple case, suppose there are just people with pretty clothes (who are both rich and have good taste) and people with ugly clothes (who either have bad taste, or lack resources or will). Then do observers come to think of ‘rich good taste’ type and a ‘poor bad taste’ type? Or do they pay more attention to the actual structure of the space, and know for instance that this doesn’t mean learning that someone really has bad taste actually means they are probably poorer.

Note that I’m not merely suggesting that a person with more wealth can send signals to look like they are better—that much is clear. I’m suggesting that at a population level, if the wealthier people can’t be distinguished from the better people on some axis, then observers may come to think that the two are associated in general, even if they are not at all.

If so, this would be important, because it would apply in a huge range of cases of signaling. So that the properties of poverty and weak-willedness and such would appear to us to be much worse than they really were.
Against naming things, and so on
by whales — 2017-10-15T23:48:05.600Z

Recent discussion on naming concepts mostly focuses on arguments in favor, noting only a few caveats, as LW user Conor Moreton in Why and How to Name Things:

What you lose by the proliferation of jargon is ease-of-entry and cross-cultural intelligibility and hard-drive space in the brains of people trying to track all of it.

I think you lose more than that, so I'll try to name [sic] a few more reasons your caution might run deeper.

1. The nomothetic fallacy

Diagnosis isn't a cure, but it can bring a sense of relief untethered from prognosis. You might still have the same symptoms that still need the same treatment, but it feels like a large part of the problem has been solved. This is, I'd argue, a decent chunk of why knowing about biases can hurt you. Don't get complacent just because you have a name for something.

2. Weaponized rationality

Another big chunk of "knowing about biases can hurt you" comes from wielding concepts not in introspection but against others. (Like many of the considerations below, taking this as compelling is a general argument against much of the rationality project. But proliferating jargon, to my eye, makes most of the problems here relatively worse for external use as compared to internal use.)

3. Being wrong

Sometimes your analysis of the thing you're trying to crystallize just isn't very good. Your analogy doesn't play out the way you think, the pattern doesn't actually exist, you're not carving reality at the joints, your framing could be better, it's an inappropriate level of abstraction anywhere you'd actually want to use it. But, hey, the name stuck, so you can communicate all that in just a word!

4. Lossy compression

Maybe you got it right this time, but the name doesn't capture everything. In practice, nobody's going to remember your entire blog post every time someone utters the title. I hope you didn't need that nuance. But at least this one went viral!

5. Thinking on the page

From Conor's post:

When you define a concept rigorously and clearly, you almost always learn new things from playing around with their edges and trying to get them to interface with other rigorous, clear concepts.

And if you don't define it rigorously and clearly—as is the case for pretty much everything happening here—if you play directly with the compressed concepts and

...