trentmkelly/LessWrong-43k huggingface dataset:
My IRB Nightmare
by Scott Alexander — 2017-09-28T16:47:54.661Z

[Epistemic status: Pieced together from memory years after the event. I may have mis-remembered some things or gotten them in the wrong order. Aside from that – and the obvious jokes – this is all true. I’m being deliberately vague in places because I don’t want to condemn anything specific without being able to prove anything.]

September 2014

There’s a screening test for bipolar disorder. You ask patients a bunch of things like “Do you ever feel really happy, then really sad?”. If they say ‘yes’ to enough of these questions, you start to worry.

Some psychiatrists love this test. I hate it. Patients will say “Yes, that absolutely describes me!” and someone will diagnose them with bipolar disorder. Then if you ask what they meant, they’d say something like “Once my local football team made it to the Super Bowl and I was really happy, but then they lost and I was really sad.” I don’t even want to tell you how many people get diagnosed bipolar because of stuff like this.

There was a study that supposedly proved this test worked. But parts of it confused me, and it was done on a totally different population that didn’t generalize to hospital inpatients. Also, it said in big letters THIS IS JUST A SCREENING TEST IT IS NOT INTENDED FOR DIAGNOSIS, and everyone was using it for diagnosis.

So I complained to some sympathetic doctors and professors, and they asked “Why not do a study?”

Why not do a study? Why not join the great tradition of scientists, going back to Galileo and Newton, and make my mark on the world? Why not replace my griping about bipolar screening with an experiment about bipolar screening, an experiment done to the highest standards of the empirical tradition, one that would throw the entire weight of the scientific establishment behind my complaint? I’d been writing about science for so long, even doing my own informal experiments, why not move on to join the big leagues?

For (it would turn out) a whole host of excellent reasons that I was about to learn.

A spring in my step, I journeyed to my hospital’s Research Department, hidden in a corner office just outside the orthopaedic ward. It was locked, as always. After enough knocking, a lady finally opened the door and motioned for me to sit down at a paperwork-filled desk.

“I want to...
Blind Goaltenders: Unproductive Disagreements
by PDV — 2017-09-28T16:19:07.241Z

If you're worried about an oncoming problem and discussing it with others to plan, your ideal interlocutor, generally, is someone who agrees with you about the danger. More often, though, you'll be discussing it with people who disagree, at least in part.

The question that inspired this post was "Why are some forms of disagreement so much more frustrating than others?" Why do some disagreements feel like talking to a brick wall, while others are far more productive?

My answer is that some interlocutors are 'blind goaltenders'. They not only disagree about the importance of your problem, they don't seem to understand what it is you're worried about. For example, take AI Safety. I believe that it's a serious problem, most likely the Most Important Problem, and likely to be catastrophic. I can argue about it with someone who's read a fair chunk of LessWrong or Bostrom, and they may disagree, but they will understand. Their disagreement will probably have gears. This argument may not be productive, but it won't be frustrating.

Or I could talk to someone who doesn't understand the complexity of value thesis or orthogonality thesis. Their position may have plenty of nuances, but they are missing a key concept about our disagreement. This argument may be just as civil - or, given my friends in the rationalsphere, more civil - but it will be much more frustrating, because they are a blind goaltender with respect to AI safety. If I'm trying to convince them, for example, not to support an effort to create an AI via a massive RL model trained on a whole datacenter, they may take into account specific criticisms, but will not be blocking the thing I care about. They can't see the problem I'm worried about, and so they'll be about as effective in forestalling it as a blind goalie.

Things this does not mean

Blind goaltenders are not always wrong. Lifelong atheists are often blind goaltenders with respect to questions of sin, faith, or other religiously-motivated behavior.

Blind goaltenders are not impossible to educate. Most people who understand your pet issue now were blind about it in the past, including you.

Blind goaltenders are not stupid. Much of the problem in AI safety is that there are a great deal of smart people working in ML who are nonetheless blind goaltenders.

Goaltenders who cease

...
For signaling? (Part I)
by KatjaGrace — 2017-09-28T07:00:00.605Z

Your T-shirt is embarrassing. Have you considered wearing a less embarrassing T-shirt?

You are suggesting I spend my precious time trying to look good. Well I am good, and so I’m not going to do that. Because signaling is bad. You can tell something is bad when the whole point of it is to have costs. Signaling is showing off. Signaling benefits me at someone else’s equal expense. I won’t wear a less embarrassing T-shirt because to Hell with signaling.

Hmm. That seems wrong. Signaling is about honest communication when the stakes are high—which is often important! And just because it’s called ‘costly’ doesn’t mean it is meant to have costs. It only has to be too costly for liars, and if it’s working then they won’t be doing any signaling anyway. ‘Costly signals’ can be very cheap for those who use them. I think signaling is often wonderful for society.

Give me three examples where it is ‘wonderful’.

Driver’s licences. Showing a driver’s licence is a costly signal of being a decent driver, which communicates something useful honestly, is cheap for the people who are actually good drivers, and lets the rest of society distinguish people who are likely to drive safely from people who are not, which is amazingly great.

Driving tests don’t seem that cheap to me, but I’ll grant that they are probably worth it. Still, this seems like a strange corner case of ‘signaling’ that was explicitly designed by humans. It fits the economic definition of ‘costly signaling’ but if you have to go that far from the central examples to find something socially beneficial, that doesn’t increase my regard for signaling. Next?

One of the most famous examples of signaling is in the job market. Potential candidates show a hirer their qualifications, which allows the hirer to employ more appropriate candidates. You might disagree about whether all of the signals that people use are socially optimal—for instance if education is mostly for signaling, it seems fairly destructive, because it is so expensive. But you must agree that companies do a lot better hiring the people they choose than they would hiring random people they would get if good candidates couldn’t signal their quality. And at least many aspects of the interview process are cheap enough to be totally worth it. For instance, being able to have a polite and friendly conversation about the subject matter.

Of course companies are bette

...
Musings on Double Crux (and "Productive Disagreement")
by Raemon — 2017-09-28T05:26:01.246Z

Epistemic Status: Thinking out loud, not necessarily endorsed, more of a brainstorm and hopefully discussion-prompt.

Double Crux has been making the rounds lately (mostly on Facebook but I hope for this to change). It seems like the technique has failed to take root as well as it should. What's up with that?

(If you aren't yet familiar with Double Crux I recommend checking out Duncan's post on it in full. There's a lot of nuance that might be missed with a simple description.)

Observations So Far

Double Crux hasn't percolated beyond circles directly adjacent to CFAR (it seems to be learned mostly be word of mouth). This might be evidence that it's too confusing or nuanced a concept to teach without word of mouth and lots of examples. It might be evidence that we have not yet taught it very well.
"Double Crux" seems to refer to two things: the specific action of "finding the crux(es) you both agree the debate hinges on" and "the overall pattern of behavior surrounding using Official Doublecrux Technique". (I'll be using the phrase "productive disagreement" to refer to the second, broader usage)

Double Crux seems hard to practice, for a few reasons.

Filtering Effects

In local meetups where rationality-folk attempt to practice productive disagreement on purpose, they often have trouble finding things to disagree about. Instead they either:

are already filtered to have similar beliefs,
quickly realize their beliefs shouldn't be that strong (i.e. they disagree on Open Borders, but soon as they start talking they admit that neither of them really have that strong an opinion)
they have wildly different intuitions about deep moral sentiments that are hard to make headway on in a reasonable amount of time - often untethered to anything empirical. (i.e. what's more important? Preventing suffering? Material Freedom? Accomplishing interesting things?)

Insufficient Shared Trust

Meanwhile in many online spaces, people disagree all the time. And even if they're both nominally rationalists, they have an (arguably justified) distrust of people on the internet who don't seem to be arguing in good faith. So there isn't enough foundation to do a productive disagreement at all.
One failure mode of Double Crux is when people disagree on what frame to even be using to evaluate truth, in which c

...
Exposition and guidance by analogy
by whales — 2017-09-28T03:33:36.672Z

(Below I attempt to reproduce a chart I can't [?] embed, from Models and Analogies in Science by Mary Hesse; I don't know how it will display on all screens, but the source is this image at the top of the cross-post version here.)

At an elementary level we can set up the following correspondences:

WATER WAVES | SOUND | LIGHT
Produced by motion | Produced by motion | *Produced by moving
of water particles | of gongs, strings | flame, etc.
Reflection | Echoes, etc. | Reflection in mirrors
Properties of dif- | Hearing round | Diffraction through
fraction | corners | small slits, etc.
Amplitude | Loudness | Brightness
Frequency | Pitch | Color
Medium: Water | Medium: Air | *Medium: "Ether"

In the table, we see a lot of apparent correspondences between water waves, sound, and light. The “horizontal” notion of similarity lets us notice that sound echoes and light reflects, or that these things all have some sort of “intensity” and “flavor”.

But it’s important to introduce the “vertical” analogy—the items in each column are related by some causal, organizing principle, and there’s a correspondence between those principles. We expect all sorts of things to have similar traits entirely by accident. You won’t get very far filling in a table’s gaps by arguing “this is like that”—better to say “this model is like that model”. You’re taking advantage of a ready-made language (with entailed internal relationships) for a metaphoric redescription of a new subject. In this way analogies can be a useful guide to teaching and learning new models in new domains. (You’ll realize, for example, that “produced by moving flame” isn’t really the appropriate correspondence there, because the motion of the flame doesn’t have to do with color in the way that the motion of a gong has to do with pitch, and eventually you’ll learn something about the production of light.) But what’s this about the medium of light—“ether”?

Well, if we observe that the vertical analogy works for the first two models, and it works for the third up to that point, then light having a medium starts seeming plausible enough for us to start looking into it. And it additionally suggests to us how to go about looking. But while the vertical analogy gives us a stronger induc...
Speed & Performance is our current top priority
by habryka — 2017-09-27T23:36:25.321Z

TLDR: Fewer bugfixes and support for the next week or two while we are focusing on making the site fast and stable. General shift in direction towards more stable and polished features over a large breadth of slightly buggy features.

It's been one week since the launch of the open beta. Since then we had some amazing posts, fixed tons of bugs, answered over 200 of your support questions on Intercom and added over 60 user-submitted issues to our Github. I think this week has been a good start for the new LessWrong, in large parts because of your continuous feedback and input. Thanks!

But despite the positives, and this became clear to us thanks to your input, there is a core problem with the site. LessWrong 2.0 is slow right now. Like, unbearably slow sometimes (I've experiences load times of 10+ seconds, which is totally unacceptable). This has probably caused many of you headaches, and has made the experience of using the site frustrating.

In addition to that, a good chunk of features are quite buggy. Sometimes when you create a post, it doesn't show up where it's supposed to, and you can't find it any other way than via a link. Or sometimes the page reloads and you lose all the things that you've written because we don't automatically save new posts and comments as drafts.

Here is a quick story of how we got to this state:

Our top priority until now was to build an MVP that mostly has feature parity with reddit (so we can port over the data from the old site)
We wanted to have a modular codebase that allows us to extend and change that functionality easily
This means our development so far had been focused on developing features and getting the basic functionality going, not on making everything smooth and bug-free

And while it is tempting to continue to focus more on feature improvements and make the site more complete, I think it's time for LW 2.0 to focus its development on stability and speed. And only as soon as that has reached a high level, start expanding the feature set of the platform again.

This means, for the next few weeks, I and the rest of the team will do much less development on new features and focus much more on speed and the core user experience of a small selected set of features.

This might mean we temporarily turn off features on LW 2.0, if that is necessary to create a stable and pleasant experien...
Personal thoughts on careers in AI policy and strategy [x-post EA Forum]
by crmflynn — 2017-09-27T17:09:34.496Z

Summary:

The AI strategy space is currently bottlenecked by entangled and under-defined research questions that are extremely difficult to resolve, as well as by a lack of current institutional capacity to absorb and utilize new researchers effectively.

Accordingly, there is very strong demand for people who are good at this type of “disentanglement” research and well-suited to conduct it somewhat independently. There is also demand for some specific types of expertise which can help advance AI strategy and policy. Advancing this research even a little bit can have massive multiplicative effects by opening up large areas of work for many more researchers and implementers to pursue.

Until the AI strategy research bottleneck clears, many areas of concrete policy research and policy implementation are necessarily on hold. Accordingly, a large majority of people interested in this cause area, even extremely talented people, will find it difficult to contribute directly, at least in the near term.

If you are in this group whose talents and expertise are outside of these narrow areas, and want to contribute to AI strategy, I recommend you build up your capacity and try to put yourself in an influential position. This will set you up well to guide high-value policy interventions as clearer policy directions emerge. Try not to be discouraged or dissuaded from pursuing this area by the current low capacity to directly utilize your talent! The level of talent across a huge breadth of important areas I have seen from the EA community in my role at FHI is astounding and humbling.

Depending on how slow these “entangled” research questions are to unjam, and on the timelines of AI development, there might be a very narrow window of time in which it will be necessary to have a massive, sophisticated mobilization of altruistic talent. This makes being prepared to mobilize effectively and take impactful action on short notice extremely valuable in expectation.

In addition to strategy research, operations work in this space is currently highly in demand. Experienced managers and administrators are especially needed. More junior operations roles might also serve as a good orientation period for EAs who would like to take some time after college before either pursuing graduate school or a specific career in this space. This can be a great way

...
The Great Filter isn't magic either
by Stuart_Armstrong — 2017-09-27T17:04:56.708Z

A post suggested by James Miller's presentation at the Existential Risk to Humanity conference in Gothenburg.

Seeing the emptiness of the night sky, we can dwell upon the Fermi paradox: where are all the alien civilizations that simple probability estimates imply we should be seeing?

Especially given the ease of moving within and between galaxies, the cosmic emptiness implies a Great Filter: something that prevents planets from giving birth to star-spanning civilizations. One worrying possibility is the likelihood that advanced civilizations end up destroying themselves before they reach the stars.

The Great Filter as an Outside View

In a sense, the Great Filter can be seen as an ultimate example of the Outside View: we might have all the data and estimation we believe we would ever need from our models, but if those models predict that the galaxy should be teeming with visible life, then it doesn't matter how reliable our models seem: they must be wrong.

In particular, if you fear a late great filter - if you fear that civilizations are likely to destroy themselves - then you should increase your fear, even if "objectively" everything seems to be going all right. After all, presumably the other civilizations that destroyed themselves thought everything seemed to going all right. Then you can adjust your actions using your knowledge of the great filter - but presumably other civilizations also thought of the great filter and adjusted their own actions as well, but that didn't save them, so maybe you need to try something different again or maybe you can do something that breaks the symmetry from the timeless decision theory perspective like send a massive signal to the galaxy...

The Great Filter isn't magic

It can all get very headache-inducing. But, just as the Outside View isn't magic, the Great Filter isn't magic either. If advanced civilizations destroy themselves before becoming space-faring or leaving an imprint on the galaxy, then there is some phenomena that is the cause of this. What can we say, if we look analytically at the great filter argument?

First of all suppose we had three theories - early great filter (technological civilizations are rare), late great filter (technological civilizations destroy themselves before becoming space-faring), or no great filter. Then we look up at the empty skies, and notice

...
The Great Filter isn't magic either
by Stuart_Armstrong — 2017-09-27T16:56:39.642Z

Crossposted at Less Wrong 2.0. A post suggested by James Miller's presentation at the Existential Risk to Humanity conference in Gothenburg.

Seeing the emptiness of the night sky, we can dwell upon the Fermi paradox: where are all the alien civilizations that simple probability estimates imply we should be seeing?

Especially given the ease of moving within and between galaxies, the cosmic emptiness implies a Great Filter: something that prevents planets from giving birth to star-spanning civilizations. One worrying possibility is the likelihood that advanced civilizations end up destroying themselves before they reach the stars.

The Great Filter as an Outside View

In a sense, the Great Filter can be seen as an ultimate example of the Outside View: we might have all the data and estimation we believe we would ever need from our models, but if those models predict that the galaxy should be teeming with visible life, then it doesn't matter how reliable our models seem: they must be wrong.

In particular, if you fear a late great filter - if you fear that civilizations are likely to destroy themselves - then you should increase your fear, even if "objectively" everything seems to be going all right. After all, presumably the other civilizations that destroyed themselves thought everything seemed to going all right. Then you can adjust your actions using your knowledge of the great filter - but presumably other civilizations also thought of the great filter and adjusted their own actions as well, but that didn't save them, so maybe you need to try something different again or maybe you can do something that breaks the symmetry from the timeless decision theory perspective like send a massive signal to the galaxy...

The Great Filter isn't magic

It can all get very headache-inducing. But, just as the Outside View isn't magic, the Great Filter isn't magic either. If advanced civilizations destroy themselves before becoming space-faring or leaving an imprint on the galaxy, then there is some phenomena that is the cause of this. What can we say, if we look analytically at the great filter argument?

First of all suppose we had three theories - early great filter (technological civilizations are rare), late great filter (technological civilizations destroy themselves before becoming space-faring), or no great filter....
The Outside View isn't magic
by Stuart_Armstrong — 2017-09-27T14:37:16.881Z

Crossposted at Less Wrong 2.0.

The planning fallacy is an almost perfect example of the strength of using the outside view. When asked to predict the time taken for a project that they are involved in, people tend to underestimate the time needed (in fact, they tend to predict as if question was how long things would take if everything went perfectly).

Simply telling people about the planning fallacy doesn't seem to make it go away. So the outside view argument is that you need to put your project into the "reference class" of other projects, and expect time overruns as compared to your usual, "inside view" estimates (which focus on the details you know about the project.

So, for the outside view, what is the best way of estimating the time of a project? Well, to find the right reference class for it: the right category of projects to compare it with. You can compare the project with others that have similar features - number of people, budget, objective desired, incentive structure, inside view estimate of time taken etc... - and then derive a time estimate for the project that way.

That's the outside view. But to me, it looks a lot like... induction. In fact, it looks a lot like the elements of a linear (or non-linear) regression. We can put those features (at least the quantifiable ones) into a linear regression with a lot of data about projects, shake it all about, and come up with regression coefficients.

At that point, we are left with a decent project timeline prediction model, and another example of human bias. The fact that humans often perform badly in prediction tasks is not exactly new - see for instance my short review on the academic research on expertise.

So what exactly is the outside view doing in all this?

The role of the outside view: model incomplete and bias human

The main use of the outside view, for humans, seems to be to point out either an incompleteness in the model or a human bias. The planning fallacy has both of these: if you did a linear regression comparing your project with all projects with similar features, you'd notice your inside estimate was more optimistic than the regression - your inside model is incomplete. And if you also compared each person's initial estimate with the ultimate duration of their project, you'd notice a systematically optimistic bias - you'd notice the planning fallacy.

The first type of err...