trentmkelly/LessWrong-43k huggingface dataset:
Are conferences an inefficient/terrible discussion forum (in addition to academic papers)?
by InquilineKea — 2015-06-04T19:14:03.165Z

I'm asking this as a follow-up to http://lesswrong.com/lw/d5y/why_academic_papers_are_a_terrible_discussion/, which was written a few years ago, and which I find very interesting.

Many of the arguments advanced in http://lesswrong.com/lw/d5y/why_academic_papers_are_a_terrible_discussion/ (especially inaccessibility) could just as well apply to conferences, too.

I'd also wonder - would you consider conferences to also be a terrible discussion forum? What do you think would be some good alternatives?

The audience for conferences is limited, and people seem to remember only a tiny tiny fraction of everything they've encountered in a conference. The ideas in conferences don't seem to do much for building up platforms of public discussions around the new subjects that are often announced in conferences (rather than, say, on online platforms).

I suppose one could advance the argument that ideas often get brought up/discussed at conferences that wouldn't be conveniently discussed in any other medium (for now..). But is this mostly because people are too comfortable with what they're been brought up with?
Meetup : Munich June Meetup
by jasticE — 2015-06-04T12:27:13.567Z

Discussion article for the meetup : Munich June Meetup

WHEN:
20 June 2015 03:00:51PM (+0200)

WHERE:
Café Puck, Türkenstraße 33, Munich

Please see the announcement on Meetup.com for details, discussion and exact location: http://www.meetup.com/LessWrongMunich/events/223011559/

Discussion article for the meetup : Munich June Meetup
Taking the reins at MIRI
by So8res — 2015-06-03T23:52:28.074Z

Hi all. In a few hours I'll be taking over as executive director at MIRI. The LessWrong community has played a key role in MIRI's history, and I hope to retain and build your support as (with more and more people joining the global conversation about long-term AI risks & benefits) MIRI moves towards the mainstream.

Below I've cross-posted my introductory post on the MIRI blog, which went live a few hours ago. The short version is: there are very exciting times ahead, and I'm honored to be here. Many of you already know me in person or through my blog posts, but for those of you who want to get to know me better, I'll be running an AMA on the effective altruism forum at 3PM Pacific on Thursday June 11th.

I extend to all of you my thanks and appreciation for the support that so many members of this community have given to MIRI throughout the years.

Hello, I'm Nate Soares, and I'm pleased to be taking the reins at MIRI on Monday morning.

For those who don't know me, I've been a research fellow at MIRI for a little over a year now. I attended my first MIRI workshop in December of 2013 while I was still working at Google, and was offered a job soon after. Over the last year, I wrote a dozen papers, half as primary author. Six of those papers were written for the MIRI technical agenda, which we compiled in preparation for the Puerto Rico conference put on by the FLI in January 2015. Our technical agenda is cited extensively in the research priorities document referenced by the open letter that came out of that conference. In addition to the Puerto Rico conference, I attended five other conferences over the course of the year, and gave a talk at three of them. I also put together the MIRI research guide (a resource for students interested in getting involved with AI alignment research), and of course I spent a fair bit of time doing the actual research at workshops, at researcher retreats, and on my own. It's been a jam-packed year, and it's been loads of fun.

I've always had a natural inclination towards leadership: in the past, I've led a F.I.R.S.T. Robotics team, managed two volunteer theaters, served as president of an Entrepreneur's Club, and co-founded a startup or two. However, this is the first time I've taken a professional leadership role, and I'm grateful that I'll be able to call upon the experience and expertise of the board, of our advisors, and of o...
Summary of my Participation in the Good Judgment Project
by Gunnar_Zarncke — 2015-06-03T21:51:07.821Z

Follow-Up to Good Judgment Project, Season Three.

During the last forecasting season I took part in the Good Judgment Project (GJP; see also the blog) and this is a short summary of my participation (actually triggered by hamnox comment).

The GJP estimates world events like

Ukraine conflict

Arctic ice cap melting

Ebola outbreak duration

Chinese sea conflict

ISIS attacks

Terrorist attacks

Oil price

Certain exchange rates

Election results

and many other political events

To participate in that study one has to register (can't remember where exactly I stumbled over the link, possibly the one at the top). And one has to do an preparatory online course and one has to pass an online test. At least I had to complete it. Whether the result affected my assignment to any group I can't say. The course explains the scoring and gives recommendations for making good forecasts (choose forecasts one has an edge in, estimate early, update often, do post-mortems). The test seems to test for calibration and accuracy by asking for known (mostly political) events and whether one is sure about them.

The current forecasting season started in November 2014 and has just ended. I invested significantly less then half an hour a week on 8 questions of about 100 (and thus less than I projected in an early questionaire). I did 2 to 15 updates for these questions and I earned a score in the middle range (mostly due to getting hit by an unexpected terrorist attack). As I just learned I was assigned to the study condition were I could neither see the total group estimate nor the estimates of the other group members - only their comments. I was somewhat disappointed by this as I had hoped to learn something from how the scores developed. Too bad I wasn't in a prediction marked group. But I hope to get the study results later.

I will not take part in further rounds as I shy the effort for the types of forecasts which are mostly political. They are political because the sponsor (guess who) is interested mostly in political events - less in economical, environmental, scientific or other types. But I enjoyed forecasting artic ice cap melting and ebola - and netted a better than average score on that.

The scoring - at least in this group - is interesting and uses an averaged Brier Score - averaged over a) all forecast questiontion and b) within a question over all the days f...
Simultaneous Overconfidence and Underconfidence
by abramdemski — 2015-06-03T21:04:01.038Z

Follow-up to this and this on my personal blog. Prep for this meetup. Cross-posted on my blog.

Eliezer talked about cognitive bias, statistical bias, and inductive bias in a series of posts only the first of which made it directly into the LessWrong sequences as currently organized (unless I've missed them!). Inductive bias helps us leap to the right conclusion from the evidence, if it captures good prior assumptions. Statistical bias can be good or bad, depending in part on the bias-variance trade-off. Cognitive bias refers only to obstacles which prevent us from thinking well.

Unfortunately, as we shall see, psychologists can be quite inconsistent about how cognitive bias is defined. This created a paradox in the history of cognitive bias research. One well-researched and highly experimentally validated effect was conservatism, the tendency to give estimates too middling, or probabilities too near 50%. This relates especially to integration of information: when given evidence relating to a situation, people tend not to take it fully into account, as if they are stuck with their prior. Another highly-validated effect was overconfidence, relating especially to calibration: when people give high subjective probabilities like 99%, they are typically wrong with much higher frequency.

In real-life situations, these two contradict: there is no clean distinction between information integration tasks and calibration tasks. A person's subjective probability is always, in some sense, the integration of the information they've been exposed to. In practice, then, when should we expect other people to be under- or over- confident?

Simultaneous Overconfidence and Underconfidence

The conflict was resolved in an excellent paper by Ido Ereve et al which showed that it's the result of how psychologists did their statistics. Essentially, one group of psychologists defined bias one way, and the other defined it another way. The results are not really contradictory; they are measuring different things. In fact, you can find underconfidence or overconfidence in the same data sets by applying the different statistical techniques; it has little or nothing to do with the differences between information integration tasks and probability calibration tasks. Here's my rough drawing of the phenomenon (apologies for my hand-drawn illustrations):

Overc

...
Perceptual Entropy and Frozen Estimates
by Davidmanheim — 2015-06-03T19:27:31.074Z

A Preface

During the 1990’s, a significant stream of research existed around how people process information, which combined very different streams in psychology and related areas with explicit predictive models about how actual cognitive processes differ from the theoretical ideal. This is not only the literature by Kahneman and Tversky about cognitive biases, but includes research about memory, perception, scope insensitivity, and other areas. The rationalist community is very familiar with some of this literature, but fewer are familiar with a masterful synthesis produced by Richards Heuer for the intelligence community in 1999[1], which was intended to start combating these problems, a goal we share. I’m hoping to put together a stream of posts based on that work, potentially expanding on it, or giving my own spin – but encourage reading the book, Psychology of Intelligence Analysis, itself (PDF) as well[2]. (This essay is based on Chapter 3.)

This will hopefully be my first set of posts, so feedback is especially welcome, both to help me refine the ideas, and to refine my presentation.

Entropy, Pressure, and Metaphorical States of Matter

Eliezer recommends updating incrementally but has noted that it’s hard. The central point, that it is hard to do so, is one that some in our community have experienced and explicated, but there is deep theory I’ll attempt to outline, via an analogy, that I think explains how and why it occurs. The problem is that we are quick to form opinions and build models, because humans are good at pattern finding. We are less quick to discard them, due to limited mental energy. This is especially true when the pressure of evidence doesn’t shift overwhelmingly and suddenly.

I’ll attempt to answer the question of how this is true by stretching a metaphor and create an intuition pump for thinking about how our minds might be perform some think using uncertainty.

Frozen Perception

Heuer notes a stream of research about perception, and notes that “once an observer has formed an image – that is, once he or she has developed a mind set or expectation concerning the phenomenon being observed – this conditions future perceptions of that phenomenon.” This seems to follow a standard Bayesian practice, but in fact, as Eliezer noted, people fail to update. The following set of images, which Heuer reproduced from a 1976 book by Robert Jervis, sh

...
FAI Research Constraints and AGI Side Effects
by JustinShovelain — 2015-06-03T19:25:14.508Z

Ozzie Gooen and Justin Shovelain

Summary

Friendly artificial intelligence (FAI) researchers have at least two significant challenges. First, they must produce a significant amount of FAI research in a short amount of time. Second, they must do so without producing enough general artificial intelligence (AGI) research to result in the creation of an unfriendly artificial intelligence (UFAI). We estimate the requirements of both of these challenges using two simple models.

Our first model describes a friendliness ratio and a leakage ratio for FAI research projects. These provide limits on the allowable amount of artificial general intelligence (AGI) knowledge produced per unit of FAI knowledge in order for a project to be net beneficial.

Our second model studies a hypothetical FAI venture, which is responsible for ensuring FAI creation. We estimate necessary total FAI research per year from the venture and leakage ratio of that research. This model demonstrates a trade off between the speed of FAI research and the proportion of AGI research that can be revealed as part of it. If FAI research takes too long, then the acceptable leakage ratio may become so low that it would become nearly impossible to safely produce any new research.

Introduction

A general artificial intelligence (AGI) is an AI that could perform all the intellectual tasks a human can.[1] When one is created, it may recursively become more intelligent to the point where it is vastly superior to human intelligences.[2] This AGI could be either friendly or unfriendly, where friendliness means it would have values that humans would favor, and unfriendliness means that it would not.[3]

It is likely that if we do not explicitly understand how to make a friendly general artificial intelligence, then by the time we make a general artificial intelligence, it will be unfriendly.[4] It is also likely that we are much further from understanding how to make a friendly artificial intelligence than we are from understanding how to make a general artificial intelligence.[5][6]

Thus, it is important to create more FAI research, but it may also be important to make sure to not produce much AGI research when doing so. If it is 10 times as difficult to understand how to make an FAI than to understand how to make an AGI, then a FAI research paper that produces 0.3 equivalent papers worth of AGI research will probably i...
A tractable, interpretable formulation of approximate conditioning for pairwise-specified probability distributions over truth values
by János Kramár — 2015-06-03T19:08:18.000Z

These results from my conversations with Charlie Steiner at the May 29-31 MIRI Workshop on Logical Uncertainty will primarily be of interest to people who've read section 2.4 of Paul Christiano's Non-Omniscience paper.

If we write a reasoner that keeps track of probabilities of a collection of sentences φ1,…,φn (that grows and shrinks as the reasoner explores), we need some way of tracking known relationships between the sentences. One way of doing this is to store the pairwise probability distributions, ie not only Pr(φi) for all i but also Pr(φi∧φj) for all i,j.

If we do this, a natural question to ask is: how can we update this data structure if we learn that eg φ1 is true?

We'll refer to the updated probabilities as Pr(⋅|φ1).

It's fairly reasonable for us to want to set Pr(φi|φ1):=Pr(φi∧φ1)/Pr(φ1); however, it's less clear what values to assign to Pr(φi∧φj|φ1), because we haven't stored Pr(φi∧φj∧φ1).

One option would be to find the maximum entropy distribution over truth assignments to φ1,…,φn under the constraint that the stored pairwise distributions are correct. This seems intractable for large n; however, in the spirit of locality, we could restrict our attention to the joint truth value distribution of φ1,φi,φj. Maximizing its entropy is simple (it boils down to either convex optimization or solving a cubic), and yields a plausible candidate for Pr(φi∧φj∧φ1) that we can derive Pr(φi∧φj|φ1) from. I'm not sure what global properties this has, for example whether it yields a positive semidefinite matrix (Pr(φi∧φj))i,j.

A different option, as noted in section 2.4.2, is to observe that the matrix (Pr(φi∧φj))i,j must be positive semidefinite under any joint distribution for the truth values. This means we can consider a zero-mean multivariate normal distribution with this matrix as its covariance; then there's a closed-form expression for the Kullback-Leibler divergence of two such distributions, and this can be used to define a sort of conditional distribution, as is done in section 2.4.3.

However, as the paper remarks, this isn't a very familiar way of defining these updated probabilities. For example, it lacks the desirable property that Pr(φi|φ1)=Pr(φi∧φ1)/Pr(φ1).

Fortunately, there is a natural construction that combines these ideas: namely, if we consider the maximum-entropy distribution for the truth assignment vector (1φ1,…,1φn) with the given second mome

...
In Defense of the Fundamental Attribution Error
by OrphanWilde — 2015-06-03T18:46:44.839Z

The Fundamental Attribution Error

Also known, more accurately, as "Correspondence Bias."

http://lesswrong.com/lw/hz/correspondence_bias/

The "more accurately" part is pretty important; bias -may- result in error, but need not -necessarily- do so, and in some cases may result in reduced error.

A Simple Example

Suppose I write a stupid article that makes no sense and rambles on without any coherent point. There might be a situational cause of this; maybe I'm tired. Correcting for correspondence bias means that more weight should be given to the situational explanation than the dispositional explanation, that I'm the sort of person who writes stupid articles that ramble on. The question becomes, however, whether or not this increases the accuracy of your assessment of me; does correcting for this bias make you, in fact, less wrong?

In this specific case, no, it doesn't. A person who belongs to the class of people who write stupid articles is more likely to write stupid articles than a person who doesn't belong to that class - I'd be surprised if I ever saw Gwern write anything that wasn't well-considered, well-structured, and well-cited. If somebody like Gwern or Eliezer wrote a really stupid article, we have sufficient evidence that he's not a member of that class of people to make that conclusion a poor one; the situational explanation is better, he's having some kind of off day. However, given an arbitrary stupid article written by somebody for which we have no prior information, the distribution is substantially different. We have different priors for "Randomly chosen person X writes article" and "Article is bad" implies "X is a bad writer of articles" than we do for "Well-known article author Y writes article" and "Article is bad" implies "Y is a bad writer of articles".

Getting to the Point

The FAE is putting emphasis on internal factors rather than external. It's jumping first to the conclusion that somebody who just swerved is a bad driver, rather than first considering the possibility that there was an object in the road they were avoiding, given only the evidence that they swerved. Whether or not the FAE is an error - whether it is more wrong - depends on whether or not the conclusion you jumped to was correct, and more importantly, whether, on average, that conclusion would be correct.

It's very ...
An Oracle standard trick
by Stuart_Armstrong — 2015-06-03T14:25:08.000Z

An idea I thought I'd been mentioning to everyone, but a recent conversation reveals I haven't been assiduous about it.

It's quite simple: whenever designing an Oracle, you should, as a default, run it's output channel through a probabilistic process akin to the false thermodynamic miracle, in order to make the Oracle act as if it believed its message will never be read.

This reduces the possibility of the Oracle manipulating us through message content, because it's action as if that content will never be seen by anyone.

Now, some Oracle designs can't use that (eg if accuracy is defined in terms of the reaction of people that read its output). But in general, if your design allows such a precaution, there's no reason not to put it on, so it should be default in the Oracle design.

Even if the Oracle design precludes this directly, some version of it can be often be used. For instance, if accuracy is defined in terms of the reaction of the first person to read the output, and that person is isolated from the rest of the world, then we can get the Oracle to act as if it believed a nuclear bomb was due to go off before the person could communicate with the rest of the world.
An Oracle standard trick
by Stuart_Armstrong — 2015-06-03T14:17:11.645Z

A putative new idea for AI control; index here.

EDIT: To remind everyone, this method does not entail the Oracle having false beliefs, just behaving as if it did; see here and here.

An idea I thought I'd been mentioning to everyone, but a recent conversation reveals I haven't been assiduous about it.

It's quite simple: whenever designing an Oracle, you should, as a default, run it's output channel through a probabilistic process akin to the false thermodynamic miracle, in order to make the Oracle act as if it believed its message will never be read.

This reduces the possibility of the Oracle manipulating us through message content, because it's action as if that content will never be seen by anyone.

Now, some Oracle designs can't use that (eg if accuracy is defined in terms of the reaction of people that read its output). But in general, if your design allows such a precaution, there's no reason not to put it on, so it should be default in the Oracle design.

Even if the Oracle design precludes this directly, some version of it can be often be used. For instance, if accuracy is defined in terms of the reaction of the first person to read the output, and that person is isolated from the rest of the world, then we can get the Oracle to act as if it believed a nuclear bomb was due to go off before the person could communicate with the rest of the world.
Systemic review of antidepressants vs placebo commentary
by michael_b — 2015-06-03T12:40:08.548Z

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4172306/pdf/zfp_222_3_128.pdf

The background on this story is a community of science people found a bunch of unpublished studies that, when weighed with the studies which supported antidepressant approval, showed they were no more effective than placebo in mild-moderate cases.

Except unlike placebo, antidepressants express a wider range of severe side effects, like worsening depression and suicide.

Isn't this a scandal? How do psychiatrists still prescribe these en masse?