trentmkelly/LessWrong-43k huggingface dataset:
Too good to be true
by PhilGoetz — 2014-07-11T20:16:24.277Z

A friend recently posted a link on his Facebook page to an informational graphic about the alleged link between the MMR vaccine and autism. It said, if I recall correctly, that out of 60 studies on the matter, not one had indicated a link.

Presumably, with 95% confidence.

This bothered me. What are the odds, supposing there is no link between X and Y, of conducting 60 studies of the matter, and of all 60 concluding, with 95% confidence, that there is no link between X and Y?

Answer: .95 ^ 60 = .046. (Use the first term of the binomial distribution.)

So if it were in fact true that 60 out of 60 studies failed to find a link between vaccines and autism at 95% confidence, this would prove, with 95% confidence, that studies in the literature are biased against finding a link between vaccines and autism.

In reality, you should adjust your literature survey for known biases of literature. Scientific literature has publication bias, so that positive results are more likely to be reported than negative results.

They also have a bias from errors. Many articles have some fatal flaw that makes their results meaningless. If the distribution of errors is random, I think--though I'm not sure--that we should assume this bias causes regression towards an equal likelihood of positive and negative results.

Given that both of these biases should result, in this case, in more positive results, having all 60 studies agree is even more incredible.

So I did a quick mini-review this morning, looking over all of the studies cited in 6 reviews on the results of studies on whether there is a connection between vaccines and autism:

National Academies Press (2004). Immunization safety review: Vaccines and autism.

National Academies Press (2011). Adverse effects of vaccines: Evidence and causality.

American Academy of Pedatricians (2013): Vaccine safety studies.

The current AAP webpage on vaccine safety studies.

The Immunization Action Coalition: Examine the evidence.

Taylor et al. (2014). Vaccines are not associated with autism: an evidence-based meta-analysis of case-control and cohort studies. Vaccine Jun 17;32(29):3623-9. Paywalled, but references given here.

I listed all of the studies that were judged usable in at least one of these reviews, removed duplicates, then went through them all and determined, either from the review article or from the study's abstract, what it concluded. There w...
[Question] Adoption and twin studies confounders
by Stuart_Armstrong — 2014-07-11T16:44:31.765Z

Adoption and twin studies are very important for determining the impact of genes versus environment in the modern world (and hence the likely impact of various interventions). Other types of studies tend to show larger effects for some types of latter interventions, but these studies are seen as dubious, as they may fail to adjust for various confounders (eg families with more books also have more educated parents).

But adoption studies have their own confounders. The biggest ones are that in many countries, the genetic parents have a role in choosing the adoptive parents. Add the fact that adoptive parents also choose their adopted children, and that various social workers and others have great influence over the process, this would seem a huge confounder interfering with the results.

This paper also mentions a confounder for some types of twin studies, such as identical versus fraternal twins. They point out that identical twins in the same family will typically get a much greater shared environment than fraternal twins, because people will treat them much more similarly. This is to my mind quite a weak point, but it is an issue nonetheless.

Since I have very little expertise in these areas, I was just wondering if anyone knew about efforts to estimate the impact of these confounders and adjust for them.
Weekly LW Meetups
by FrankAdamek — 2014-07-11T16:00:01.049Z

This summary was posted to LW Main on July 4th, the following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

Houston, TX: 12 July 2014 02:00PM

Upper Canada LW Megameetup: Ottawa, Toronto, Montreal, Waterloo, London: 18 July 2014 07:00PM

Moscow meet up: 06 July 2014 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Canberra: Paranoid Debating: 12 July 2014 06:00PM
[Melbourne] July Rationality Dojo: Disagreement: 06 July 2014 03:00PM

Sydney Rationality Dojo - Agency: 06 July 2014 04:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Mountain View, New York, Philadelphia, Research Triangle NC, Salt Lake City, Seattle, Sydney, Toronto, Vienna, Washington DC, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Cincinnati, Cleveland, Frankfurt, Helsinki, Marin CA, Ottawa, Pittsburgh, Portland, Southern California (Los Angeles/Orange County area), St. Louis, Tel Aviv, Vancouver.

Whether or not there's currently a meetup in your area, you can sign up to be notified automatically of any future meetups. And...
[QUESTION]: Driverless car forecasts
by VipulNaik — 2014-07-11T00:25:35.644Z

Of the technologies that have a reasonable chance of come to mass market in the next 20-25 years and having a significant impact on human society, driverless cars (also known as self-driving cars or autonomous cars) stand out. I was originally planning to collect material discussing driverless cars, but Gwern has a really excellent compendium of statements about driverless cars, published January 2013 (if you're reading this, Gwern, thanks!). There have been a few developments since then (for instance, Google's announcement that it was building its own driverless car, or a startup called Cruise Automation planning to build a $10,000 driverless car) but the overall landscape remains similar. There's been some progress with understanding and navigating city streets and with handling adverse weather conditions, and it's more or less on schedule.

My question is about driverless car forecasts. Driverless Future has a good summary page of forecasts made by automobile manufacturer, insurers, and professional societies. The range of time for the arrival of the first commercial driverless cars varies between 2018 and 2030. The timeline for driverless cars to achieve mass penetration is similarly stagged between the early 2020s and 2040. (The forecasts aren't all directly comparable).

A few thoughts come to mind:

Insurer societies and professional societies seem more conservative in their estimates than manufacturers (both automobile manufacturers and people manufacturing the technology for driverless cars). Note that the estimates of many manufacturers are centered on their projected release dates for their own driverless cars. This suggests an obvious conflict of interest: manufacturers may be incentivized to be optimistic in their projections of when driverless cars will be released, insofar as making more optimistic predictions wins them news coverage and might also improve their market valuation. (At the same time, the release dates are sufficiently far in the future that it's unlikely that they'll be held to account for false projections, so there isn't a strong incentive to be conservative the same way as there is with quarterly sales and earning forecasts). Overall, then, I'd defer more to the judgment of the professional societies, namely the IEEE and the Society of Autonomous Engineers.

The statements compiled by Gwern point to the many legal hurdles and ot

...
Meetup : Israel Less Wrong Meetup - Social Topics and LW outreach
by Anatoly_Vorobey — 2014-07-10T10:26:07.025Z

Discussion article for the meetup : Israel Less Wrong Meetup - Social Topics and LW outreach

WHEN:
10 July 2014 07:00:00PM (+0300)

WHERE:
Google Tel-Aviv

We're going to have a meetup on Thursday, July 10th at Google Israel's offices, Electra Tower, 98 Yigal Alon st., Tel Aviv.

We're having a social meetup. We'll discuss social issues and problems in a free-form discussion. Everyone will have a chance (if they want to!) to tell a story about their social life and issues, ask for feedback or advice, offer hard-won or easy-inherited wisdom...

We will also discuss LW outreach in Israel.

We'll start the meetup at 19:00, and we'll go on as much as we like to. Feel free to come a little bit later, as there is no agenda.

We'll meet at the 29th floor of the building (Note: Not the 26th where Google Campus is). If you arrive and can't find your way around, call Anatoly who's hosting us at 054-245-1060. Email at avorobey@gmail.com also works.

Things that might happen: - You'll trade cool ideas with cool people from the Israel LW community. - You'll discover kindred spirits who agree with you about one/two boxing. - You'll kick someone's ass (and teach them how you did it) at some awesome boardgame. - You'll discover how to build a friendly AGI running on cold fusion (well probably not) - You'll discuss interesting AI topics with new friends!

Things that will happen for sure: - You'll get to hang out with awesome people and have fun!

See you there!

Discussion article for the meetup : Israel Less Wrong Meetup - Social Topics and LW outreach
Meetup : Washington, D.C.:  Prisoner's Dilemna tournament
by Markas — 2014-07-09T18:40:10.676Z

Discussion article for the meetup : Washington, D.C.: Prisoner's Dilemna tournament

WHEN:
13 July 2014 03:00:00PM (-0400)

WHERE:
National Portrait Gallery, Washington, DC 20001, USA

We'll be meeting in the Kogod Courtyard of the National Portrait Gallery for our second Prisoner's Dilemna tournament. We'll play several different versions of iterated and one-shot games; feel free to suggest your own formats. There will be prizes (baked goods).

Discussion article for the meetup : Washington, D.C.: Prisoner's Dilemna tournament
[link] [poll] Future Progress in Artificial Intelligence
by Pablo — 2014-07-09T13:51:49.243Z

Vincent Müller and Nick Bostrom have just released a paper surveying the results of a poll of experts about future progress in artificial intelligence. The authors have also put up a companion site where visitors can take the poll and see the raw data. I just checked the site and so far only one individual has submitted a response. This provides an opportunity for testing the views of LW members against those of experts. So if you are willing to complete the questionnaire, please do so before reading the paper. (I have abstained from providing a link to the pdf to create a trivial inconvenience for those who cannot resist temptaion. Once you take the poll, you can easily find the paper by conducting a Google search with the keywords: bostrom muller future progress artificial intelligence.)
Domains of forecasting
by VipulNaik — 2014-07-09T13:45:50.501Z

Note: This post is part of my series on forecasting for MIRI. I recommend reading my earlier post on the general-purpose forecasting community, my post on scenario planning, and my post on futures studies. Although this post doesn't rely on those, they do complement each other.

Note 2: If I run across more domains where I have substantive things to say, I'll add them to this post (if I've got a lot to say, I'll write a separate post and add a link to it as well). Suggestions for other domains worth looking into, that I've missed below, would be appreciated.

Below, I list some examples of domains where forecasting is commonly used. In the post, I briefly describe each of the domains, linking to other posts of mine, or external sources, for more information. The list is not intended to be comprehensive. It's just the domains that I investigated at least somewhat and therefore have something to write about.

Weather and climate forecasting

Agriculture, crop simulation

Business forecasting, including demand, supply, and price forecasting

Macroeconomic forecasting

Political and geopolitical forecasting: This includes forecasting of election results, public opinion on issues, armed conflicts or political violence, and legislative changes

Demographic forecasting, including forecasting of population, age structure, births, deaths, and migration flows.

Energy use forecasting (demand forecasting, price forecasting, and supply forecasting, including forecasting of conventional and alternative energy sources; borrows some general ideas from business forecasting)

Technology forecasting

Let's look into these in somewhat more detail.

Note that for some domains, scenario planning may be more commonly used than forecasting in the traditional sense. Some domains have historically been more closely associated with machine learning, data science, and predictive analytics techniques (this is usually the case when a large number of explanatory variables are available). Some domains have been more closely associated with futures studies, that I discussed here. I've included the relevant observations for individual domains where applicable.

Climate and weather forecasting

More details are in my posts on weather forecasting and weather and climate forecasting, but here are a few main points:

The best weather forecasting methods use physical models rather than statistical models (though

...
Confound it! Correlation is (usually) not causation! But why not?
by gwern — 2014-07-09T03:04:26.084Z

It is widely understood that statistical correlation between two variables ≠ causation. But despite this admonition, people are routinely overconfident in claiming correlations to support particular causal interpretations and are surprised by the results of randomized experiments, suggesting that they are biased & systematically underestimating the prevalence of confounds/common-causation. I speculate that in realistic causal networks or DAGs, the number of possible correlations grows faster than the number of possible causal relationships. So confounds really are that common, and since people do not think in DAGs, the imbalance also explains overconfidence.

Full article: http://www.gwern.net/Causality
The insularity critique of climate science
by VipulNaik — 2014-07-09T01:17:03.466Z

Note: Please see this post of mine for more on the project, my sources, and potential sources for bias.

One of the categories of critique that have been leveled against climate science is the critique of insularity. Broadly, it is claimed that the type of work that climate scientists are trying to do draws upon insight and expertise in many other domains, but climate scientists have historically failed to consult experts in those domains or even to follow well-documented best practices.

Some takeaways/conclusions

Note: I wrote a preliminary version of this before drafting the post, but after having done most of the relevant investigation. I reviewed and edited it prior to publication. Note also that I don't justify these takeaways explicitly in my later discussion, because a lot of these come from general intuitions of mine and it's hard to articulate how the information I received explicitly affected my reaching the takeaways. I might discuss the rationales behind these takeaways more in a later post.

Many of the criticisms are broadly on the mark: climate scientists should have consulted best practices in other domains, and in general should have either followed them or clearly explained the reasons for divergence.

However, this criticism is not unique to climate science: academia in general has suffered from problems of disciplines being relatively insular (UPDATE: Here's Robin Hanson saying something similar). And many similar things may be true, albeit in different ways, outside academia.

One interesting possibility is that bad practices here operate via founder effects: for an area that starts off as relatively obscure and unimportant, setting up good practices may not be considered important. But as the area grows in importance, it is quite rare for the area to be cleaned up. People and institutions get used to the old ways of doing things. They have too much at stake to make reforms. This does suggest that it's important to get things right early on.

(This is speculative, and not discussed in the post): The extent of insularity of a discipline seems to be an area where a few researchers can have significant effect on the discipline. If a few reasonably influential climate scientists had pushed for more integration with and understanding of ideas from other disciplines, the history of climate science research would have been different.

Relevant domains...
Meetup : Brussels - August (topic TBD)
by Roxolan — 2014-07-09T00:26:14.613Z

Discussion article for the meetup : Brussels - August (topic TBD)

WHEN:
09 August 2014 01:00:00PM (+0200)

WHERE:
Rue des Alexiens 55 1000 Bruxelles

[This post will be updated when we have decided what the meetup topic will be, if any.]

We will meet at 1 pm at "La Fleur en papier doré, close to the Brussels Central station. The meeting will be in English to facilitate both French and Dutch speaking members.

If you are coming for the first time, please consider filling out this one minute form to share your contact information.

The Brussels meetup group communicates through a Google Group.

Meetup announcements are also mirrored on meetup.com

Discussion article for the meetup : Brussels - August (topic TBD)
Meetup : Brussels - July meetup
by Roxolan — 2014-07-09T00:23:36.883Z

Discussion article for the meetup : Brussels - July meetup

WHEN:
12 July 2014 01:00:00PM (+0200)

WHERE:
Rue des Alexiens 55 1000 Bruxelles

No theme has been chosen this month.
We will meet at 1 pm at "La Fleur en papier doré", close to the Brussels Central station. The meeting will be in English to facilitate both French and Dutch speaking members.
If you are coming for the first time, please consider filling out this one minute form to share your contact information.
The Brussels meetup group communicates through a Google Group. Meetup announcements are also mirrored on meetup.com

Discussion article for the meetup : Brussels - July meetup
Effective Altruism Summit is One Month Away
by Nevin — 2014-07-08T23:39:52.611Z

This is a followup to Ben's post announcing the 2014 EA Summit.

The Effective Altruism Summit is now exactly one month away.

This year, 175 EAs will gather in Berkeley, CA for a two-day conference -- the largest gathering of people in the EA movement to date.

We still have spots left, and we are especially interested in having people who are new to the movement and aren't yet working on something related to EA full-time. The event is a great place to meet everyone who is serious about EA, learn a whole lot about the different projects people are working on, build friendships, and start collaborations.

Here is a short video of what last year's Summit was like.

There will be people there from:

The Center for Applied Rationality

GiveWell

GiveDirectly

The Machine Intelligence Research Institute

The Future of Humanity Institute

The Life You Can Save

80,000 Hours

Giving What We Can

Leverage Research

-- and others.

The event costs $600, but we can offer discounted tickets to people who can't pay full price. If you are interested in coming but money is a barrier, please don't be shy -- let us know through the form on the website and we will do everything we can to get you a spot. If you can pay full price, you'll be helping to cover other EAs. None of the organizations involved will profit from the event.

You can get more info and register to attend by filling out the form on the summit website.

The Retreat, mentioned in Ben's previous post, is full. If you note interest in the retreat on your Summit registration, we'll let you know if any space opens up last-minute.

Questions? Email effectivealtruismsummit@gmail.com for fastest response, or post in this thread for public response.
Change Contexts to Improve Arguments
by palladias — 2014-07-08T15:51:43.204Z

On a recent trip to Ireland, I gave a talk on tactics for having better arguments (video here). There's plenty in the video that's been discussed on LW before (Ideological Turing Tests and other reframes), but I thought I'd highlight one other class of trick I use to have more fruitful disagreements.

It's hard, in the middle of a fight, to remember, recognize, and defuse common biases, rhetorical tricks, emotional triggers, etc. I'd rather cheat than solve a hard problem, so I put a lot of effort into shifting disagreements into environments where it's easier for me and my opposite-number to reason and argue well, instead of relying on willpower. Here's a recent example of the kind of shift I like to make:

A couple months ago, a group of my friends were fighting about the Brendan Eich resignation on facebook. The posts were showing up fast; everyone was, presumably, on the edge of their seats, fueled by adrenaline, and alone at their various computers. It’s a hard place to have a charitable, thoughtful debate.

I asked my friends (since they were mostly DC based) if they’d be amenable to pausing the conversation and picking it up in person. I wanted to make the conversation happen in person, not in front of an audience, and in a format that let people speak for longer and ask questions more easily. If so, I promised to bake cookies for the ultimate donnybrook.

My friends probably figured that I offered cookies as a bribe to get everyone to change venues, and they were partially right. But my cookies had another strategic purpose. When everyone arrived, I was still in the process of taking the cookies out of the oven, so I had to recruit everyone to help me out.

“Alice, can you pour milk for people?”

“Bob, could you pass out napkins?”

“Eve, can you greet people at the door while I’m stuck in the kitchen with potholders on?”

Before we could start arguing, people on both sides of the debate were working on taking care of each other and asking each others’ help. Then, once the logistics were set, we all broke bread (sorta) with each other and had a shared, pleasurable experience. Then we laid into each other.

Sharing a communal experience of mutual service didn’t make anyone pull their intellectual punches, but I think it made us more patient with each other and l...