# Is my view contrarian?

Previously: [Contrarian Excuses](http://www.overcomingbias.com/2009/11/contrarian-excuses.html), [The Correct Contrarian Cluster](/lw/1kh/the_correct_contrarian_cluster/), [What is bunk?](/lw/28i/what_is_bunk/), [Common Sense as a Prior](/lw/iao/common_sense_as_a_prior/), [Trusting Expert Consensus](/lw/iu0/trusting_expert_consensus/), [Prefer Contrarian Questions](http://www.overcomingbias.com/2014/03/prefer-contrarian-questions-vs-answers.html).

Robin Hanson once [wrote](http://www.overcomingbias.com/2009/11/contrarian-excuses.html):

> On average, contrarian views are less accurate than standard views. Honest contrarians should admit this, that neutral outsiders should assign most contrarian views a lower probability than standard views, though perhaps a high enough probability to warrant further investigation. Honest contrarians who expect reasonable outsiders to give their contrarian view more than normal credence should point to strong outside indicators that correlate enough with contrarians tending more to be right.

I tend to think through the issue in three stages:

1.  When should I consider myself to be holding a contrarian^[\[1\]](#fn:1 "see footnote")^ view? What is the relevant expert community?
2.  If I seem to hold a contrarian view, when do _I_ have enough reason to think I’m correct?
3.  If I seem to hold a _correct_ contrarian view, what can I do to give _other_ people good reasons to accept my view, or at least to take it seriously enough to examine it at length?

I don’t yet feel that I have “answers” to these questions, but in this post (and hopefully some future posts) I’d like to organize some of what has been said before,^[\[2\]](#fn:2 "see footnote")^ and push things a bit further along, in the hope that further discussion and inquiry will contribute toward significant progress in [social epistemology](http://plato.stanford.edu/entries/epistemology-social/).^[\[3\]](#fn:3 "see footnote")^ Basically, I hope to say a bunch of obvious things, in a relatively well-organized fashion, so that less obvious things can be said from there.^[\[4\]](#fn:4 "see footnote")^

In this post, I’ll just address stage 1. Hopefully I’ll have time to revisit stages 2 and 3 in future posts.

#### Is my view contrarian?

##### World model differences vs. value differences

Is my [effective altruism](/lw/hx4/four_focus_areas_of_effective_altruism/) a contrarian view? It seems to be more of a contrarian _value judgment_ than a contrarian _world model_,^[\[5\]](#fn:5 "see footnote")^ and by “contrarian view” I tend to mean “contrarian world model.” Some apparently contrarian views are probably actually contrarian _values_.

##### Expert consensus

Is my [atheism](http://commonsenseatheism.com/) a contrarian view? It’s definitely a world model, not a value judgment, and [only 2% of people are atheists](https://www.cia.gov/library/publications/the-world-factbook/fields/2122.html#xx).

But what’s the relevant _expert_ population, here? Suppose it’s “academics who specialize in the arguments and evidence concerning whether a god or gods exist.” If so, then the expert population is probably dominated by academic theologians and religious philosophers, and my atheism is a contrarian view.

We need [some heuristics](/lw/4ba/some_heuristics_for_evaluating_the_soundness_of/) for evaluating the soundness of the academic consensus in different fields. ^[\[6\]](#fn:6 "see footnote")^

For example, we should consider the selection effects operating on communities of experts. If someone doesn’t believe in God, they’re unlikely to spend their career studying arcane arguments for and against God’s existence. So most people who specialize in this topic are theists, but nearly all of them were theists _before_ they knew the arguments.

Perhaps instead the relevant expert community is “scholars who study the fundamental nature of the universe” — maybe, philosophers and physicists? They’re mostly atheists. ^[\[7\]](#fn:7 "see footnote")^ This is starting to get pretty ad-hoc, but maybe that’s unavoidable.

What about my view that the overall long-term impact of [AGI](http://intelligence.org/2013/08/11/what-is-agi/) will be, most likely, extremely bad? A recent survey of the top 100 authors in artificial intelligence (by citation index)^[\[8\]](#fn:8 "see footnote")^ suggests that my view is somewhat out of sync with the views of those researchers.^[\[9\]](#fn:9 "see footnote")^ But is that the relevant expert population? My impression is that AI experts know a lot about contemporary AI methods, especially within their subfield, but usually haven’t thought much about, or read much about, long-term AI impacts.

Instead, perhaps I’d need to survey “[AGI impact experts](http://intelligence.org/2013/05/01/agi-impacts-experts-and-friendly-ai-experts/)” to tell whether my view is contrarian. But who is that, exactly? There’s no standard credential.

Moreover, the most plausible candidates around today for “AGI impact experts” are — like the “experts” of many other fields — mere “scholastic experts,” in that they^[\[10\]](#fn:10 "see footnote")^ know a lot about the arguments and evidence typically brought to bear on questions of long-term AI outcomes.^[\[11\]](#fn:11 "see footnote")^ They generally are _not_ experts in the sense of “[Reliably superior performance on representative tasks](http://commonsenseatheism.com/wp-content/uploads/2012/12/Ericsson-An-Introduction-to-Cambridge-Handbook-of-Expertise-and-Expert-Performance.pdf)” — they don’t have uniquely good track records on predicting long-term AI outcomes, for example. As far as I know, they don’t even have uniquely good track records on predicting short-term geopolitical or sci-tech outcomes — e.g. they aren’t among the “[super forecasters](http://www.edge.org/conversation/how-to-win-at-forecasting)” discovered in [IARPA’s forecasting tournaments](http://www.iarpa.gov/Programs/ia/ACE/ace.html).

Furthermore, we might start to worry about selection effects, again. E.g. if we ask AGI experts when they think AGI will be built, they may be overly optimistic about the timeline: after all, if they didn’t think AGI was feasible soon, they probably wouldn’t be focusing their careers on it.

Perhaps we can salvage this approach for determining whether one has a contrarian view, but for now, let’s consider another proposal.

##### Mildly extrapolated elite opinion

Nick Beckstead instead [suggests](/lw/iao/common_sense_as_a_prior/) that, at least as a strong prior, one should believe what one thinks “a broad coalition of trustworthy people would believe if they were trying to have accurate views and they had access to \[one’s own\] evidence.”^[\[12\]](#fn:12 "see footnote")^ Below, I’ll propose a modification of Beckstead’s approach which aims to address the “Is my view contrarian?” question, and I’ll call it the “mildly extrapolated elite opinion” (MEEO) method for determining the relevant expert population. ^[\[13\]](#fn:13 "see footnote")^

First: which people are “trustworthy”? With Beckstead, I favor “giving more weight to the opinions of people who can be shown to be trustworthy by clear indicators that many people would accept, rather than people that seem trustworthy to you personally.” (This guideline aims to avoid parochialism and self-serving cognitive biases.)

What are some “clear indicators that many people would accept”? Beckstead suggests:

> IQ, business success, academic success, generally respected scientific or other intellectual achievements, wide acceptance as an intellectual authority by certain groups of people, or success in any area where there is intense competition and success is a function of ability to make accurate predictions and good decisions…

> Of course, trustworthiness can also be domain-specific. Very often, elite common sense would recommend deferring to the opinions of experts (e.g., listening to what physicists say about physics, what biologists say about biology, and what doctors say about medicine). In other cases, elite common sense may give partial weight to what putative experts say without accepting it all (e.g. economics and psychology). In other cases, they may give less weight to what putative experts say (e.g. sociology and philosophy).

Hence MEEO outsources the challenge of evaluating academic consensus in different fields to the “generally trustworthy people.” But in doing so, it raises several new challenges. How do we determine which people are trustworthy? How do we “mildly extrapolate” their opinions? How do we weight those mildly extrapolated opinions in combination?

This approach might also be promising, or it might be even harder to use than the “expert consensus” method.

#### My approach

In practice, I tend to do something like this:

*   To determine whether my view is contrarian, I ask whether there’s a fairly obvious, relatively trustworthy expert population on the issue. If there is, I try to figure out what their consensus on the matter is. If it’s different than my view, I conclude I have a contrarian view.
*   If there _isn’t_ an obvious trustworthy expert population on the issue from which to extract a consensus view, then I basically give up on step 1 (“Is my view contrarian?”) and just move to the model combination in step 2 (see below), retaining pretty large uncertainty about how contrarian my view might be.

####   

#### When do I have good reason to think I’m correct?

Suppose I conclude I have a contrarian view, as I plausibly have about long-term AGI outcomes,^[\[14\]](#fn:14 "see footnote")^ and as I might have about the technological feasibility of preserving myself via cryonics.^[\[15\]](#fn:15 "see footnote")^ How much evidence do I need to conclude that my view is justified despite the informed disagreement of others?

I’ll try to tackle that question in a future post. Not surprisingly, my approach is a kind of [model combination and adjustment](/lw/hzu/model_combination_and_adjustment/).

* * *

*   I don’t have a concise definition for what counts as a “contrarian view.” In any case, I don’t think that searching for an exact definition of “contrarian view” is what matters. In an email conversation with me, Holden Karnofsky concurred, making the point this way: “I agree with you that the idea of ‘contrarianism’ is tricky to define. I think things get a bit easier when you start looking for patterns that should worry you rather than trying to Platonically define contrarianism… I find ‘Most smart people think I’m bonkers about X’ and ‘Most people who have studied X more than I have _plus seem to generally think like I do_ think I’m wrong about X’ both worrying; I find ‘Most smart people think I’m wrong about X’ and ‘Most people who spend their lives studying X within a system that seems to be clearly dysfunctional and to have a bad track record think I’m bonkers about X’ to be less worrying.” [ ↩](#fnref:1 "return to article")
    
*   For a diverse set of perspectives on the social epistemology of disagreement and contrarianism not influenced (as far as I know) by the Overcoming Bias and Less Wrong conversations about the topic, see [Christensen (2009)](http://fitelson.org/seminar/christensen.pdf); [Ericsson et al. (2006)](http://www.amazon.com/Cambridge-Expertise-Performance-Handbooks-Psychology-ebook/dp/B00AKE1Z1Y/); [Kuchar (forthcoming)](http://www.iel.carloalberto.org/public/pavel.k/pkuchar-consensus.pdf); [Miller (2013)](http://commonsenseatheism.com/wp-content/uploads/2014/03/Miller-When-is-consensus-knowledge-based.pdf); [Gelman (2009)](http://andrewgelman.com/2009/12/14/how_do_i_form_m/); [Martin & Richards (1995)](http://www.bmartin.cc/pubs/95handbook2.pdf); [Schwed & Bearman (2010)](http://commonsenseatheism.com/wp-content/uploads/2014/03/Shwed-Bearman-The-temporal-structure-of-scientific-consensus-formation.pdf); [Intemann & de Melo-Martin (2013)](http://commonsenseatheism.com/wp-content/uploads/2014/03/Intemann-de-Melo-Martin-Are-there-limits-to-scientists-obligations-to-seek-and-engage-dissenters.pdf). Also see Wikipedia’s article on [scientific consensus](http://en.wikipedia.org/wiki/Scientific_consensus). [ ↩](#fnref:2 "return to article")
    
*   I suppose I should mention that my entire inquiry here is, _ala_ [Goldman (1998)](http://www.amazon.com/Knowledge-Social-World-Alvin-Goldman-ebook/dp/B002B557GE/), premised on the assumptions that (1) the point of epistemology is the pursuit of correspondence-theory truth, and (2) the point of _social_ epistemology is to evaluate which social institutions and practices have instrumental value for producing true or well-calibrated beliefs. [ ↩](#fnref:3 "return to article")
    
*   I borrow this line from [Chalmers (2014)](http://consc.net/papers/progress.pdf): “For much of the paper I am largely saying the obvious, but sometimes the obvious is worth saying so that less obvious things can be said from there.” [ ↩](#fnref:4 "return to article")
    
*   Holden Karnofsky [seems to agree](http://intelligence.org/2013/08/25/holden-karnofsky-interview/): “I think effective altruism falls somewhere on the spectrum between ‘contrarian view’ and ‘unusual taste.’ My commitment to effective altruism is probably better characterized as ‘wanting/choosing to be an effective altruist’ than as ‘believing that effective altruism is correct.’” [ ↩](#fnref:5 "return to article")
    
*   Without such heuristics, we can also rather quickly arrive at contradictions. For example, the majority of scholars who specialize in Allah’s existence believe that Allah is the One True God, and the majority of scholars who specialize in Yahweh’s existence believe that Yahweh is the One True God. Consistency isn’t everything, but contradictions like this should still be a warning sign. [ ↩](#fnref:6 "return to article")
    
*   According to the [PhilPapers Surveys](http://philpapers.org/surveys/), 72.8% of philosophers are atheists, 14.6% are theists, and 12.6% categorized themselves as “other.” If we look only at metaphysicians, atheism remains dominant at 73.7%. If we look only at analytic philosophers, we again see atheism at 76.3%. As for physicists: [Larson & Witham (1997)](http://commonsenseatheism.com/wp-content/uploads/2014/02/Larson-Witham-Scientists-are-still-keeping-the-faith.pdf) found that 77.9% of physicists and astronomers are disbelievers, and [Pew Research Center (2009)](http://commonsenseatheism.com/wp-content/uploads/2014/02/Pew-Research-Center-Scientific-achievements-less-prominent-than-a-decade-ago.pdf) found that 71% of physicists and astronomers did not believe in a god. [ ↩](#fnref:7 "return to article")
    
*   Muller & Bostrom (forthcoming). “Future Progress in Artificial Intelligence: A Poll Among Experts.” [ ↩](#fnref:8 "return to article")
    
*   But, this is unclear. First, I haven’t read the forthcoming paper, so I don’t yet have the full results of the survey, along with all its important caveats. Second, distributions of expert opinion can vary widely between polls. For example, [Schlosshauer et al. (2013)](http://arxiv.org/pdf/1301.1069v1.pdf) reports the results of a poll given to participants in a 2011 quantum foundations conference (mostly physicists). When asked “When will we have a working and useful quantum computer?”, 9% said “within 10 years,” 42% said “10–25 years,” 30% said “25–50 years,” 0% said “50–100 years,” and 15% said “never.” But when the exact same questions were asked of participants at another quantum foundations conference just two years later, [Norsen & Nelson (2013)](http://arxiv.org/pdf/1306.4646.pdf) report, the distribution of opinion was substantially different: 9% said “within 10 years,” 22% said “10–25 years,” 20% said “25–50 years,” 21% said “50–100 years,” and 12% said “never.” [ ↩](#fnref:9 "return to article")
    
*   I say “they” in this paragraph, but I consider myself to be a plausible candidate for an “AGI impact expert,” in that I’m unusually familiar with the arguments and evidence typically brought to bear on questions of long-term AI outcomes. I _also_ don’t have a uniquely good track record on predicting long-term AI outcomes, nor am I among the discovered “super forecasters.” I haven’t participated in IARPA’s forecasting tournaments myself because it would just be too time consuming. I would, however, very much like to see these super forecasters grouped into teams and tasked with forecasting longer-term outcomes, so that we can begin to gather scientific data on which psychological and computational methods result in the best predictive outcomes when considering long-term questions. Given how long it takes to acquire these data, we should start as soon as possible. [ ↩](#fnref:10 "return to article")
    
*   [Weiss & Shanteau (2012)](http://commonsenseatheism.com/wp-content/uploads/2014/02/Weiss-Shanteau-Decloaking-the-privileged-expert.pdf) would call them “privileged experts.” [ ↩](#fnref:11 "return to article")
    
*   Beckstead’s “elite common sense” prior and my “mildly extrapolated elite opinion” method are epistemic notions that involve some kind idealization or extrapolation of opinion. One earlier such proposal in social epistemology was Habermas’ “ideal speech situation,” a situation of unlimited discussion between free and equal humans. See Habermas’ “Wahrheitstheorien” in [Schulz & Fahrenbach (1973)](http://www.worldcat.org/title/wirklichkeit-und-reflexion-festschrift-walter-schulz-zum-60-geburtstag/oclc/462814955) or, for an English description, [Geuss (1981)](http://www.amazon.com/The-Idea-Critical-Theory-Philosophy/dp/0521284228/), pp. 65–66. See also the discussion in [Tucker (2003)](http://commonsenseatheism.com/wp-content/uploads/2014/03/Tucker-The-epistemic-significance-of-consensus.pdf), pp. 502–504. [ ↩](#fnref:12 "return to article")
    
*   Beckstead calls his method the “elite common sense” prior. I’ve named my method differently for two reasons. First, I want to distinguish MEEO from Beckstead’s prior, since I’m using the method for a slightly different purpose. Second, I think “elite common sense” is a confusing term even for Beckstead’s prior, since there’s some extrapolation of views going on. But also, it’s only a “mild” extrapolation — e.g. we aren’t asking what elites would think if they knew _everything_, or if they could rewrite their cognitive software for better reasoning accuracy. [ ↩](#fnref:13 "return to article")
    
*   My rough impression is that among the people who seem to have thought long and hard about AGI outcomes, and seem to me to exhibit fairly good epistemic practices on most issues, my view on AGI outcomes is still an outlier in its pessimism about the likelihood of desirable outcomes. But it’s hard to tell: there haven’t been systematic surveys of the important-to-me experts on the issue. I also wonder whether my views about long-term AGI outcomes are more a matter of seriously tackling a contrarian _question_ rather than being a matter of having a particularly contrarian _view_. On this latter point, see [this Facebook discussion](https://www.facebook.com/lukeprog/posts/10104316308390320?stream_ref=5). [ ↩](#fnref:14 "return to article")
    
*   I haven’t seen a poll of cryobiologists on the likely future technological feasibility of cryonics. Even if there were such polls, I’d wonder whether cryobiologists _also_ had the relevant philosophical and neuroscientific expertise. I should mention that I’m not personally signed up for cryonics, for [these reasons](/lw/guy/open_thread_march_115_2013/8kin). [ ↩](#fnref:15 "return to article")