Extraordinary claims require extraordinary evidence

[[File:Carl Sagan Planetary Society.jpg|thumb|right|165px|Carl Sagan laughs at your baseless conspiracy theories!]]
{{skepnav}}
'''"Extraordinary claims require extraordinary evidence"''' (a.k.a., the '''Sagan standard''') was a phrase made popular by [[Carl Sagan]]. Its roots are much older, however, with the [[French]] [[mathematician]] {{wpl|Pierre-Simon Laplace}} stating that: "...the more extraordinary a fact, the more it needs to be supported by strong evidence"{{efn|"De ce qui précède, nous devons généralement conclure que, '''plus un fait est extraordinaire, plus il a besoin d'être appuyé de fortes preuves'''; car, ceux qui l'attestent pouvant ou tromper ou avoir été trompés, ces deux causes sont d'autant plus probables que la réalité du fait l'est moins en elle-même."<ref>[https://archive.org/details/theorieanaldepro00laplrich/page/n31/ ''Théorie analytique des probabilités''] by (1812). Gauthier-Villars. Œuvres Complète de Laplace: Tome Septiême. Troisième Édition (1820) . Page xvi.</ref>}} Also, [[David Hume]] wrote in 1748: "A wise man … proportions his belief to the evidence", and "No testimony is sufficient to establish a [[miracle]], unless the testimony be of such a kind, that its falsehood would be more miraculous than the fact which it endeavors to establish."<ref>{{Cite book | last = Hume| first = David| authorlink = David Hume| title = An Enquiry concerning Human Understanding, chap. 10.4 | year=1748 | url=http://www.davidhume.org/texts/?text=ehu#10}}</ref> and [[Marcello Truzzi]] says: "An extraordinary claim requires extraordinary proof."<ref>Marcello Truzzi, ''On the Extraordinary: An Attempt at Clarification, Zetetic Scholar'', Vol. 1, No. 1, p. 11, 1978.</ref>

Either way, the phrase is central to the [[scientific method]], and a key issue for [[critical thinking]], [[rational]] thought, and [[skepticism]] everywhere.

The evidence put forth by proponents of such things as [[god]]s, [[ghosts]], the [[paranormal]], and [[unidentified flying object|UFO]]s is highly questionable at best and offers little in the way of proof. Even if we accepted what evidence there is as valid (and it is highly debatable if we should), limited and weak evidence is not enough to overcome the extraordinary nature of these claims.

==Analogy==
[[File:Historiae animalium 1551 De Monocerote.jpg|thumb|left|100px|Your engraving is not evidence.]]
Alice and Bob are two friends talking after school. Alice tells Bob that she watched a movie the previous evening. Bob believes her easily because he knows that movies exist, that Alice exists, and that Alice is capable and fond of watching movies. If he doubts her, he might ask for a ticket stub or a confirmation from one of her friends. If, however, Alice tells Bob that she flew on a [[unicorn]] to a fairy kingdom where she participated in an ambrosia-eating contest, and she produces a professionally-printed contest certificate and a friend who would testify to the events described, Bob would still not be inclined to believe her without strong evidence for the existence of flying unicorns, fairies, and ambrosia-eating contests.

==Probability theory==
{{main|Probability}}
<div style="float:right; margin:1em; padding:1em; text-align:center;"><math>P(A|B)=\frac{P(B|A)\cdot P(A)}{P(B)}</math> <br /> [[Bayes]]' Theorem</div>

While the idea that a sufficiently outlandish claim requires a lot more compelling evidence is quite intuitive, it can be quantified nicely with probability theory in a [[Bayesian]] framework. In short, sufficient evidence must be capable of raising a highly improbable claim to be highly probable — and the more improbable the evidence, the better. By application of Bayes' theorem, it's possible to show this in action mathematically.

Assume, for instance, someone claims to be able to predict what way a coin<ref group=note>Tossed by someone else, because methods of tossing a coin may affect the outcome.</ref> will land almost perfectly. We know this is an extraordinary claim, so we'll say that just by guessing if the person is telling the truth or not that it's a million-to-one chance. In reality, the number would be even more improbable, but this can be used for illustration. So we ask them to demonstrate the skill. They're ''almost'' perfect, so let's assume they guess right about 90% of the time — this allows them the opportunity for their skill to mess up once in a while, but still prove to be pretty good. This gives us all the information we need to know to quantify ''how'' extraordinary the evidence must be.

Consider if they guessed a single coin toss correctly. The odds of guessing by chance is a mere 50%, or 50:50.

<center>
<math>\frac{0.9 \cdot 0.000001}{0.5} = 0.0000018</math>
</center>

A single coin toss doesn't improve our odds dramatically. The evidence just isn't ''extraordinary enough'' — you can guess a single coin toss correctly 50% of the time with no special skills involved. It all rests on how improbable our evidence, P(B), actually is and a 50:50 chance isn't particularly improbable. For two coin tosses, P(B) becomes 0.25, and for 10 coin tosses, it comes to roughly 0.00097. Plugging those numbers into Bayes' theorem gives us a probability of genuine skill (given P(A) of a million-to-one) of around 0.0009, which although still small is a considerable improvement on that original million-to-one chance. By 20 or so correctly guessed coin tosses, the skill is starting to look a lot more genuine.

This is the basic idea underpinning [[statistical significance]]; is it more likely that our evidence is [[random]], or due to a real effect, and is the improbability of the evidence presented in proportion to the improbability of the claim being made? But Sagan's quip about extraordinary evidence doesn't just mean that we can take someone's word for it if they managed to toss so many coins in a row. [[Derren Brown]] can pull off such a feat with some effort and misdirection as shown in his special on ''The System'', so we always need to consider alternative [[hypotheses]] and compare how likely they are. As with Derren Brown tossing a coin with 10 heads in a row, is it more likely that they're [[psychic]], or are cheating? So, tests such as [[James Randi]]'s [[million-dollar challenge]] will control for this potential factor, making sure that the probability of foul play, fraud, and cheating is far less than the probability of genuine psychic power.

==Additional note==
The converse of the proposition that 'extraordinary claims require extraordinary evidence' would be that ''any'' claim requires ''some'' evidence. Therefore, to argue that a competing ordinary claim is more likely to be true than an extraordinary one, simply because the extraordinary one has no "extraordinary" evidence to support it, fails to account for the possibility that the competing "ordinary" claim has no evidence at all to support it (ordinary or otherwise).

==See also==
*[[Logical fallacy]]
*[[The Fine Art of Baloney Detection]]
*[[Russell's Teapot]]

==Notes==
{{reflist|group=note}}

==References==
{{reflist}}

[[Category:Assessing pseudoscience]]
[[Category:Evaluating arguments]]
[[category:Philosophy of science]]
[[Category:Skepticism]]
[[Category:Clichéed sayings]]
[[ru:Экстраординарные утверждения требуют экстраординарных доказательств]]

Categories: Assessing pseudoscience, Evaluating arguments, Philosophy of science, Skepticism, Clichéed sayings