Texas sharpshooter fallacy

{{logicnav}}{{hatnote|Not to be confused with the [[John F. Kennedy assassination conspiracy theory]].}}{{cquote|A [[nuclear bomb]] always hits ground zero.|||[[Russian]] military joke<ref>{{cite web|url=https://books.google.com/books?id=nIp1AgAAQBAJ&pg=PA51&lpg=PA51&dq=%22A+nuclear+bomb+always+hits+ground+zero%22&source=bl&ots=LxudOt4LF-&sig=8pRblj44a0B0Uyfao4xjI_FvcuE&hl=en&sa=X&ved=0ahUKEwjDkLfB5f3JAhWMOCYKHQiJC4wQ6AEIHTAA#v=onepage&q=%22A%20nuclear%20bomb%20always%20hits%20ground%20zero%22&f=false |title=Ukraine - A joke?|author=Stig-Arne Kristoffersen}}</ref>}}
The '''Texas sharpshooter fallacy''' (or '''clustering fallacy''') occurs when the same data is used both to construct and test a [[hypothesis]].

The fallacy is an imprecision fallacy and an [[informal fallacy]].

==Explanation==
{{cquote|One way to make a list of accurate [[prophecies]] is[:]<br />(1) Have a bunch of people make a bunch of prophecies.<br />(2) Wait and see which ones come true.<br />(3) Throw all the others in the trash.<br />(4) Compile your list of prophecies that came true, and parade it around triumphantly.|||/u/charlaron<ref>https://www.reddit.com/r/atheism/comments/4gicjc/can_someone_please_go_through_this_short_list_of/d2hswib</ref>}}
The fallacy's name comes from a parable in which a [[Texas|Texan]] fires his gun at the side of a barn, paints a bullseye around the bullet hole, and claims to be a sharpshooter. Though the shot may have been totally [[random]], he makes it appear as though he has performed a highly non-random act. In normal target practice, the bullseye defines a region of significance, and there's a low probability of hitting it by firing in a random direction. However, when the region of significance is determined after the event has occurred, any outcome at all can be made to appear spectacularly improbable.

The Texas sharpshooter fallacy uses the same data to both construct and test a hypothesis. A hypothesis must be constructed before data is collected based on that hypothesis. If one data set is used to construct a hypothesis, then a new data set must be generated (ideally, in a different way, based on predictions made by the hypothesis) to test it.

==Examples==
*The [[physicist]] [[Richard Feynman]] once started a lecture on statistical physics by reciting a license plate number he had seen on the way in and asking his students what the probability was that he had seen that particular number. The probability, of course, was quite low.  But this is true no matter what license plate number one sees, and unless it has an independently defined significance, this probability is meaningless.

*A million participant raffle was drawn, and Joe was found to be the winner. Afterwards, someone points out that the odds of Joe winning are a million to one, and thus, he couldn't have won randomly and must have cheated. Of course, the chances of anyone else winning was also a million to one, and this person could've accused the winner of cheating no matter who they were. However, the chances of there ''being'' a winner is 100% guaranteed. In this case, Joe lucked out. ''Somebody'' had to have lucked out.

==="Random" evolution===
[[Creationist]] and [[intelligent design]] arguments claim that the chances of a [[protein]] molecule forming "[[random chance|randomly]]", or a [[cell]] forming "randomly" via [[abiogenesis]], or the [[Universe]] forming "randomly" into what we see today are incredibly low, and thus it must have been [[argument from design|designed]]. This argument is extremely faulty in that it doesn't acknowledge that physical processes are not random, but are guided by the laws of physics, [[chemistry]], and eventually, [[biology]]: [[evolution]] via variation and [[natural selection]].

In late 2010, "[[Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect]]", a [[psychology]] paper by [[Daryl Bem]], ostensibly provided evidence of [[precognition]]. In Bem's experiments, a small but statistically significant number of test subjects' responses appeared to be influenced by conditions which appeared ''later'' in the tests.<ref name=NS>See the coverage in ''New Scientist'', [http://www.newscientist.com/article/dn19712-evidence-that-we-can-see-the-future-to-be-published.html "Is this evidence that we can see the future?"], Peter Aldhous, 11 November 2010.</ref> However, Bem has acknowledged forming some of his conclusions ''after'' the tests, rather than testing fixed hypotheses as any rigorous application of the [[scientific method]] should. He has also stated that, before concluding and publishing his research, "I purposely waited until I thought there was a critical mass that wasn't a statistical fluke".<ref name=NS/> While this may seem logical at first glance, deliberately waiting for such a "critical mass" actually means stopping research at a point when the results appear favourable to the hypotheses rather than continuing through a pre-set number of experiments before checking for overall findings.

===Young Earth thinking===
Much of [[young earth creationism]] relies on this form of ''[[Post hoc, ergo propter hoc|post hoc]]'' reasoning. This is most clearly demonstrated in [[fundamentalist]] [[Christians]]' discussions of how the [[global flood|flood]] [[hydroplate theory|created geologic structures]]. Their ideas rely on finding data and constructing a hypothesis around that data, with no further testing of these ideas after this construction. This is a clear example of this particular [[fallacy]].

===Crazification===
The [[crazification factor]] is often an example in popular usage: people jokingly point out about this factor about general online discourse though the angriest responders are obviously not representative of people online and are just people most likely to comment.

===Alternative medicine===
Those arguing in favor of [[alternative medicine]] often cite examples of traditional medicines (such as willow bark) or derivatives thereof that were incorporated into mainstream medicine, and urge that people should keep an [[open mind]] about whatever remedy the practitioner happens to be promoting. For instance, according to one [[homeopath]]:<ref>''[http://web.archive.org/web/20130614215107/http://gaia-health.com/gaia-blog/2012-04-11/anecdotal-evidence-the-basis-of-all-knowledge/ Anecdotal Evidence: The Basis of All Knowledge]'', a rebuttal to which has been written by [[Orac]]: ''[http://scienceblogs.com/insolence/2012/04/16/a-homeopath-lectures-scientists-about-an/ A homeopath lectures scientists about anecdotal evidence]''.</ref>
{{cquote|We learned that certain herbs had beneficial effects by trying them and passing on the information of what resulted: pure [[anecdotal evidence]]. But that’s how we know, for example, that milk thistle is good for the liver and hawthorn is good for the heart. No studies needed to be done. We learned through experience and anecdote.}}

This ignores, however, [[Bloodletting|all]] [[Lead|the]] [[Mercury|traditional]] [[Birthwort|treatments]] [[Asbestos|that]] were later scientifically shown to be ineffective (or outright dangerous), and that were also supported by anecdotal evidence. Over the course of history, countless remedies have been thought [[Panacea|effective for virtually all diseases]], and, obviously, this means that some of these indications will happen to be true just by chance. If you claim pretty much everything is good for dozens of, or even all, diseases, and it turns out that ''some'' of these remedies are good for ''a few'' of the countless original indications, this is a prediction failure rate of well over 99%. The author's two examples are [[cherry-picking]], and certainly do not prove the validity of anecdotal evidence.

===Cancer clusters===
If you plot incidences of [[cancer]] or other (non-infectious) diseases with dots on a map then chance dictates that the dots will randomly seem to congregate in certain places, which get a higher than average number of occurrences. You may naively expect cases to be distributed evenly if the condition occurred purely randomly, but in practice the [[clustering illusion]] intrudes and a totally even distribution is much less probable than you might expect. Therefore, a cluster is likely to occur, and when you see a cluster there is a natural tendency to start panicking and look for a reason why people are dying. If you can find something to blame, then the result can be unwarranted condemnation. Two notable examples in the [[UK]] occurred with studies into childhood leukemia where what was probably a statistical artifact was mistaken for something else. In the first, a cluster was discovered near the nuclear reactor at Sellafield, and another near a nuclear reactor at Dounreay, although there was no evidence of higher [[radiation]] levels in this area or higher levels of leukemia in other areas near Sellafield, or anything that would suggest a [[Statistical significance|statistically-significant]] causal relationship — but it provided ammunition for those who think [[nuclear power]] is bad. In the second case, a study of leukemia clusters by E. G. Knox found that they tended to occur near railways, leading to the hypothesis it involved [[fossil fuels]], although it's probable he had failed to properly control for the fact that people tend to live near railways, as well as for the random occurrence of clusters within the population: clusters will occur and they will be where people live.<ref name=cruk>[https://scienceblog.cancerresearchuk.org/2016/10/31/sellafield-radiation-and-childhood-cancer-shedding-light-on-cancer-clusters-near-nuclear-sites/ Sellafield, radiation and childhood cancer – shedding light on cancer clusters near nuclear sites], Cancer Research UK blog, Oct 31, 2016</ref><ref>[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1059986/ Leukaemia clusters in childhood: geographical analysis in Britain]. E G Knox, J Epidemiol Community Health. 1994 Aug; 48(4): 369–376.</ref><ref>[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1060220/ Review of the article by Knox: leukemia clusters in childhood.] G M Swaen. J Epidemiol Community Health. 1996 Feb; 50(1): 109.</ref>

These examples were complicated by the fact that people on a map are not distributed randomly but cluster in towns (and near railways). Any cluster is likely to be near something interesting and potentially blamable (a town, a major employer, transport infrastructure, etc) rather than in the middle of nowhere. This shows that the Texas sharpshooter fallacy combines with other fallacies such as the [[gambler's fallacy]] and [[clustering illusion]] where the misinterpretation of random data is involved, and of course [[correlation does not imply causation]]. It was even suggested that childhood leukemia could have an infectious component which would lead to clusters as one person infected others nearby.<ref name=cruk/>

==Demarcation==
You do not commit this fallacy if you:

*calculate the probability of a particular event after the fact based on a criterion that would have been clearly significant even before the event occurred.
*re-test an observation to determine if previous clustering may have been due to chance.

==See also==
*[[List of cognitive biases]]
*[[Anthropic principle]]
*[[P-value fishing]]
*[[Stopped clock]]

==External links== 
*{{wpa|Texas sharpshooter fallacy}}
*[http://tvtropes.org/pmwiki/pmwiki.php/Main/SharpshooterFallacy Sharpshooter Fallacy], [[TV Tropes]]
*[http://www.fallacyfiles.org/texsharp.html Texas Sharpshooter], [[Fallacy Files]]
*[http://skepdic.com/texas.html Texas Sharpshooter's Fallacy], [[Skeptic's Dictionary]]
*[http://www.youtube.com/watch?v=QwLO3ASs5sU ID/creationism: Texas Sharpshooter Fallacy]
*[https://yourlogicalfallacyis.com/the-texas-sharpshooter Your logical fallacy is the texas sharpshooter], [[YLFI]]

==References==
{{reflist}}

{{logfal}}

[[Category:Imprecision fallacies]]
[[Category:Statistics]]
[[Category:Texas]]

[[ru:Ошибка техасского снайпера]]

Categories: Imprecision fallacies, Statistics, Texas