Anecdotal evidence

Anecdotal evidence (also proof by selected instances, or, more pejoratively, anecdata) is use of one or more anecdotes (specific instances of an event; stories) to either support or refute a claim. The use of anecdotal evidence to draw a conclusion is like using the NBA all-star teams to estimate the average height of Americans.
Whereas anecdotal evidence is sometimes the starting point of a proper scientific investigation, it is all too often the ending point and every point of a pseudoscientific investigation. In the world of pseudoscience, an anecdote is treated as the equivalent of a peer-reviewed, double-blind, repeatable scientific experiment with consistent results. 
Anecdotal evidence is often used in politics, journalism, blogs, and many other contexts to make or imply generalisations based on very limited and cherry-picked examples, rather than reliable statistical studies. A classic instance was Ronald Reagan's story of a "welfare queen" who was abusing the system, who Reagan attempted to portray as indicative of the average welfare recipient. It turned out she didn't even exist when some reporters finally decided to look for her (even if she had, this would not actually prove anything about welfare recipients generally).
Anecdotal evidence is especially vulnerable to confabulation or outright deceit.
Remember: the plural of "anecdote" is not "data".Unless, of course, you gather so many anecdotes that it's possible to construct a representative statistical model. And even then, a double-blind study would be preferable.
==Form==
For example, "Billy-Bob ate 3 pounds of apples a day for a year and his cancer disappeared – so can you!"
It should be clear that P2 is invalid. Take the following syllogism:
Consume that goat, and face the wrath of the Goat Overlord!
Another form:
It's clear from the above that fallacies like post hoc ergo propter hoc and argument from incredulity can play supporting roles in the perception of anecdotes as convincing.
===Legitimate use===
In two instances, it is possible to use anecdotes non-fallaciously:
*If you use one or more anecdotes to refute the claim that there are no instances of the event that the anecdote describes. This is not fallacious because one counterexample is all it takes to prove a universal rule false, or an existential rule true.
*If you use one or more anecdotes as an example of a general rule which is already supported by a broad, comprehensive investigation (i.e., your evidence/argument does not rely on the anecdotes, they are just used to illustrate the point).
==Problems==
While anecdotes — when believed to be true, at least — are nice illustrative stories, they do not constitute good repeatable scientific evidence. This is because anecdotes only ever apply to individuals or individual experiences and are subject to the biases that this brings with it. Even without bias, the small scope of an anecdote means that it cannot be generalized to a larger group, due to variations between, say, the physiology of individual humans (and of individual pathogens), the structural integrity of rocks, or whatever the anecdote is about meaning that what works to cure an illness in one person or shatter one stone may or may not work to cure the same illness in another person or to shatter another stone of the same kind. And even if those differences could be accounted for, that still leaves the circumstances of an anecdote unaccounted for; the influence of environmental factors on a given procedure's result should not be overlooked. Anecdotes also tend to begin when a problem is first noticed, without any awareness of how long it may have been festering quietly without being observed;For instance, many diseases have an "" — a length of time between a or toxin entering the body and the onset of its symptoms, during which a pathogen might quietly proliferate without the host being consciously aware of its presence. Given that most people do not have different diseases have different incubation periods (and these are just diseases with external causes; genetic disorders such as or can have "incubation periods" of multiple decades), and that a person generally does not instinctively know what disease they are suffering from without the aid of a trained doctor's diagnosis, it can often be difficult or impossible for a person to deduce when and how a given pathogen entered their body and started causing an illness. thus, the root cause of the problem is often unaccounted for. For all of these reasons and more, it is impossible to say that an individual anecdote is representative, and it is also impossible to actually detect the real cause of the anecdote.
For instance, with life-saving medical treatments (say, pills that reduce heart disease and subsequently lower the death rate), there are some deaths that occur whether or not the medication was taken (and not all of these deaths are even caused by heart disease). Therefore, if someone who is on the medication dies, you cannot tell if they would have died anyway without it — you can't prove that the medical intervention worked, or not, from the one case study.
It is very rare for an intervention to be, by itself, a sufficient cause of something. Rather, they tend to change the probability of a given event occurring. This means, obviously, that one can cherry pick examples that show something does or does not work, regardless of what effect it actually has. For instance, if the municipal government of some city enacts a law to reduce crime, one can find anecdotes "demonstrating" the exact opposite of the law's actual effect, whatever it may be. If it's effective: "Before the law I never had any problems with crime, but since its enactment I've been mugged once and had my house burglarized! This law is useless!" If it's not: "Before the law, I was robbed twice, but ever since the law, I haven't been robbed once! It must be working!"
Ultimately, anecdotal evidence is very prone to (and is also prone to false negatives).
===Regression to the mean===
If someone with a rash gets better after taking a homeopathic remedy, that doesn't mean the remedy caused the improvement, as it could simply have been an exception to the rule. Maybe it would have gotten better anyway — for example, it could have just been an allergic reaction, or a sweat-rash or anything along those effects. The anecdote doesn't tell you anything about all the people who did not get better; many other people with rashes could have taken homeopathic remedies to no effect.The anecdote also says nothing about other people who took the treatment and got better for reasons that may or may not be related to the treatment or the cause of our given anecdote subject's recovery.
===Adulteration===
Not to mention that these remedies can contain actual active substances, which are misleadingly listed under "non-medicinal ingredients". A "homeopathic" anti-acne cream may contain tea tree oil, which is an effective antifungal.
In addition, some people selling alternative medicines sometimes adulterate them with drugs in order to give them the desired medicinal effect and thus generate return customers. For instance, a "herbal" cold remedy may be adulterated with paracetamol and antihistamines.Media Releases for 2003, MedSafe. In these sorts of cases, the improvements may very well be caused by the drugs rather than the herbs.
===Improbability===
There are also anecdotes in which a person suffered from a disease for many years, and no conventional treatment was able to cure them, but they improved after applying some alternative remedy for a few days/weeks/months. For instance:
The implication is that the improvement cannot possibly have been a coincidence, because it was so unlikely. Of course, since millions of people use alternative medicines every day, such coincidences are certain to happen all the time to many people, even if the odds of it happening in one specific instance are low. The odds of a particular person winning the lottery are extremely low, but somebody always does, regardless. Not to mention that most people get sick many times in the course of their life, and the number of such events may number in the thousands or even tens of thousands, if one includes such mundane things as pain, cuts, nausea, or blemishes. Some improbable coincidence is bound to happen during at least one, if not several, of these many illnesses. It is worth also considering a practitioner's situation. Over the course of her career, she will see countless thousands of patients. Even if her treatments are effective, many of her patients' recoveries (and possible website testimonials) will be unrelated to her treatments, and the large number of cases almost certainly means that some of these coincidental recoveries will be extremely unlikely.
While "case studies" exist within medicine that are essentially anecdotal, these are treated mostly as curiosities or communications between doctors, rather than hard evidence on par with real trials.
===Selection bias===
Selective reporting is the main reason that popularly disseminated anecdotes will be almost certainly unrepresentative. People tend to focus on the interesting, the unusual, or the powerfully emotive events while ignoring the mundane. After all, no one ever sold a newspaper where the front-page headline read "girl receives new vaccine; suffers zero complications and is now alive and well". People who believe in alternative medicine may, due to confirmation bias, selectively remember stories that support their belief (which they will, of course, tell everybody about), and ignore those that don't.
Selection bias can have an impressive effect in terms of PR. Let's assume 1 million people (1,000,000) decide to take some ineffective remedy to cure their cancer. Let's further assume (for the sake of argument) that only 0.1% of this million will experience spontaneous remission (the actual remission rate, for breast cancer and basal cell carcinoma at least, is closer to 20%Spontaneous Regression of Melanoma May Offer Insight Into Cancer Immunology, Carrie Printz, JNCI J. Natl. Cancer Inst. (2001) 93 (14): 1047-1048.Study reveals surprising cancer remission rate, Dartmouth Medicine Magazine.), and that 0.3% were misdiagnosed and so do not actually have cancer. This makes for a total of 0.4%, or 4000 people. Now, 4000 people translates into a lot of testimonials, "CAM cured my cancer" blogs, Internet comments, newspaper articles, and real-life word-of-mouth, so this makes an extremely positive impression for the therapy. But the other 99.6% died and so are not around to leave any testimonials, positive or otherwise. Thus, even in a hypothetical scenario that assumes statistically very low false positive rates, the quantity of false positives is nevertheless numerically quite large. In reality, the number of past and present cancer patients relying on alternative medicine is much larger than one million, and the false positive rate is significantly higher than 0.4%, making the possibility of a given supposedly successful "cancer cure" anecdote being a coincidence even more likely.
Even if a given treatment were effective, anecdotal evidence would still be useless, because many of the cases would be the result of natural improvement, and it would be impossible to tell which anecdotes are cures and which ones are flukes. There is no way to separate the signal from the noise with only single data points suffering from selection bias.
=== Incompleteness and lack of follow-up ===
Another issue arises: anecdotes are usually not complete — they do not necessarily tell the full story. They often cover only the period of the improvement, and anything happening after the punchline or after the publication of the anecdote is not included. In addition, the anecdote-narrator may omit important factors relating to the "cure" (such as, for instance, a patient using conventional medicine together with an alternative remedy). Mark Twain's essay A Majestic Literary Fossil, which describes various forms of old-fashioned quackery, notes an anecdote:
If, in this case, the fact that the bloodletting was discontinued gets "mentioned only in the most casual way", it is easy to imagine that many anecdotes leave out important facts like this altogether.
A cancer patient named Kim Tinkham went on The Oprah Winfrey Show with a proponent of the alkaline diet and claimed the diet had cured her, but she died of the cancer a few years later. Now, the eventual fate of Tinkham was revealed in news sources because she was famous, but if anonymous (or at least non-famous) authors of online testimonials die of a supposedly "cured" illness, it is not likely that this fact will get published on the website of the original testimonial, or possibly anywhere.
In a case-study involving spontaneous remission, a woman's went away completely, with no treatment, shortly after she woke up from a coma. In a CAM anecdote, that would be that — cure proven. But since this was a case-study rather than a testimonial, the person's post-recovery situation was observed, and two years later, a mild form of diabetes reappeared."Spontaneous Remission of Diabetes", R. B. Haining and R. G. Haining. Similarly, in an anecdotal case of a "cured" disease, the disease may later turn out to have been illusory.
==Advantages of anecdotal evidence==
My grandma heeded anecdotal evidence and she lived to 106.
Also, Jeanne Calment, the person with the longest confirmed lifespan (122 years), smoked for almost her entire adult life and nothing bad happened to her.http://extremelongevity.net/supercentenarians/worlds-top-ten-oldest-people-ever/oldest-person-ever-jeanne-calment/http://www.endlesshumanpotential.com/jeanne-calment.html And besides Calment, there are many centenarians who are heavy smokers.Even the Long-Lived Smoke, Drink, and Don't Exercise The composer Jean Sibelius was an alcoholic and smoker for most of his life, and he lived to be 91, even saying: "All the doctors who wanted to forbid me to smoke and to drink are dead."The Rest Is Noise, Alex Ross. By alternative medicine logic, this should be indisputable proof that tobacco is actually good for you. As noted by Dan King in his 1861 book Tobacco: What it is, and What it does:
==The power of anecdotes==
Anecdotal evidence is not accepted as proper proof or scientific evidence for good reason, yet they still dominate people's thinking. This is because while facts and figures may be true, they are difficult for people to relate to. Anecdotes have convincing and strong narratives that resonate with people on a key individual level — someone telling an anecdote will lead the listener to think that it just could happen to them too. There are further biases at work. Anecdotes, even implausible ones, can contain a high level of detail. It is this detail that causes people to become involved in the story and to imagine it more clearly; a story about a young girl won't be nearly as widely spread and accepted as a story about a five year-old girl called Sally from Wisconsin who was cycling to school one day.
The biases at work in causing people to add disproportionate weight to anecdotes cause major problems during health scares. A major case in point is vaccine hysteria. 
*Hard, scientific and proven (yet dry and difficult to grasp) evidence about the value of vaccines exists on the one side.
*Strong narrative stories told by concerned parents competes on the other side. 
It is often advised in such circumstances that those on the side of science should also use their own anecdotes to take advantage of their persuasive power. For example, instead of giving statistics of illnesses increasing when vaccination uptake dropped, tell — in great detail — stories about specific deaths that have occurred from vaccine-preventable illnesses. Respectable scientists, however, may be against such manipulative tactics.
==See also==
*What's The Harm? — Anecdotal evidence against alternative medicine
*Essay:List of CAM-free anecdotes — "Miraculous" recoveries without alternative medicine.
*Pragmatic fallacy
*Magical thinking
*Exception that proves the rule
*Credulity
*Miracle
*Trepanation
*Shoehorning
==External links==
*Why Bogus Therapies Seem to Work, Barry Beyerstein
*Why We Need Science: “I saw it with my own eyes” Is Not Enough, Harriet Hall
*The Futility of Anecdotal Evidence, The "What's going on at RationalWiki" blog
*Homeopathy and Its Kindred Delusions, by Oliver Wendell Holmes Sr., which dwells at some length on the topic of anecdotal evidence
*Cancer: "Miraculous Recoveries", Quackwatch
*Understanding alternative medicine "testimonials" for cancer cures, Orac
*A homeopath lectures scientists about anecdotal evidence, Orac
*The key difference between alternative medicine and evidence, Orac
*Popular but Weird and Dangerous Cures, Paul Ingraham
*Your logical fallacy is anecdotal, YLFI
*Sociologist Considers Own Behavior Indicative Of Larger Trends, The Onion
==Notes==
==References==
Category:Imprecision fallacies
Category:Language
Category:Pseudoscience
Category:Rhetoric