Misuse of p-values
Misuse of "p"-values is common in scientific research and scientific education. "p"-values are often used or interpreted incorrectly; the American Statistical Association states that "p"-values can indicate how incompatible the data are with a specified statistical model. From a Neyman–Pearson hypothesis testing approach to statistical inferences, the data obtained by comparing the "p"-value to a significance level will yield one of two results: either the null hypothesis is rejected (which however does not prove that the null hypothesis is "false"), or the null hypothesis "cannot" be rejected at that significance level (which however does  not prove that the null hypothesis is "true"). From a Fisherian statistical testing approach to statistical inferences, a low "p"-value means "either" that the null hypothesis is true and a highly improbable event has occurred "or" that the null hypothesis is false. Clarifications about "p"-values. The following list clarifies some issues that are commonly misunderstood regarding "p"-values: Representing probabilities of hypotheses. A frequentist approach rejects the validity of representing probabilities of hypotheses: hypotheses are true or false, not something that can be represented with a probability. Bayesian statistics actively models the likelihood of hypotheses. The "p"-value does not in itself  allow reasoning about the probabilities of hypotheses, which requires multiple hypotheses or a range of hypotheses, with a prior distribution of likelihoods between them, in which case Bayesian statistics could be used. There, one uses a likelihood function for all possible values of the prior instead of the "p"-value for a single null hypothesis. The "p"-value describes a property of data when compared to a specific null hypothesis; it is not a property of the hypothesis itself. For the same reason, "p"-values do not give the probability that the data were produced by random chance alone. Multiple comparisons problem. The  multiple comparisons problem occurs when one considers a set of statistical inferences simultaneously or infers a subset of parameters selected based on the observed values. It is also known as the look-elsewhere effect. Errors in inference, including confidence intervals that fail to include their corresponding population parameters or hypothesis tests that incorrectly reject the null hypothesis, are more likely to occur when one considers the set as a whole. Several statistical techniques have been developed to prevent this from happening, allowing significance levels for single and multiple comparisons to be directly compared. These techniques generally require a higher significance  threshold for individual comparisons, so as to compensate for the number of inferences being made. The webcomic "xkcd" satirized misunderstandings of "p"-values by portraying scientists investigating the claim that eating jellybeans caused acne. After failing to find a significant ("p" < 0.05) correlation between eating jellybeans and acne, the scientists investigate 20 different colors of jellybeans individually, without adjusting for multiple comparisons. They find one color (green) nominally associated with acne ("p" < 0.05). The results are then reported by a newspaper as indicating that green jellybeans are linked to acne at a 95% confidence level—as if green were  the only color tested. In fact, if 20 independent tests are conducted at the 0.05 significance level and all null hypotheses are true, there is a 64.2% chance of obtaining at least one false positive and the expected number of false positives is 1 (i.e. 0.05 × 20). In general, the family-wise error rate (FWER)—the probability of obtaining at least one false positive—increases with the number of tests performed. The FWER when all null hypotheses are true for "m" independent tests, each conducted at significance level α, is: