Opinion ID: 1919140
Heading Depth: 1
Heading Rank: 14

Heading: Potential Sources of Error

Text: Researchers study a small part of the relevant population. Thus, the findings in an epidemiological study could differ from the true association in the larger population because of random variations, or chance, in the selected sample. [47] Epidemiologists refer to this problem as a sampling error. [48] When researchers find an association (positive or negative), they use significance testing to assess the likelihood of a sampling error. [49] A statistically significant result is unlikely to be the result of random variations in a selected population sample. [50] In evaluating whether a sampling error caused a study's results, experts often use a convention called the p-value. [51] The p-value is a calculation, based on a study's data, of the probability that a positive association in the study would have resulted from a sampling error when no real association existed. [52] If the p-value falls below a preselected, acceptable significance level, the study's results are statistically significant. [53] Epidemiologists generally consider a p-value that falls below a significance level of .05 to be statistically significant. [54] A significance level of .05 presents a 5-percent probability that researchers observed an association because of chance variations. [55] But statistical significance addresses only the likelihood that a relative risk would have resulted from chance even if no real association existed between the disease and agent. Statistical significance does not show an association's magnitude. [56] So researchers often express a study's results through confidence intervals. Confidence intervals show the association's magnitude and how statistically stable the association is. [57] Using the study's relative risk and preselected significance level, researchers calculate the range of values within which the study's results would likely fall if researchers repeated the study many times. [58] Graphically, the calculation is an asymmetrical bell curve around the relative risk point, showing the distribution of possible results. The confidence interval is the range of values between the boundaries of the curve on a numerical axis. [59] If researchers selected .05 for the study's significance level, then the study will show a corresponding 95-percent confidence level in the plotted confidence interval. [60] This means that if a confidence level of .95 is selected for a study, 95% of similar studies would result in the true relative risk falling within the confidence interval.... [T]he narrower the confidence interval, the greater the confidence in the relative risk estimate found in the study. Where the confidence interval contains a relative risk of 1.0, the results of the study are not statistically significant. [61] For example, a trial judge might see a hypothetical study stating its results as follows: relative risk of 1.6 (95% confidence interval = 1.1 to 2.4). This statement indicates that the study's positive association (greater than 1.0) is statistically significant because the confidence interval does not include 1.0 or less. That is, the confidence interval, with 95-percent confidence, excludes the possibility of no association or a negative association. Conversely, another hypothetical study showing a relative risk of 1.6 (95% confidence interval = 0.9 to 1.2) is not statistically significant because the confidence interval includes the possibility that no association exists between the agent and the disease. This logic can be applied to other statistical measures of association. [62] But significance testing shows only that random chance probably did not produce the observed association. [63] Experts also consider whether a data collection error or design error affected the study's results. Also, they ask whether researchers failed to consider some other exposure or characteristic that varies between the groups and could explain the incidence of disease. Experts refer to these types of errors as bias and uncontrolled confounding, respectively. [64] A poorly conceived or conducted study that is statistically significant could be far less reliable than a well-conceived and conducted study that is not statistically significant. [65]