In this system, the statistics of interest are computed on the main sample and are checked against the validation sample to verify that the effects are real. Using this technique, results that are spurious will usually be revealed by the validation sample. However, measurements are seldom perfect. Therefore, close attention must be paid to the effects of measurement errors. This is especially important when dealing with noisy data such as questionnaire responses or processes which are difficult to measure precisely.
Methods are available for taking measurement error into account in some statistical models.
In addition to difficulties with bias and methodology, there are a number of problems which can arise in the context of substantive interpretation as well. These problems usually involve determining the significance of certain findings, avoiding confusion between precision and accuracy, and unraveling the causal relationships among variables. Precision and Accuracy are two concepts which seem to get confused frequently. Estimates can be precise without being accurate, a fact often glossed over when interpreting computer output containing results specified to the fourth or sixth or eighth decimal place. Assessing causality is the reason of most statistical analysis, yet its subtleties escape many statistical consumers.