The final formal study requires collecting the data for a meaningful sample size. The effect size, in turn, can be used to estimate a sample size which will yield sufficient statistical power. If the sample size is too small, statistical power is low, and the probability of detecting an effect, even if that effect actually exists, will go down. Thus, the pilot studies also serve to ensure that the final study will not fail to find an effect because of lack of statistical power. The formal study provides the data, but data analysis is still necessary to demonstrate the experimental hypotheses.
Data analysis requires a careful and thorough exploration of the data.
These details of analysis are not addressed in the context of the individual experiments presented here, but the statistical confidences reported are based on a thorough analysis. All of the experiments described in this thesis use within-subjects designs counterbalanced for order of presentation. This means that each subject performs all experimental conditions, but that the order in which subjects perform the conditions is systematically varied. This is sometimes referred to as a latin squares design. A latin squares design helps to ensure that the results will not be biased by order of presentation effects, because order is an explicit between-subjects factor that can be analyzed separately.