If the data are in a particular order, consider the possibility of dependence. In fact, if the blocks are random, then there generally is a correlation for measurements within a block, although it is assumed to be constant. If the serial correlation is positive, then the actual P value is much greater than the reported P value, making it possible to falsely conclude that there are significant differences when none exist. Conversely, if the serial correlation is negative, then the actual P value is much smaller than the reported P value, making it possible to falsely conclude that there are no significant differences when the treatment means are in fact significantly different. Examining the residuals , from which the linear effects of both the treatment factor and the blocking factor have been removed, may be an easier way to uncover an implicit factor.
Outliers are anomalous values in the data.
They may be due to recording errors, which may be correctable, or they may be due to the sample not being entirely from the same population. Apparent outliers may also be due to the values being from the same, but nonnormal , population. The F statistic is based on the sample means and the sample variances, each of which is sensitive to outliers . In particular, a large outlier can inflate the overall variance, decreasing the F statistic and thus perhaps eliminating a significant difference. A nonparametric test may be a more powerful test in such a situation.