Statistics are used to assess the variability that is always present, and then make reasonable, mathematics-based guesses as to whether or not observed effects are due to chance or to treatments. When we conclude that there is a reasonable chance that differences were, in fact, due to treatments, then we say treatments had a significant effect . This conclusion does not mean that we proved that the treatments caused differences, only that we are satisfied that our guess is probably correct. When we are unable to draw the conclusion that treatments differed, we say that the treatments are not significantly different .
Small effects are very difficult to prove.
It is simply the standard deviation of all samples in a trial divided by the overall mean of all samples. It is usually expressed as a percentage of the overall mean. In some on-farm research trials, it may be decided that a wrong decision may not be very costly. This could be the case where treatment costs are essentially the same, e.g., seed costs in variety comparisons. Increasing the sample size or replicates reduces the chances of making an incorrect decision when the same decision rule is applied.