Fundamentals of Experimental Design: Guidelines for Designing Successful Experiments

Published in Agron. J. 107:692–705 (2015) doi:10.2134/agronj2013.0114 Available freely online through the author-supported open access option. Copyright © 2015 by the American Society of Agronomy, 5585 Guilford Road, Madison, WI 53711. All rights reserved. No part of this periodical may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Designing experiments involves a marriage of biological and mathematical sciences. Th e mathematical, or statistical, science is obvious. We use scientifi c fundamentals and principles that have been developed during the past century to conduct three types of experiments. Observational experiments are those designed to measure or verify an assumed constant, such as the velocity of light or the mass of an atom. Measurement experiments are those designed to measure the properties of a population, the members of which are variable, such as commodity prices, production statistics, or neutrino frequencies. As biologists, we are principally concerned with comparative (or manipulative) experiments, in which our global goal is to compare or contrast two or more practices or systems that may have some relevance to our fi eld of scientifi c inquiry. It is the solutions, or the specifi c choices we must make, that are not so obvious. In conducting comparative experiments, we routinely follow the general theory of scientifi c inquiry shown in Fig. 1. We begin with questions and/or hypotheses that must be translated to models based on the specifi c subject matter, e.g., cotton (Gossypium hirsutum L.) plants, farm machinery, or hay bales. Th e subject matter model is translated into a statistical model, which is developed in concert with the statistical design. Th e statistical design includes both the treatment design and the experimental design and provides a set of rules and procedures that allow us to conduct the experiment. For many of us, this process becomes routine, such that we tend to forget the fundamental nature and assumptions of statistical designs, instead forming designs and forging ahead using time-honored and traditional approaches that have worked well for us in the past. We oft en favor familiarity, simplicity, and constancy over any thought of change or concept of improvement. Researchers must recognize that designing comparative experiments is a massively decision-based exercise, so that the truism “If you choose not to decide, you still have made a choice” (Peart, 1980) is an appropriate concept in biological research. Once the experiment has been conducted and the data collected, the statistical analysis proceeds as determined by the researcher before the experiment was conducted. Th e analysis leads to specifi c interpretations of the results, creating inferences and conclusions that bring the research back to the original question or hypothesis. Finally, and a point oft en ignored in most models, is the feedback loop that comes when the experiment is completed (Fig. 1). As scientists, we are intimately familiar with this feedback loop, oft en correctly opining that, “Any good experiment leads to more questions and new hypotheses than it answers.” What we tend to forget or ignore is the fact that two types of information travel along this feedback loop: scientifi c answers to our biological questions and mathematical or statistical information that can be used to design better, more effi cient, future experiments. Just as any experiment helps the ABSTRACT