SAMPLING ERRORS -- NECESSARY BUT 

CONTROLLABLE 

by Phil Kott 

Every year NASS publishes thousands of numbers that measure the amount of agricultural 

activity in the United States. Enumerators do not talk to every farmer; that approach 

would be too time-consuming and expensive. Instead, the enumerators talk to a sample of 

farmers. The sample represents the entire population of farmers, and relatively accurate 

indications about this population can be calculated from the sample. NASS uses the term 

“indication” to represent the statistical point estimate calculated from survey data. This avoids 

confusion with the term “estimate” which is reserved for the official published estimate. An 

indication from a sample may differ somewhat from the census value that would have been 

obtained if the enumerators had interviewed every farmer. This difference is known as sampling 

error. 

NASS does not know the exact difference between one of our indications and the 

corresponding census value. If we did, we could easily remove the sampling error from the 

indication. Still, there is a way of measuring the potential for sampling error in NASS 

indications. One of the nice properties of probability sampling is that it provides a means for 

developing indications and it also provides a measure of the precision of these indications. 

That measure is called standard error. 

Suppose we select a sample, collect the data, and calculate an indication from the sample. 

That indication may be larger than the census value that would have been calculated if the 

entire population of farmers had been interviewed. Suppose we select another sample, collect 

the data, and generate a second indication. This new indication might be smaller than the 

census value. As we kept selecting different samples, we would see the indication “bounce” 

around the census value. The standard error gives us an idea of how large this “bouncing” can 

be. As the standard error gets larger, the potential for a difference between the sample 

indication and the census value gets larger. Conversely, as the standard error gets lower, the 

potential for a difference gets smaller. 

Statisticians say that the lower the standard error of an indication, the more precise it is. One 

sure way of increasing the precision of an indication (all other things being kept the same) is 

to increase the size of the sample on which it is based. As the sample size approaches the 

population size, the standard error of the indication tends toward zero. It reaches zero when 

the entire population of farmers is interviewed. 

Standard errors are measured in the same units as the indications to which they refer. For 

example, when estimating hogs, standard errors are expressed in terms of how many hogs. 

This fact can cause a problem in interpretation. Suppose NASS estimates that there are 5 

million hogs in Illinois with a standard error of 750,000 hogs and 10 million hogs in Iowa with 

a standard error of 1 million hogs. Which indication is more precise - the 5 million hogs in 
Illinois or the 10 million hogs in Iowa? Since a standard error of 750,000 is less than a 

standard error of 1 million, you might first think that the indication of hogs in Illinois is more 

precise. 

Something seems wrong, doesn't it? The standard error in Iowa is only 10 percent of the hog 

indication in Iowa, while the standard error in Illinois is 15 percent of its hog indication. 

Relatively speaking, the Iowa indication should be viewed as more precise. 

Statisticians were quick to realize this potential problem. They like to use the standard error 

expressed as a percentage of the indication itself. They call this percentage the coefficient of 

variation (CV) of the indication. In the above example, the CV in Iowa is 10 percent, while 

the CV in Illinois is 15 percent. As the CV of an indication gets smaller, the more precise the 

indication becomes in relative terms. 

A national indication computed by NASS has a larger standard error than any State 

indications going into it, but the CV of the national indication is generally smaller than the 

State CV's. In other words, a national indication is usually more precise than its component 

State indications. In the Quarterly Crops/Stocks Survey, for example, the national indications 

are based on a sample of about 75,000 farms while each State sample usually contains from 

1,000 to 5,000 farms. The relatively large sample at the national level causes a national 

indication generally to have a smaller CV than any of its component State indications. 

Sampling error is a necessary evil caused by NASS's need to produce statistics in a timely and 

cost-effective manner. Fortunately, this problem is controllable by (among other things) 

increasing the size of NASS samples. One of the most important jobs of statisticians in the 

Agency is to establish sample sizes that strike a proper balance between the needs for 

containing costs and for providing relatively precise indications of agricultural activity. 

Of course, sampling is not the only factor that can cause a NASS indication to be different 

from the "truth." Indications from surveys can be affected by factors such as nonresponse or 

poor wording on the questionnaire. These effects are called nonsampling errors, which is the 

subject of another article in this series.