NONSAMPLING ERRORS - We know 

you’re out there ... And we’re gonna get ya’! 

by Toni Tremblay 

Sampling errors receive a great deal of attention - they are printed in the summaries, they are 

fairly easy to understand, and their impact is discussed immediately. In that sense, sampling 

errors are highly visible. All of the other problems that can affect survey indications are 

lumped together as "nonsampling errors." They are difficult to detect, more difficult to 

measure, and even more difficult to control. The danger which they present can be serious. 

This danger is confounded by the fact that most nonsampling errors lay hidden among all the 

survey activities. 

It is not unusual for a survey's nonsampling errors to be more detrimental than its sampling 

errors. In fact, a U .S. Census Bureau study found some nonsampling errors that were 10 

times the magnitude of sampling errors. Therefore, it is well worth the effort to understand 

under what conditions they can occur and to strive to lessen their impact on the survey 

indications. 

Nonsampling errors are the remaining errors after accounting for sampling errors. They 

include all the biases that can affect surveys. Even complete censuses are plagued by 

nonsampling errors. After all, even a census might have poorly worded questions, improperly 

trained personnel, or badly defined concepts. 

Each and every stage of a survey process or census is a potential contributor to nonsampling 

errors. Even harder to believe is that we are among the culprits - survey designers, manual 

writers, trainers, enumerators, respondents, coders, keyers, printers, analysts, managers, etc. 

No single person or group is to blame, but we must take responsibility as a whole agency for 

the nonsampling errors. 

A few illustrations of nonsampling errors are: 

(1) failure to obtain data on sampled units due to refusals, not-at-homes, unreturned 

 or lost questionnaires, 

(2) long or repeated interviews which result in inaccurate responses, 

(3) the wording of the question causes the farmer to misunderstand the concept, 

(4) moisture meter is "just a hair off," and 

(5) incorrect decimal placement during keyentry. 

How can we measure the nonsampling errors? Measurement of nonsampling errors usually 

includes subsampling the data and remeasuring this subset. However, new measurement 

techniques must be applied so as to avoid a repetition of the original errors. This situation is 

where special research studies come into the picture. We understand that they are sometimes 

difficult and always irritating to do; however, they are necessary if we are to learn how to best 
deal with the subtle and troublesome problems caused by nonsampling errors. 

The control and prevention of nonsampling errors are as varied as their sources. Actions 

taken may include clarification of purpose and definitions of the survey, attention to 

confidentiality concerns, key verification, improved recall techniques, consistency in training, 

callbacks, and use of tested and verified software and instruments. 

Since their control and measurement must be an integral part of the total survey design, NASS 

has analyzed a great many nonsampling errors at all stages of the survey process. A few of them 

are: 

(1) The Influence of Using Previous Survey Data in the 1986 April ISP Grain Stocks 

 Survey (Pafford, 1988) 

(2) Problems with Telephone Surveys (Warde, 1986) 

(3) The Effect of Enumeration on Soybean OY, 1986 (Battaglia, 1987) 

(4) Nonsampling Errors from Lab Procedures of the Wheat OY Survey (Bond, 1984). 

Above all, awareness and discussion of nonsampling errors throughout all phases of the 

survey is critical. Constant feedback from those of you at the "front lines" is essential when 

trying to keep these nonsampling errors visible.