United States 

Department of

Agriculture

National

Agricultural

Statistics

Service

Research and

Development Division

Washington DC 20250

RDD Research Report

RDD-08-06

June 2008

Assessing the Effect of 

Calibration on 

Nonresponse Bias in the 

2006 ARMS Phase III

Sample Using Census 

2002 Data

Morgan S. Earp

Jaki S. McCarthy

Nick D. Schauer

Phillip S. Kott

This paper was prepared for limited distribution to the research community outside the United 

States Department of Agriculture. The views expressed herein are not necessarily those of the 

National Agricultural Statistics Service or of the United States Department of Agriculture. 

i

EXECUTIVE SUMMARY

Phase III of the Agricultural Resource Management Survey (ARMS) is one of the most 

complex and detailed sample survey data collections conducted by the United States 

Department of Agriculture’s (USDA) National Agricultural Statistics Service (NASS). 

For this survey, NASS collects calendar year economic data from agricultural producers 

nationwide. 

In September 2006 the Executive Office of the President released the Office of 

Management and Budget Standards and Guidelines for Statistical Surveys. The Office of 

Management and Budget’s (OMB) new standards and guidelines for statistical surveys 

addressed a number of federal statistical agency issues. Standard 3.2 specifically 

addressed the issue of low response rates and analysis of unit nonresponse; Guideline 

3.2.9 required that surveys failing to meet an 80 percent response rate be subject to 

nonresponse bias analysis. 

The 2005 Agricultural Resource Management Survey (ARMS) Phase III Survey 

Administration Analysis (Hopper, 2006) reported a response rate of 70.5 percent and the

2006 Agricultural Resource Management Survey (ARMS) Phase III Survey 

Administration Analysis (Hopper, 2007) reported an even lower response rate of 67.6

percent, both requiring NASS to conduct analyses of nonresponse bias. This latter 

analysis uses the same methodology used in the former study which is described in the 

NASS Research and Development Division (RDD) report, Assessing the Effect of 

Calibration on Nonresponse Bias in the 2005 ARMS Phase III Sample Using Census 

2002 Data (Earp, McCarthy, Schauer, & Kott, 2008). 

Records sampled for the 2006 ARMS Phase III were matched against records from the

long-form sample of the 2002 Census of Agriculture. Nonresponse bias in the ARMS 

respondent sample was assessed using census data. Three weighted means of census data 

were computed and compared across 20 regions: 1) the mean for all the matching records

computed using base sampling weights, 2) the mean for the matching records responding

to the ARMS using the same base sampling weights, and 3) the mean for the latter group 

using the sampling weights adjusted by calibration. 

Relative bias of the mean was assessed for 17 “study variables” using a variation of the 

formula provided by OMB in Guideline 3.2.9. Although significant biases were 

exhibited in 11 of 17 variables using the 2006 ARMS Phase III base sampling weights, 

the 2006 ARMS Phase III calibration weights were able to reduce the bias to statistical 

insignificance (i.e. p > .05) in over 90 percent (10/11) of the study variables. For this 

analysis, the calibration process varied slightly from that of the 2006 ARMS Phase III in 

that egg and milk production were not measured by the 2002 Census and could not be 

included in the calibration. The inability to replicate the 2006 ARMS Phase III 

calibration process fully may in part account for the one variable, total sales, 

demonstrating a significant level of bias even using the calibrated weights. This study 

suggests, however, that the process of calibration is an effective tool in reducing 

nonresponse bias levels. 

ii

RECOMMENDATIONS

1. Nonresponse bias of all study variables, especially livestock purchases and 

fertilizer expenses, should be reevaluated when the 2007 Census data are 

available. This Census will contain equivalent calibration target variables for 

egg and milk production, as well as expenditure data for all Census respondents,

allowing for an assessment that will be consistent with the calibration targets 

used for the 2006 ARMS Phase III. 

2. Methods should be developed to assess biases not measured in this analysis, 

especially those that may exist in only a single region.1

 

1 ARMS PHASE III estimation regions include the 15 leading cash receipts states (Arkansas, California, 

Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Carolina, Texas, 

Washington, and Wisconsin). The remaining 33 states (Alaska and Hawaii were not sampled for ARMS)

were assigned to one of the five main production regions (Atlantic, South, Midwest, Plains, and West).

Assessing the Effect of Calibration on Nonresponse Bias in the 2006 ARMS 

Phase III Sample Using Census 2002 Data

Morgan Earp, Jaki McCarthy, Nick Schauer, & Phil Kott2 

Abstract

The United States Department of Agriculture’s National Agricultural Statistics Service

(NASS) conducts the annual Agricultural Resource Management Survey (ARMS) in 

three phases. The third phase of the ARMS collects detailed economic data which is 

highly sensitive. As a consequence, this phase suffers from relatively low response rates

for a federal survey. According to the 2006 Office of Management and Budget Standards 

and Guidelines for Statistical Surveys, response rates lower than 80 percent may not only 

result in nonresponse bias, but they can jeopardize the future of surveys carried out by 

federal agencies. NASS has been operating under the assumption that the use of 

calibrated weights derived from appropriate targets addresses nonresponse bias in the 

2006 Phase III ARMS. This assumption was tested using Census 2002 expendituresample data. 

The results showed that calibrated weights decreased bias levels so that they were no 

longer significantly different from zero at the 0.05 level for over 90 percent of the 

variables evaluated.

Key Words: Nonresponse; response rate; bias; calibration weights.

 

2

Jaki McCarthy provided assistance with this research while the Chief Research Methodologist with the 

USDA’s National Agricultural Statistics Service (NASS) – Research and Development Division (RDD). 

Phil Kott provided assistance with this research while the Chief Research Statistician with the USDA’s 

NASS – RDD. Morgan Earp is a survey and mathematical statistician in the RDD, located in Room 305, 

3251 Old Lee Highway, Fairfax, VA 22030. Nick Schauer is a mathematical statistician in the Agency’s 

Statistics Division.
2

1. INTRODUCTION

On September 22, 2006, the Executive Office of the President released the Office of 

Management and Budget Standards and Guidelines for Statistical Surveys based on the 

recommendations of the Federal Committee on Statistical Methodology’s (FCSM) 

Subcommittee on Standards for Statistical Surveys. The Office of Management and 

Budget’s (OMB) new standards and guidelines for statistical surveys pertain to aspects of 

surveys conducted by federal statistical agencies. 

Federal statistical agencies, such as the United States Department of Agriculture’s 

(USDA) National Agricultural Statistics Service (NASS), are directly affected by OMB’s 

new standards and guidelines for statistical surveys. One of the standards (3.2) issued by 

OMB addresses response rates and analysis of nonresponse bias. According to Standard 

3.2, 

Agencies must appropriately measure, adjust for, report, and analyze unit and 

item nonresponse to assess their effects on data quality and to inform users. 

Response rates must be computed using standard formulas to measure the 

proportion of the eligible sample that is represented by the responding units in 

each study, as an indicator of potential nonresponse bias. (Office of Management 

and Budget, 2006, p. 14)

In 2005, the Agricultural Resource Management Survey (ARMS) Phase III response rate 

was 70.5 percent (n = 34,937), which fell below the OMB response rate threshold of 80 

percent listed in Guideline 3.2.9; therefore, NASS was required by OMB to research the 

effect of nonresponse bias (Earp, McCarthy, Schauer, & Kott, 2008). In 2006, the 

Agricultural Resource Management Survey (ARMS) Phase III response rate was 67.6

percent (n = 34,192), which again fell below the 80 percent threshold, and thus NASS 

was again required by OMB to research the effect of nonresponse bias. Since the Phase 

II response rate of 81.3 percent exceeded OMB’s 80 percent threshold, nonresponse bias 

assessment was only required for Phase III, the “problem” stage. Specifically, Guideline 

3.2.9 states

Given a survey with an overall unit response rate of less than 80 percent, conduct 

an analysis of nonresponse bias using unit response rates as defined above, with 

an assessment of whether the data are missing completely at random. As noted 

above, the degree of nonresponse bias is a function of not only the response rate 

but also how much the respondents and nonrespondents differ on the survey 

variables of interest. For a sample mean, an estimate of the bias of the sample 

respondent mean is given by: 

r nr

nr

r r t

y y

n

n

B y y y

where, 

t

y the mean based on all sample cases;
3

r

y the mean based only on respondent cases;

nr y

the mean based only on nonrespondent cases;

n the number of cases in the sample; and

nnr

 the number of nonrespondent cases.

For a multistage (or wave) survey, focus the nonresponse bias analysis on each 

stage, with particular attention to the “problem” stages. A variety of methods can 

be used to examine nonresponse bias, for example, make comparisons between 

respondents and nonrespondents across subgroups using available sample frame 

variables. In the analysis of unit nonresponse, consider a multivariate modeling 

of response using respondent and nonrespondent frame variables to determine if 

nonrespondent bias exists. (Office of Management & Budget, 2006, p. 16)

Currently, NASS calculates the unweighted unit response rates (RRU) for the ARMS 

based on the formula provided under Guideline 3.2.2 of the Office of Management and 

Budget Standards and Guidelines for Statistical Surveys: 

C R NC O e(U)

C

RRU

where,

C = the number of completed cases or sufficient partials;

R = the number of refused cases;

NC = the number of noncontacted sample units known to be eligible;

O = the number of eligible sample units not responding for reason other than 

refusal;

U = the number of sample units of unknown eligibility, not completed; and

e = the estimated proportion of sample units of unknown eligibility that are 

eligible. (p. 14)

Thus, NASS sums the number of positive usables, out-of-business, and non-farms and 

calculates the percentage this sum represents of the total number of reports to calculate 

the response rate for ARMS Phase III. 

The ARMS is conducted in three phases. Phase I screens for potential samples for Phases 

II and III. Phase II collects data on cropping practices and agricultural chemical usage, 

while Phase III collects detailed economic information about the agricultural operation, 

as well as the operator’s household. Phase III is the only phase of the 2006 ARMS with 

response rates lower than 80 percent. 

Due to lower response rates with 2006 ARMS Phase III, the potential for nonresponse 

bias is greater there. NASS weights the ARMS Phase III respondent sample in such a 

way that estimated variable totals for a large set of items match “targets” determined 

from other sources. This is done through a weighting process called “calibration.” 

Calibration is the process of adjusting survey weights so that certain targets are met. 
4

NASS uses official estimates of farm numbers, corn, soybean, wheat, cotton, fruit and 

vegetable acres as well as cattle, milk production, hogs, broilers, eggs and turkeys as 

calibration targets. For example, after calibration, the calibration-weighted sum of the

survey data will equal the NASS estimate for corn acres. In addition to reducing 

confusion in the user community that might result from NASS releasing alternative 

estimates for the same totals, calibration weighting produces 2006 ARMS Phase III

estimates with generally lower variances and reduces nonresponse biases. This report 

describes an ongoing research effort aimed at measuring the potential for nonresponse 

bias in the ARMS Phase III and the success or failure of calibration in removing it. 

Nonresponse bias is very difficult to measure directly. Fortunately, an indirect measure 

of nonresponse bias is available for the 2006 ARMS Phase III, hereafter called simply the 

“ARMS.” 

The Census of Agriculture is a mandatory collection of data from all known agricultural

operations. NASS has data from the Census on items of interest for many of the ARMS 

nonrespondents; however, the Census itself is incomplete. An estimated 17.90 percent of 

all farms were missing from the 2002 Census Mailing List, and 12.26 percent of farms on 

the List failed to respond to the Census. Moreover, not all ARMS sampled farms could 

be matched to 2002 Census records. Nevertheless, by comparing the 2002 Census 

values of ARMS respondents to the full sample of ARMS respondents as a whole, we can

measure the difference between the average ARMS respondent and the average of the full 

sample without any nonresponse adjustment. Additionally, this analysis intends to 

measure the reduction of that difference from using a calibration-weighting process 

similar to the one used for the 2006 ARMS. 

Although the 2002 Census data do not perfectly match the 2006 ARMS Phase III data, 

they are highly correlated (see Appendix A). The present evaluation will effectively 

compare 2006 ARMS Phase III survey respondents to nonrespondents using their 2002 

Census data. The 2002 Census expenditure data, which were required for all Census 

reports considered usable in this research, were available for 43 percent of 2006 ARMS 

Phase III reports.3

 

2. METHODS

Our analytical data set consists of census values for farms sampled for the ARMS that 

also provided 2002 expenditure-sample data on the Census. In the 2002 Census only a 

sample of farms received the long version of the questionnaire which asked the

expenditure questions. 

The base sampling weight for a farm in our analytical data set was its ARMS sample 

weight before calibration multiplied by its Census sample weight. Each ARMS 

responding farm was calibrated to produce weighted totals for the calibration variables 

that were equal to the base-sampling-weighted totals computed from both respondents 

 

3 The match rate for 2006 ARMS Phase III reports with 2002 Census expenditure data was significantly 

higher for nonrespondents (47.5%) than for respondents (40.5%) (z = 12.24, p < .05).
5

and nonrespondents. The calibration variables used were inventory/acreage numbers for 

cattle, corn, cotton, pigs, soybeans, wheat, fruit, vegetables, broilers, and turkeys. Each 

of these target variables, plus egg and milk production, was used operationally in 

calibrating the ARMS data. 

As in the operational program, the ARMS respondent subset was calibrated 

independently in 20 regions. These included the 15 leading cash receipts states

(Arkansas, California, Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Minnesota,

Missouri, Nebraska, North Carolina, Texas, Washington, and Wisconsin). The remaining 

33 states (Alaska and Hawaii are not sampled for the ARMS) were grouped using the five 

production regions: 1) Atlantic, 2) South, 3) Midwest, 4) Plains, and 5) West.

Our analysis focuses on 17 specific (non-calibration) variables collected on both the 

ARMS and the Census: 

1. Total Acres

2. Total Sales

3. Acres Rented

4. Cropland Acres

5. Total Production Expenses

6. Crop Expenses

7. Seed Expenses

8. Fertilizer Expenses

9. Chemical Expenses

10. Livestock Purchases

11. Feed Purchases

12. Hired Labor Expenses

13. Machinery and Equipment Value

14. Government Payments

15. Operator’s Age

16. Operator’s Race

17. Farm Type. 

These variables were also included in a similar analysis for ARMS PHASE III 2005 

(Earp et al., 2008). 

Letting 

r

y

denote the base-sample or calibrated-sample mean among the ARMS 

respondent subset for a study variable, and 

t

y

denote the corresponding base-sample 

mean among the entire matched sample, it is a simple matter to compute the relative bias 

of the former with respect to the latter, relBias 

t

r t

y

y y

. The statistical significance of 

this value is much harder to assess since the samples on which 

r

y

and 

t

y

are based are

complex and overlapping. 
6

Fortunately, we can easily test the persistence or absence of a systematic bias across the 

20 regions. To this end, we compute the following measure of bias of an ARMSrespondent mean (before or after calibration) with respect to the Census mean in every 

region: 

M = log( ) log( )

r t

y y

 = log r

t

y

y

 =

t

r t

y

y y

log 1 

t

r t

y

y y

This measure is conveniently symmetric,

log( ) log( ) [log( ) log( )] t r r t y y y y

, while 

retaining the scale-invariance property of the relative bias (i.e., multiplying the reported 

item value on each farm by a fixed factor does not affect the overall relative bias). 

The bias measure M for a study variable in a region can be treated as an independent 

random variable. The null hypothesis of no bias (again, either before or after calibration) 

can be tested against an alternative hypothesis of a persistent bias (p%) across all the 

regions. The conventional t test based on the 20 observations (one per region) is 

asymptotically normal under both the null and alternative hypotheses. We follow the 

standard practice of approximating the distribution of this test statistic with a Student’s t

having 19 degrees of freedom. This may lead to liberal inferences (the inappropriate 

rejection of the null hypothesis when it is true) because the M-values for the study 

variable may not be normally distributed with a common variance across regions. 

Nevertheless, by taking logs we create a test statistic that is more nearly normal and

homoscedastic than absolute biases would be. 

A sign and a signed-rank test of the 20 paired observations for a study variable before and 

after calibration was conducted. The sign test is not as powerful as the other two tests 

(i.e., it more often fails to find that M is significantly different from 0 when, in fact, there 

is a persistent bias across the regions), but it assumes neither that M is normal nor 

homoscedastic. The signed-rank test assumes the latter, but not the former. We include 

it in our results for completeness. 

3. RESULTS AND DISCUSSION

Our results are summarized in Table 1. Chemical expenses, machinery and equipment 

value, government payments, acres rented, farm type, fuel and oil expenses, operator’s 

age, and cropland acres (Variables 1-6) do not exhibit significant biases using either 

calibrated or uncalibrated weights. Results slightly varied from those in the previous 

analysis of ARMS PHASE III 2005 data (Earp et al., 2008); total acres operated no 

longer exhibits significant bias using either calibrated or uncalibrated weights; on the 

contrary, chemical expenses, machinery and equipment value, and fuel and oil expenses 

now exhibit significant bias using the uncalibrated weights, but not the calibrated weights

(Earp et al., 2008). 
7

Table 1: Mean Comparisons and Indicated Biases for Matching Records Using Base Sampling Weights versus Calibrated Weights 
8

Table 1 (Cont.): Mean Comparisons and Indicated Biases for Matching Records Using Base Sampling Weights versus Calibrated Weights 
9

In over 90 percent (10/11) of the study variables exhibiting persistent biases using the 

base sample weights (i.e., variables 7-17), calibration weighting is able to reduce the bias 

so that it was no longer significantly different from zero using a t-test with p < .05. The 

rate of bias elimination remained consistent from 2005 to 2006, although the rate of bias 

and the rank of variable bias varied (Earp et al., 2008). All of these variables show a 

significant reduction in bias levels using a paired t-test. After calibration, only one study 

variable, total sales, has a significant bias. This result varied from 2005, where fertilizer

expense was the only variable exhibiting significant bias. Using the 2006 data, fertilizer 

expense no longer exhibits significant bias using the calibrated weights, but total sales 

which did not exhibit significant bias using the calibrated weights does (Earp et al., 2008. 

As in 2005, the estimated bias of livestock purchases remains the largest among the study 

variables. Using only the base-sampling weights, this bias was highly significant using 

all three test statistics. After calibration, although still large in magnitude, the estimated

bias was reduced to statistical insignificance using all the tests. For this variable, 

calibration continues to reduce the bias significantly, if not completely. 

4. CONCLUSION

ARMS data are used by farm organizations, commodity groups, agribusiness, Congress, 

State Departments of Agriculture, and the USDA. The USDA uses ARMS data to 

evaluate the financial performance of farms and ranches, which influence agricultural 

policy decisions. The Department also uses Phase III data for objective evaluation of 

critical issues related to agriculture and the rural economy; therefore, it is essential that 

measures be taken to minimize the effect of nonresponse bias in ARMS, specifically 

Phase III. 

In assessing the adjustment for nonresponse bias in the 2006 ARMS Phase III, the 2002 

Census mean estimates of total production expenses, livestock purchases, hired labor 

expenses, feed purchases, fuel and oil expenses, chemical expenses, machinery and 

equipment value, seed expenses, cropland expenses, and fertilizer expenses demonstrated 

significant bias using just the base sample weights. Although the magnitude of the 

relative bias of the mean estimate remained high for livestock purchases using the 

calibrated weights, calibration reduced the magnitude of this bias to statistical 

insignificance (see Table 1). For this analysis, the calibration process varied slightly 

from that of the 2006 ARMS Phase III. Egg and milk production were not included as

calibration targets, because these data items were not collected for the 2002 Census. This 

may help to explain why the magnitude of the estimated relative bias of the mean for 

livestock in Table 1 remained high even after the data were calibrated. Although it was 

not possible to use these as calibration targets in this analysis, their use in the ARMS 

PHASE III survey may reduce the bias for livestock purchases in published ARMS data. 

According to Guideline 3.2.13 of the Office of Management and Budget Standards and 

Guidelines for Statistical Surveys, NASS should:
10

Base decisions regarding whether or not to adjust or impute data for item 

nonresponse on how the data will be used, the assessment of nonresponse bias 

that is likely to be encountered in the review of collections, prior experience with 

this collection, and the nonresponse analysis discussed in this section. When 

used, imputation and adjustment procedures should be internally consistent, 

sampled on theoretical and empirical considerations, appropriate for the analysis, 

and make use of the most relevant data available. If multivariate analysis is 

anticipated, care should be taken to use imputations that minimize the attenuation 

of underlying relationships.

Due to the broadness of the ARMS Phase III data user community and the survey’s 

impact on agricultural policy, it is crucial that the calibration process effectively adjusts 

for nonresponse bias. Assuming that the adjustment process is even more effective than 

demonstrated here (particularly for livestock purchases and total sales) when all 

calibration targets (including egg and milk production) are available and used, it appears 

that NASS is appropriately addressing the issue of nonresponse bias in ARMS Phase III 

through the calibration process.

Limitations of this analysis include: 1) Inability to replicate the 2006 ARMS Phase III 

calibration process exactly without egg and milk items; 2) Inability to assess farms not 

covered or responding to the Census of Agriculture; and 3) Inability to recognize 

localized biases in the ARMS data (tests were limited to persistent biases across regions).

Knowing that the analyzed data come from the 2002 Census and not from the 2005 

ARMS Phase III survey does not limit, but strengthens the analysis. It allows us to focus 

entirely on the impact of the nonresponse per se. 

5. Recommendations

Based on the results of the present study, the following recommendations are offered: 

1. Nonresponse bias of all study variables, especially livestock purchases and 

fertilizer expenses should be reevaluated when the 2007 Census data are 

available. This Census will contain equivalent calibration target variables for 

egg and milk production, as well as expenditure data for all Census respondents,

allowing for an assessment that will be consistent with the calibration targets 

used for the 2006 ARMS Phase III. 

2. Methods should be developed to assess biases not measured in this analysis, 

especially those that may exist in only a single region. 
11

6. References

Earp, M.S., McCarthy, J.S., Shauer, N.D., & Kott, P.S. (2008), Assessing the Effect of 

Calibration on Nonresponse Bias in the 2005 ARMS Phase III Sample Using Census 

2002 Data. Research and Development Division Staff Report RDD-08-01, United 

States Department of Agriculture, National Agricultural Statistics Service.

Kott, P.S. (2005), “Using Calibration Weighting to Adjust for Nonresponse and Coverage 

Errors” Survey Methodology 32: 133-142.

Hopper, R. (2007). 2006 Agricultural Resource Management Survey (ARMS) Phase III 

Survey Administration Analysis. Census and Survey Division Staff Report SAB-07-

14, United States Department of Agriculture, National Agricultural Statistics Service.

United States. Department of Agriculture. (2002). 2002 Census of Agriculture, Vol. 1, 

Appendix C. Washington, DC: U.S. Department of Agriculture.

United States. Department of Agriculture. 2007. 2006 Agricultural Resource 

Management Survey (ARMS) Phase III Survey Administration Analysis. Washington, 

DC: U.S. Department of Agriculture.

United States. Department of Education. 2003. National Center for Education Statistics 

Statistical Standards. Washington, DC: U.S. Department of Education.

United States. Executive Office of the President. 2006. Office of Management and Budget 

Standards and Guidelines for Statistical Surveys. Washington, DC: U.S. Executive 

Office of the President.
12 Census Census Census Census Census Census Census Census Census Census Census Census

APPENDIX A: Census 2002 and ARMS Phase III 2006 Correlations & Scatter 

Plots

Table A1: Census 2002 and ARMS Phase III 2006 Variable Correlations with Outliers

r r 2Scatter Plots

Total Acres Operated .89399

( n = 9,380 )

.79922

 

ARMS PHASE III

Acres Rented .63828

( n = 9,380 )

.40740

 

ARMS PHASE III

Cropland Acres .64025

( n = 9,380 )

.40992

 

ARMS PHASE III

Total Production Expenses .83655

( n = 9,380 )

.69982

 

ARMS PHASE III

Seed Expenses .57250

( n = 9,380 )

.32776

 

ARMS PHASE III

Fertilizer Expenses .71797

( n = 9,380 )

.51548

 

ARMS PHASE III

Chemical Expenses .79302

( n = 9,380 )

.62888

 

ARMS PHASE III

Crop Expenses .75064

( n = 9,380 )

.56346

 

ARMS PHASE III

Livestock Purchases .69380

( n = 9,380 )

.48136

 

ARMS PHASE III

Feed Purchases .83451

( n = 9,380 )

.69641

 

ARMS PHASE III

Hired Labor Expenses .74749

( n = 9,380 )

.55874

 

ARMS PHASE III

Fuel & Oil Expenses .56588

( n = 9,380 )

.32022

 

ARMS PHASE III
13 Census Census Census Census

Machinery & Equipment .49977

( n = 9,380 )

.24977

 

ARMS PHASE III

Government Payments .47336

( n = 9,380 )

.22407

 

ARMS PHASE III

Operator’s Age .63618

( n = 9,380 )

.40472

 

ARMS PHASE III

Farm Type .84887

( n = 9,380 )

.72058

 

ARMS PHASE III

1. All correlations were significant at the .05 level.

2. Correlations were only estimated for ARMS respondents.

3. Outliers were flagged using DFFITS, Cook’s D, and studentized residuals and are shown in red.
14 Census Census Census Census Census Census Census Census Census Census Census Census Census

Table A2: Census 2002 and ARMS PHASE III 2006 Variable Correlations without

Outliers

r r 2Scatter Plots

Total Acres Operated .95629

( n =9,278)

.91449

 

ARMS PHASE III

Acres Rented .87279

( n = 9,295 )

.76176

 

ARMS PHASE III

Cropland Acres .86821

( n = 8,996 )

.88723

 

ARMS PHASE III

Total Production Expenses .85281

( n = 9,177 )

.72728

 

ARMS PHASE III

Seed Expenses .68220

( n = 9,167)

.46540

 

ARMS PHASE III

Fertilizer Expenses .75103

( n =9,129)

.56405

 

ARMS PHASE III

Chemical Expenses .81290

( n = 9,077 )

.81290

 

ARMS PHASE III

Crop Expenses .81961

( n = 9,114 )

.67176

 

ARMS PHASE III

Livestock Purchases .46853

( n = 9,289 )

.21952

 

ARMS PHASE III

Feed Purchases .63679

( n =9,159)

.40550

 

ARMS PHASE III

Hired Labor Expenses .83480

( n = 9,135)

.69689

 

ARMS PHASE III

Fuel & Oil Expenses .68545

( n =9,126)

.46984

 

ARMS PHASE III

Machinery & Equipment .61670

( n = 8,976 )

.38032

 

ARMS PHASE III
15 Census Census Census

Government Payments .59126

( n =8,972)

.34959

 

ARMS PHASE III

Operator’s Age .89151

( n = 8,725)

.79479

 

ARMS PHASE III

Farm Type .95712

( n =8,842)

.91608

 

ARMS PHASE III

1. All correlations were significant at the .05 level (n = 19,483).

2. Correlations were only estimated for ARMS respondents.
16

Figures A1-A2: Census 2002 versus ARMS PHASE III 2006 Total Acres Operated Census 2002 

– Total Acres Operated (Acres)

Scatter Plot of Total Acres Operated with Outliers

0

100000

200000

300000

400000

0 100000 200000 300000 400000

ARMS PHASE III 2006 – Total Acres Operated (Acres)

Census 2002 

–Total Acres Operated (Acres)

Scatter Plot of Total Acres Operated without Outliers 

0

10000

20000

30000

40000

50000

60000

70000

0 10000 20000 30000 40000 50000 60000 70000

ARMS PHASE III 2006 – Total Acres Operated (Acres)
17

Figures A3-A4: Census 2002 versus ARMS PHASE III 2006 Acres Rented Census 2002 

– Acres Rented (Acres)

Scatter Plot of Acres Rented with Outliers 

0

100000

200000

300000

400000

0 100000 200000

ARMS PHASE III 2006 – Acres Rented (Acres)

Census 2002 

– Acres Rented (Acres)

Scatter Plot of Acres Rented without Outliers 

0

10000

20000

30000

0 10000 20000 30000

ARMS PHASE III 2006 – Acres Rented (Acres)
18

Figures A5-A6: Census 2002 versus ARMS PHASE III 2006 Cropland Acres Census 2002 

– Cropland Acres (Acres)

Scatter Plot of Cropland Acres with Outliers 

0

10000

20000

30000

0 20000 40000 60000 80000 100000 120000

ARMS PHASE III 2006 – Cropland Acres (Acres)

Census 2002 

– Cropland Acres (Acres)

Scatter Plot of Cropland Acres without Outliers 

0

1000

2000

3000

4000

5000

6000

0 2000 4000 6000 8000 10000 12000

ARMS PHASE III 2006 – Cropland Acres (Acres)
19

Figures A7-A8: Census 2002 versus ARMS PHASE III 2006 Total Production 

Expenses Census 2002 

– Total Production Expenses (Dollars)

Scatter Plot of Total Production Expenses with Outliers 

0. 00E+00

1. 00E+07

2. 00E+07

3. 00E+07

4. 00E+07

5. 00E+07

6. 00E+07

7. 00E+07

8. 00E+07

9. 00E+07

1. 00E+08

1. 10E+08

0. 00E+00 5. 00E+07 1. 00E+08 1. 50E+08 2. 00E+08

ARMS PHASE III 2006 – Total Production Expenses (Dollars)

Census 2002 

– Total Production Expenses (Dollars)

Scatter Plot of Total Production Expenses without Outliers 

0

10000000

20000000

30000000

0 10000000 20000000 30000000 40000000

ARMS PHASE III 2006 – Total Production Expenses (Dollars)
20

Figures A9-A10: Census 2002 versus ARMS PHASE III 2006 Seed Expenses Census 2002 

– Seed Expenses (Dollars)

Scatter Plot of Seed Expenses with Outliers 

0

1000000

2000000

3000000

4000000

0 2000000 4000000 6000000 8000000 10000000

ARMS PHASE III 2006 – Seed Expenses (Dollars)

Census 2002 

– Seed Expenses (Dollars)

Scatter Plot of Seed Expenses without Outliers 

0

100000

200000

300000

400000

500000

600000

0 1000000 2000000

ARMS PHASE III 2006 – Seed Expenses (Dollars)
21

Figures A11-A12: Census 2002 versus ARMS PHASE III 2006 Fertilizer Expenses Census 2002 

– Fertilizer Expenses (Dollars)

Scatter Plot of Fertilizer Expenses with Outliers 

0

1000000

2000000

3000000

0 2000000 4000000

ARMS PHASE III 2006 – Fertilizer Expenses (Dollars)

Census 2002 

– Fertilizer Expenses (Dollars)

Scatter Plot of Fertilizer Expenses without Outliers 

0

100000

200000

300000

400000

500000

600000

0 500000 1000000 1500000

ARMS PHASE III 2006 – Fertilizer Expenses (Dollars)
22

Figures A13-A14: Census 2002 versus ARMS PHASE III 2006 Chemical Expenses Census 2002 

– Chemical Expenses (Dollars)

Scatter Plot of Chemical Expenses with Outliers 

0

1000000

2000000

3000000

4000000

5000000

0 2000000 4000000 6000000

ARMS PHASE III 2006 – Chemical Expenses (Dollars)

Census 2002 

– Chemical Expenses (Dollars)

Scatter Plot of Chemical Expenses without Outliers 

0

100000

200000

300000

400000

500000

600000

700000

0 200000 400000 600000 800000 1000000

ARMS PHASE III 2006 – Chemical Expenses (Dollars)
23

Figures A15-A16: Census 2002 versus ARMS PHASE III 2006 Crop Expenses Census 2002 

– Crop Expenses (Dollars)

Scatter Plot of Crop Expenses with Outliers 

0

1000000

2000000

3000000

4000000

5000000

6000000

0 5000000 10000000 15000000 20000000

ARMS PHASE III 2006 – Crop Expenses (Dollars)

Census 2002 

– Crop Expenses (Dollars)

Scatter Plot of Crop Expenses without Outliers 

0

100000

200000

300000

400000

500000

600000

700000

800000

900000

1000000

1100000

0 1000000 2000000 3000000

ARMS PHASE III 2006 – Crop Expenses (Dollars)
24

Figures A17-A18: Census 2002 versus ARMS PHASE III 2006 Livestock Purchases Census 2002 

– Livestock Purchases (Dollars)

Scatter Plot of Livestock Purchases with Outliers

0

10000000

20000000

30000000

40000000

50000000

60000000

70000000

0 20000000 40000000 60000000 80000000

ARMS PHASE III 2006 – Livestock Purchases (Dollars)

Census 2002 

–Livestock Purchases (Dollars)

Scatter Plot of Livestock Purchases without Outliers 

0

1000000

2000000

3000000

0 1000000 2000000 3000000 4000000 5000000

ARMS PHASE III 2006 – Livestock Purchases (Dollars)
25

Figures A19-A20: Census 2002 versus ARMS PHASE III 2006 Feed Purchases Census 2002 

– Feed Purchases (Dollars)

Scatter Plot of Feed Purchases with Outliers

0

10000000

20000000

30000000

40000000

50000000

0 20000000 40000000 60000000 80000000

ARMS PHASE III 2006 – Feed Purchases (Dollars)

Census 2002 

–Feed Purchases (Dollars)

Scatter Plot of Feed Purchases without Outliers 

0

1000000

2000000

3000000

4000000

5000000

6000000

0 2000000 4000000 6000000 8000000

ARMS PHASE III 2006 – Feed Purchases (Dollars)
26

Figures A21-A22: Census 2002 versus ARMS PHASE III 2006 Hired Labor Expenses Census 2002 

– Hired Labor Expenses (Dollars)

Scatter Plot of Hired Labor Expenses with Outliers

0

1000000

2000000

3000000

4000000

5000000

6000000

7000000

8000000

9000000

10000000

11000000

12000000

13000000

0 10000000 20000000 30000000

ARMS PHASE III 2006 – Hired Labor Expenses (Dollars)

Census 2002 

–Hired Labor Expenses (Dollars)

Scatter Plot of Hired Labor Expenses without Outliers 

0

1000000

2000000

3000000

0 2000000 4000000 6000000 8000000

ARMS PHASE III 2006 – Hired Labor Expenses (Dollars)
27

Figures A23-A24: Census 2002 versus ARMS PHASE III 2006 Fuel & Oil Expenses Census 2002 

– Fuel & Oil Expenses (Dollars)

Scatter Plot of Fuel & Oil Expenses with Outliers

0

1000000

2000000

3000000

0 1000000 2000000 3000000

ARMS PHASE III 2006 – Fuel & Oil Expenses (Dollars)

Census 2002 

– Fuel & Oil Expenses (Dollars)

Scatter Plot of Fuel & Oil Expenses without Outliers 

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

110000

120000

130000

140000

150000

160000

170000

180000

190000

0 100000 200000 300000 400000 500000 600000

ARMS PHASE III 2006 – Fuel & Oil Expenses (Dollars)
28

Figures A25-A26: Census 2002 versus ARMS PHASE III 2006 Machinery & 

Equipment Value Census 2002 

– Machinery & Equipment Value (Dollars)

Scatter Plot of Machinery & Equipment Value with Outliers

0

1000000

2000000

3000000

4000000

5000000

6000000

7000000

0 5000000 10000000 15000000

ARMS PHASE III 2006 – Machinery & Equipment Value (Dollars)

Census 2002 

– Machinery & Equipment Value (Dollars)

Scatter Plot of Machinery & Equipment Value without Outliers 

0

100000

200000

300000

400000

500000

600000

700000

800000

900000

1000000

1100000

1200000

1300000

1400000

1500000

1600000

0 1000000 2000000 3000000 4000000

ARMS PHASE III 2006 – Machinery & Equipment Value (Dollars)
29

Figures A27-A28: Census 2002 versus ARMS PHASE III 2006 Government Payments Census 2002 

– Government Payments (Dollars)

Scatter Plot of Government Payments with Outliers

0

100000

200000

300000

400000

500000

600000

700000

800000

900000

1000000

1100000

1200000

1300000

0 500000 1000000 1500000

ARMS PHASE III 2006 – Government Payments (Dollars)

Census 2002 

– Government Payments (Dollars)

Scatter Plot of Government Payments without Outliers 

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

110000

120000

130000

140000

150000

0 100000 200000 300000 400000 500000

ARMS PHASE III 2006 – Government Payments (Dollars)
30

Figures A29-A30: Census 2002 versus ARMS PHASE III 2006 Operator’s Age Census 2002 

– Operator’s Age (Years)

Scatter Plot of Operator’s Age with Outliers

10

20

30

40

50

60

70

80

90

100

- 10 0 10 20 30 40 50 60 70 80 90 100

ARMS PHASE III 2006 – Operator’s Age (Years)

Census 2002 

– Operator’s Age (Years)

Scatter Plot of Operator’s Age without Outliers 

20

30

40

50

60

70

80

90

0 10 20 30 40 50 60 70 80 90

ARMS PHASE III 2006 – Operator’s Age (Years)
31

Figures A31-A32: Census 2002 versus ARMS PHASE III 2006 Farm Type Census 2002 

– Farm Type (Nominal)

Scatter Plot of Farm Type with Outliers

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

ARMS PHASE III 2006 – Farm Type (Nominal)

Census 2002 

– Farm Type (Nominal)

Scatter Plot of Farm Type without Outliers 

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

ARMS PHASE III 2006 – Farm Type (Nominal)
32