1

United States 

Department of

Agriculture

National

Agricultural

Statistics

Service

Research and

Development Division

Washington DC 20250

April 2019

Subsampling of Nonrespondents in the 

2017 Census of Agriculture

Benjamin M. Reist

Deputy Director

Research and Development Division

Joseph B. Rodhouse

Research Statistician

Research and Development Division

Survey Methodology and Technology Section

Shane T. Ball

Section Head

Survey Methodology and Technology Section

Research and Development Division 

Linda J. Young

Chief Mathematical Statistician and Director

Research and Development Division
2

Acknowledgements

Peter Quan

Franklin Duan

Christy Meyer

Fatou Thiam

Andrew Dau

Tyler Wilson

Gavin Corral
3

Table of Contents

1. Introduction......................................................................3

2. Theory and Methodology......................................................4

3. Examples of Subsampling in Surveys........................................5

4. Nonresponse Follow-up in the Census of Agriculture.....................6

5. Subsampling in the 2017 Census of Agriculture............................8

6. Conclusion.......................................................................10

7. References.......................................................................11
4

1. Introduction

As response rates decline, the costs associated with how the survey will be designed, conducted 

and analyzed to ensure the creditability of results increase. Data collection becomes more 

difficult and costly, and questions are raised about the best way to address the increased 

nonresponse bias due to low participation rates. When resources are fixed and initial 

participation rates are low, a recommended method to achieve the best possible precision levels 

is to subsample nonrespondents (Harter et al. 2007). Subsampling, also known as double 

sampling (Neyman 1938), two-phase or sequential sampling (Cochran 1977; Lohr 1999; Harter 

et al. 2007), has been shown to be effective and efficient in collecting data from nonrespondents 

and reducing nonresponse bias (Hansen, Hurwitz, and Madow 1953; Thompson 1992; Harter et 

al. 2007). 

In many surveys, responses are initially sought using the most cost-effective design possible that 

will yield maximum results. In subsampling or two-phase designs, the initial phase of data 

collection is among all sample units, and the data collection procedure is more economical. The 

second phase is then among a probability-based subsample of nonrespondents that utilizes more 

costly, yet proven and effective, data collection procedures (Groves and Heeringa 2006; 

Thompson and Kaputa 2017). While the subsampling phase uses more costly methods, it has 

been shown to save resources overall when the likelihood of obtaining participation is low, or 

when the cost per interview increases (Elliott, Little, and Lewitzky 2000). Another feature of 

subsampling is that it can be used adaptively during data collection. Meaning, subsampling does 

not require planning before data collection begins, but rather can be implemented later in the 

process if it becomes apparent that resource constraints and nonresponse bias are imminent 

threats (Harter et al. 2007). 

The 2017 Census of Agriculture conducted by the National Agricultural Statistics Service 

(NASS) is an example where higher than normal nonresponse bias became a concern during data 

collection. The Census of Agriculture is conducted on a quinquennial basis and is the only source 

of uniform, comprehensive and impartial agricultural data for every county in the United States. 

The response rate for the Census of Agriculture has declined over consecutive collections since 

2002. Despite great efforts to increase public awareness and participation (including the addition 

of a new web mode), the 2017 Census of Agriculture response in the initial phase of data 

collection was significantly lower than reasonably anticipated given the Census of Agriculture’s 

history. As a result, the nonrespondent pool was too large for NASS’s traditional census 

nonresponse follow-up (NRFU) method in the available timeframe. Given the resource 

constraints, NASS needed to adapt its NRFU method, ultimately choosing a subsampling design 

to meet precision-related benchmarks. 

The following report provides an in-depth overview of NASS’s subsampling design to address 

nonresponse in the 2017 Census of Agriculture. The theory and methodology of subsampling in 

the survey methodological literature is presented, as well as examples of large, nationally 

representative sample surveys and censuses that use subsampling to improve response and 

quality metrics. The subsampling method developed by NASS and the conclusions drawn from 

this inaugural effort are detailed near the end of the report.
5

2. Subsampling Theory and Methodology 

Subsampling nonrespondents is a special case of multi-phase sampling. Unlike a multi-stage 

sample design in a multi-phase sample design, the probabilities of selection in each round of 

sampling are conditional on the sample drawn in the prior rounds. Design unbiased estimators 

can be constructed for multi-phase sample designs using a Horvitz-Thompson estimator (Horvitz 

and Thompson 1952). The probabilities of selection can be calculated by multiplying the first 

phase probability of selection by the conditional probabilities of selection of each subsequent 

round. Joint probabilities of selection needed to calculate variances for means and totals can be 

obtained in a similar fashion. For more details on the construction of a Horvitz-Thompson 

estimators for multi-phase designs, see chapter 9 of Särndal et al. (1992) 

Multi-phase sample designs were first proposed by Neyman (1938). Multi-phase sample designs 

were originally proposed to increase sampling efficiency in situations where there is little to no 

auxiliary information of the sampling frame. In this scenario, the first phase of sampling uses a 

simple sample design, and covariates that can be used for stratification are collected from the 

sampled units. The second phase design uses the covariates collected in the first phase in the 

sample design to sample a subset of the first phase sample. For example, the National Survey of 

College Graduates (NSCG) use the American Community Survey (ACS) as its first phase. Then 

respondents from the ACS are sampled for the NSCG using strata based on respondents response 

to the ACS such as demographics, occupation, and degree type and field) (Hall et al., 2011). 

Neyman’s multi-phase sample design methodology was adapted by Hansen and Hurwitz (1946) 

to address unit nonresponse. Hansen and Hurwitz (1946) proposed subsampling nonrespondents 

before moving to the nonresponse follow-up (NRFU). The idea was to use less expensive data 

collection methods initially and then follow-up on a subset of the nonrespondents with more 

expensive methods. This strategy fits into the multi-phase sample design framework by using the 

response indicator as a stratification variable. All respondents are assigned to one stratum and 

sampled with certainty. Nonresponses are then sampled using a standard sample design. Since it 

is unknown if a unit on the frame will respond, the probability of selection for a case in the 

nonrespondent subsample is conditional on the original sample selected. 

If 100 percent response is achieved in NRFU, then the Horvitz-Thompson estimators obtained by 

using the probabilities of selection after the nonresponse subsample is design unbiased. This 

protection from nonresponse bias is not free since nonresponse subsampling induces sampling 

variance. Rarely if ever do surveys achieve 100 percent response in NRFU even when more 

expensive methods are used. If 100 percent response in not achieved, the potential for 

nonresponse bias still exists. Nonresponse bias is a function of the response rate and the 

difference in the estimate of interest between respondents and nonrespondents. For subsampling 

of nonrespondents to reduce nonresponse bias, the data collection after subsampling must reduce 

the difference in estimates of interest between respondents and nonrespondents while increasing 

the response rate. This will only be achieved by obtaining responses from respondents not well 

represented in the original respondent pool. If methods used in NRFU do not obtain a more 

representative respondent pool, nonresponse bias may not be substantially reduced and the 
6

subsampling will induce sampling error, leading to a larger mean squared error for the estimate 

of interest. 

One criticism of subsampling nonrespondents is that it focuses on one error source, namely 

nonresponse bias (Groves, 1989). One major concern should be measurement error. As stated 

above, for subsampling of nonrespondents to be successful in reducing nonresponse bias, the 

NRFU needs to obtain responses from respondents not in the original respondent pool. If the 

methods used in the NRFU have different measurement properties than the original data 

collection methods, this differential response could induce measurement error. This is 

particularly a concern when the NRFU is conducted in a different mode than the original data 

collection. 

3. Examples of Subsampling in Surveys 

There are several prominent examples of subsampling in large nationally representative surveys 

covering individuals, households and establishments. In each of these surveys, nonresponse 

during the first phase of data collection provides the impetus for two-phase designs with followup (i.e., subsampling) among the nonrespondents. The organizations that have implemented 

subsampling range from private research organizations to academic survey research centers to 

federal statistical agencies and can be found in the methodology reviews of some of the most 

widely cited and used surveys. 

For instance, the General Social Survey (GSS), which is conducted by the National Opinion 

Research Center (NORC) at the University of Chicago, has used subsampling of nonrespondents 

over the last several iterations of the survey. The GSS is the most prominent active survey that 

monitors societal characteristics and public opinion in the United States and is widely used by 

researchers and policymakers. Typically, the nonrespondent subsamples in the GSS have been 

sampled at rates ranging from about 40 percent to 55 percent (Smith et al., 2019). The sampling 

rate is built into the subsample design, with a constant rate (e.g., 45 percent in the 2006 version 

of the GSS) used across all strata in the subsample. The subsample is weighted up to represent all 

nonrespondents, and the design allows for concentration of the remaining resources to be 

allocated toward a smaller number of hard-to-reach respondents, thereby reducing response error 

and nonresponse bias (Smith et al., 2019). 

The American Community Survey (ACS), conducted by the Census Bureau, is another 

prominent example of a large national survey that uses subsampling of nonrespondents. The 

ACS uses four modes of data collection: mail, internet, telephone and in-person interviews. The 

mail, internet and telephone modes are used in the first phase of sampling to collect data, with 

nonrespondents to the mail and internet modes being automatically assigned to telephone followup. After the telephone follow-ups, all the remaining nonrespondents then become eligible for 

the sub-sample frame, where in-person computer-assisted personal interviews (CAPI) are 

conducted. Sub-sampling rates are determined based on a variety of factors related to mailable 

and unmailable addresses (e.g., mailable addresses with predicted response rates between 0 and 

35 percent), and range from as low as 33 percent in certain subsample strata to as high as 100 

percent (i.e., in remote regions of Alaska with at least 10 percent American Indian populations) 
7

(Tersine and Starsinic, 2003). The National Survey of Family Growth (NSFG) is sponsored by 

the National Center for Health Statistics (NCHS) and conducted by the University of Michigan’s 

Survey Research Center. The NSFG uses a two-phase (or double sample) design to address 

nonresponding cases. Data collections run for about 10 weeks, and all nonresponding cases that 

have not at that point been given a final disposition code fall into a subsampling frame. The 

subsampling frame of nonresponding cases are then stratified by eligibility status and estimated 

probabilities of response. A stratified random sample of about one-third of the subsample of 

nonrespondents is selected to send back to the in-person interviewers. This subsampling rate 

allows for the NSFG to better control costs and efforts to ensure that targeted response rates are 

met, such as by allowing for larger incentives to be given to the subsampled cases for completed 

interviews (Wagner et al., 2012). 

The 2003 Survey of Small Business Finances (SSBF) is an example of an establishment survey 

that used subsampling of nonrespondents. The 2003 SSBF was conducted by the NORC on 

behalf of the Federal Reserve Board (FRB) to collect detailed information on the finances of 

small businesses, their sources and use of credit. On previous iterations of the survey, the 

sampling approach constituted what was termed a “single pass,” where all cases were sampled 

once and given an equal level of effort to get completed interviews (Potok et al., 2005). In 2003, 

the decision was made to use a “two-pass” approach that would more efficiently and effectively 

work nonresponse cases. Specifically, this approach was designed to reduce efforts to reach 

difficult cases with low probabilities of completion by prioritizing the cases with the highest 

probabilities of completion first, and then subsampling the more difficult cases for further NRFU 

methods. 

The Census Bureau investigated using adaptive NRFU strategies in the 2017 Economic Census 

(Thompson and Kaputa 2017). The goal was to develop a design for the 2017 Economic Census 

that would maintain the survey’s historic response quality while reducing costs. They found that 

one-size-fits-all subsampling of nonrespondents may not be the best approach and should be 

determined based on evidence of the best approach to follow. The conclusion was that, of the 

two approaches tested – (1) only subsampled units receive NRFU and (2) all units receive at least 

some NRFU with a subsample receiving more effective yet expensive NRFU methods – the 

NRFU design should follow the evidence available from data collection about the best 

subsampling protocol to follow, which may differ from survey to survey or population to 

population (Thompson and Kaputa 2017). 

4. Nonresponse Follow-up in the Census of Agriculture 

The Census of Agriculture has historically attained relatively high response rates, although they 

have been trending downward in recent iterations. For example, in the 2007 and 2012 Censuses 

of Agriculture, the response rates were 78.2 and 74.6, respectively. In the 2012 Census of 

Agriculture, data collection primarily revolved around mailout/mailback and was supplemented 

by Electronic Data Reporting (EDR) on the internet and personal enumeration for special classes 

of records. Follow-up rounds of data collection were used until certain benchmarks and response 

criteria were met. The initial phase of data collection began with a mailout in December 2012 of 

approximately 3 million mail packets containing a cover letter, an instruction sheet, a labeled 
8

report form and a return envelope. In February of 2013, the first round of follow-up mailouts 

were sent to the approximately 1 million nonrespondents to the initial mailout phase and EDR. 

The remaining approximately 750,000 nonrespondents to EDR, the initial mailout and follow-up 

mailout received a second round of follow-up mailouts in mid-March 2013. 

The NASS nonresponse follow-up (NRFU) typically begins after the initial three rounds of 

mailouts to all remaining nonrespondents. However, select targeted groups among the Census 

nonrespondents are usually given extra effort in the form of telephone and in-person 

enumeration. In 2012, the targeted groups were those records determined to be Suspicious Outof-Scope Records, Criteria Records, Must Cases, American Indian and Alaska Native Farm 

Operators, Low Response Counties, Last-Call Nonresponse, or Not-on-Mail List (NML). These 

groups received personal follow-up through either a call from a NASS telephone call center or an 

in-person enumerator. 

Follow-up for the Criteria Record group is one example where a method akin to subsampling 

occurred in the NRFU stage of the 2012 Census of Agriculture. Criteria Records are those for 

which the farm status has not been confirmed through a response to the National Agricultural 

Classification Survey. Some Criteria Records are recently identified potential farms, and others 

have either refused or failed to respond to the survey. These records are determined to have the 

lowest probability of being farms and responding to surveys. However, it has been important to 

enumerate these records because if they are farms, they are likely to be small farms. Therefore, it 

was necessary to identify records in the Criteria Records group that were farms in order to 

improve coverage of the small farm population. In 2012, there were 276,043 Criteria Records 

included in the Census Mail List (CML). After the returns from the initial mailout phase were 

processed, a sample of the nonrespondents with a Criteria Record label was taken for telephone 

follow-up using CATI. Of the Criteria Records nonrespondents after the initial mailout phase, 

23,739 were subsampled for CATI, with 18,831 being re-contacted by certified mail after being 

unable to be reached during CATI follow-up. The subsampling of the Criteria Records 

nonrespondents resulted in 10,887 returns from both telephone and certified mail. This equates to 

a subsampling response rate of approximately 46 percent using the American Association for 

Public Opinion Research’s Response Rate 1 (RR1) method for this group. 

Another subsampling-related procedure in the 2012 Census of Agriculture took place with the 

Low Response County Follow-up methods. This method involved NRFU to only those counties 

where the nonresponse rate was greater than 25 percent (in other words: where the response rate 

was below 75 percent). Additionally, an adaptive design method was designed to prioritize 

NRFU efforts so that response from minority operations and specialty commodity producers 

would increase. Nonresponse follow-up to this group was successful in its goal to increase the 

number of minority operations covered in the census, and to ensure that each county achieved at 

least a 75 percent response rate. Under this method, NRFU ended for a given county when a 75 

percent response rate for that county was reached. 

A third form of a subsampling-related procedure in the 2012 Census of Agriculture occurred 

with the Last Call Nonresponse Follow-up group. This group was used to push the goal of 

achieving a national response rate of 80 percent and involved targeting only the remaining 
9

nonrespondents with an expected value of sales greater than $50,000. Furthermore, only these 

records from counties with response rates below 75 percent were eligible for this NRFU method. 

These records received personal CATI enumeration, and the activity terminated for a given 

county when the county reached a 75 percent response rate. The NRFU stopping rules for Low 

Response County and Last Call Nonresponse records can be compared to the Must Case Followup group, where there virtually was no stopping rule. For these specialized records, personal 

enumeration advanced to the effort level of in-person follow-up for any remaining 

nonresponding cases until all the Must Case records were accounted for in the Census. In 2012, 

all Must Cases were ultimately accounted for as a result of this NRFU method. 

Although these methods are similar to nonresponse subsampling, these methods did not use a 

probability sampling method for which a design unbiased estimator could be formed. This meant 

that these methods could only be accounted for in the nonresponse adjustment. This is not to say 

that these methods did not increase the quality of the 2012 Census estimates since they all had 

the potential to increase the representativeness of the respondent pool and thus had the potential 

to mitigate the risk of nonresponse bias. 

5. Subsampling in the 2017 Census of Agriculture 

Consistent with the downward response rate trend seen in the Census of Agriculture from 2002 

to 2012, the 2017 Census of Agriculture experienced difficulties achieving desired response rates 

through the initial rounds of data collection, however, not for a lack of effort. For the 2017 

Census of Agriculture, NASS implemented a comprehensive public relations campaign to 

increase messaging around the Census, and a pre-notification strategy was used at the outset of 

data collection to increase awareness, improve overall response rates and encourage respondents 

to report early. For this iteration of data collection, NASS also used Computer-Assisted Self 

Interview (CASI) on the internet to provide respondents with a web mode to complete the report. 

Personal enumeration, in the form of CATI and CAPI data collection instruments, was also used 

to enumerate the special classes of records important to overall response. For CATI data 

collection, enumerators from five NASS Data Collection Centers were used. For telephone and 

in-person enumeration, enumerators under contract with the National Association of State 

Departments of Agriculture (NASDA) helped get reports from respondents. The mail packet 

preparation, initial mailout and two follow-up mailings to nonrespondents were handled by the 

U.S. Census Bureau’s National Processing Center (NPC). Additionally, all records with a known 

email address were sent notifications highlighting the improved web census form and notice that 

the mail packets would be arriving. 

Pre-notification to respondents began on November 17, 2017. Between November 27 and 

November 30, 2017, letters with survey codes and instructions for completing the census online 

were sent to approximately 1 million producers. In December 2017 and January 2018, around 3 

million producers were mailed census mail packets containing cover letters, instruction sheets, 

labeled report forms and return envelopes. The first and second rounds of follow-up mailouts 

were sent in February 2018 and March 2018 to 1.5 million nonrespondents and 1 million 

nonrespondents, respectively. Similar to earlier iterations of the census, select groups of 

nonrespondents were targeted for personal enumeration via phone and in-person. However, at 
10

this point in data collection, it became apparent that, in comparison to earlier iterations of the 

census, a new approach to nonresponse follow-up would be needed to reach desired Census 

representation criteria since the volume of cases that were identified by the 2012 methodology 

for phone follow-up was larger than the capacity of the phone centers. Instead of the seven 

targeted selected groups from the 2012 iteration of the census, the number was reduced to three 

targeted selected groups from the previous iteration (Must Cases, American Indian Producers, 

and Not on Mail List (NML)) with the rest of the remaining nonrespondents falling into the new 

National Nonresponse Follow-up group. 

The National Nonresponse Follow-up group followed a subsampling design consistent with the 

literature on subsampling theory and methods. As such, this design focused on nonresponse 

follow-up that would statistically reflect the characteristics of the nonresponders and increase 

response rates. The result was a subsample of 249,521 nonrespondents from the remaining 

864,260 nonrespondents. The subsample was selected using a stratified random design, where 

strata were based on state, county, farm size, farm type, producer race and response propensity. 

Subsample data collection ran from April 2018 through July 2018, with extensive efforts to 

achieve responses from CASI pushes, autodial calls and CATI and CAPI enumeration. The result 

was a weighted farm count of 143,847 from 51,846 in-scope completed subsample reports. 

5.1 Developing the 2017 Census of Agriculture Subsample 

Throughout the initial mail and web pushes of data collection for the 2017 Census of 

Agriculture, it was apparent that the overall response rate was lower compared to the 2012 

response rate during the same time frame. Consequently, the number of counties with 

nonresponse greater than 25 percent (under 75 percent response rate) was significantly higher 

than in 2012. Given the data collection time constraints and limited staff resources of the Data 

Collection Center (DCC), a subsampling design for follow-up of nonrespondents became 

necessary. For perspective, before data collection began, the DCC had planned for approximately 

100,000 nonrespondents in counties with less than a 75 percent response rate for follow-up data 

collection. By the time the initial three rounds of mailouts were returned and processed, there 

were more than 800,000. 

In March 2018, nonresponses that were going to be taken with certainty were sent to the phone 

follow-up. This included cases with a large estimated Total Value of Production, including 

Government Payments (TVPG), cases that potentially had aquaculture, cases needed for 

coverage adjustment estimation and cases with a high Measure of Priority (MOP). The MOP is a 

measure intended to address undercoverage of certain populations, such as small farms and 

minority and women-owned operations, to name a few. Therefore, operations estimated to 

address undercoverage of targeted subpopulations of farms are given a higher priority score (or, 

higher MOP) than nonresponding operations estimated to resemble operations with generally 

good coverage. The MOP values were 0, 5, 10 and greater than 10 (with 0 being the lowest 

priority and greater than 10 being the highest). The remaining cases continued to be followed up 

using a combination of robocalls and EDR invite letters. 
11

In late May 2018, a sample of approximately 110,000 cases was selected from the 

nonrespondents who had not been taken with certainty. These cases were sent an additional 

questionnaire and then sent to the phone follow-up. These cases were selected using a stratified 

simple random sample design. Strata were defined by county, MOP and estimated propensity to 

respond. A bootstrap random forest model fit on Census 2012 data was used to estimate the 

propensity to respond. These strata were chosen to both target cases identified as important to the 

final estimates and the desire to increase the number of completed cases. 

The sample was allocated using an optimal Neyman allocation for TVPG with a cost based on 

the average propensity to respond and average MOP of the stratum. The Neyman allocation was 

adjusted to ensure that no case would have a weight greater than 10. Also, the sample size was 

increased for counties that had large coefficients of variations after the Neyman allocation. 

Finally, some cases were grouped for phone follow-up since they had an operator in common. In 

these cases, if operators were contacted by phone, they would be asked to respond for both cases. 

Because of this, if one of these cases was selected, the other case was added to the sample. 

5.2 Sampling Weights for the 2017 Census of Agriculture 

Cases that were sampled were assigned a sampling weight based on the cases inverse probability 

of selection. These weights were then adjusted at the end of data collection for a few different 

reasons: (1) Cases not in the subsample that responded after subsampling. These cases were 

given a weight of one, and cases sampled in the stratum containing this case were adjusted by 

removing the case from the population size of the stratum. (2) Sampled cases that responded 

before June 1, 2018. These cases responded before they would have received the questionnaire, 

and thus it was assumed that they would have responded even if they would not have been 

subsampled. The weight of these cases was set to one, and cases sampled in the stratum 

containing this case were adjusted by removing the case from the population size of the stratum 

and the sample size of the stratum. (3) Cases that were linked to another case in phone follow-up. 

As discussed above, weights were adjusted to account for the fact that there were two ways that 

they could have been included in the sample. 

6. CONCLUSION 

Census results can be powerful, because they serve as a foundation of farm policy and people 

make decisions based on this information. Declining response rates have increased the costs 

associated with how the Census is designed, conducted and analyzed to ensure the creditability 

of these results. 

This report provided an in-depth overview of NASS’s subsampling design to address 

nonresponse in the 2017 Census of Agriculture. The theory and methodology of subsampling in 

the survey methodological literature is presented, as well as examples of large, nationally 

representative sample surveys and censuses that use subsampling to improve response and 

quality metrics. 
12

Subsampling in the 2017 Census focused on nonresponse follow-up that would statistically 

reflect the characteristics of the nonresponders and increase response rates. The result was a 

weighted farm count of 143,847 from 51,846 in-scope completed subsample reports.

7. REFERENCES 

Cochran, W.G. (1977). Sampling Techniques, 3rd Edition. New York: John Wiley & Sons. 

Elliott, M.R., R.J.A. Little, and S. Lewitzky (2000). “Subsampling Callbacks to Improve Survey 

Efficiency.” Journal of the American Statistical Association, 95 (451), pp. 730-738. 

Groves, R.M. (1989). Survey Errors and Survey Costs. New York: John Wiley & Sons. 

Groves, R. M., and Heeringa, S. (2006). "Responsive design for household surveys: tools for 

actively controlling survey errors and costs." Journal of the Royal Statistical Society Series A: 

Statistics in Society, 169, pp. 439-457. 

Hansen, M.H. and W.N. Hurwitz (1946). The Problem of Non-Response in Sample Surveys. 

Journal of the American Statistical Association, 41 (236), pp. 517-529. 

Hansen, M.H., W.N. Hurwitz, and W.G. Madow (1953). Sample Survey Methods and Theory, 

Vol. I: Methods and Applications. New York: John Wiley & Sons. 

Harter, R. M., March, T. L., Chapline, J. F., and Wolken, J. D. (2007) “Determining 

Subsampling Rates for Nonrespondents” Proceeding of the third International Conference on 

Establishment Surveys on Establishment Surveys. Montréal, Quebec, Canada. 1293-1300. 

Hall, D., Cohen, S., Finamore, J. and Lan, F. (2011) “NSCG Sampling Issues When Using an 

ACS-Based Sampling Frame.” ASA Proceedings of the Joint Statistical Meetings, Miami Beach, 

FL pp. 3955-3963. 

Horvitz, D. G., Thompson, D. J. (1952) "A generalization of sampling without replacement from 

a finite universe", Journal of the American Statistical Association, 47, 663–685. 

Lohr, S.L. (1999). Sampling: Design and Analysis. Pacific Grove: Duxbury Press. 

Neyman, J. (1938). .Contribution to the Theory of Sampling Human Populations, Journal of the 

American Statistical Association, 33, pp. 101-116. 

Potok, N., Bailey, R., Sherman, B., Harter, R., Yang, M., Chapline, J., and Bartolone, J. (2015) 

The 2003 Survey of Small Business Finances: Methodology Report. NORC, Chicago, IL. 

Särndal, C.-E., Swensson, B., & Wretman, J. (1992). Springer series in statistics. Model assisted 

survey sampling. New York, NY, US: Springer-Verlag Publishing. 

Smith, T. W., Davern, M., Freese, J., and Hout, M. (2019) General Social Surveys, 1972-2018: 

cumulative codebook. NORC, Chicago, IL. 
13

Tersine, A. and Starsinic, M. (2003) “Optimum Nonresponse Subsampling Rate for the 

American Community Survey”. ASA Proceedings of the Joint Statistical Meetings, pp. 4205-

4211. 12 

Thompson, J.K. and Kaputa, S.J. (2017) “Investigating Adaptive Nonresponse Follow-up 

Strategies for Small Businesses through Embedded Experiments” Journal of Official Statistics, 

33, pp.835-856 

Thompson, S.K. (1992). Sampling. New York: John Wiley & Sons. 

Wagner, B. T., West, B. T., Kirgis, N., Lepkowski, J. M., Axinn, W. G., and Ndiaye, S. K. 

(2012) Journal of Official Statistics. 28, 477-499.