1

Do Names Matter? Experiments Comparing Different Branding and

Levels of Personally Identifiable Information in a Mail 

Questionnaire 

Sarah Hastedt Carroll 

National Center for Education Statistics 

 

 

Sarah.Carroll@ed.gov

Andrew Zukerberg 

National Center for Education Statistics 

Andrew.Zukerberg@ed.gov

Proceedings of the 2013 Federal Committee on Statistical Methodology (FCSM) Research Conference 

 

This paper is intended to promote the exchange of ideas among researchers and policy makers. The views 

expressed in this paper are part of ongoing research and analysis and do not necessarily reflect the position 

of the U.S. Department of Education. 

Introduction 

The survey methodology literature has long looked at ways to increase response in mail surveys. Two areas that 

have been of interest are the effects of sponsorship and level of anonymity (see for example: Jones 1979, Heberlein 

and Baumgartner 1978, and De Leeuw et al. 2007). Government sponsorship and increased anonymity are thought 

to improve response rates (Heberlein and Baumgartner 1978, Jones 1979, Dillman 2007). However, we are not 

aware of research that directly compares the impact of sponsorship by different Federal agencies on response rates. 

This paper presents results from an experiment designed to look at these issues that was conducted as part of the 

2012 National Household Education Surveys Program (NHES:2012). The experiment used a 2x2 factorial design. 

As part of the first treatment in the experiment, households were randomly assigned to receive materials 

emphasizing the Census Bureau’s involvement in the NHES:2012 data collection (Census Bureau “branding”) or

materials emphasizing Department of Education (ED) sponsorship of the NHES:2012 (ED “branding”). For the 

second treatment in the experiment, households were randomly assigned to receive a screener questionnaire that 

asked for the names of children living in the household as well as demographic information or a screener 

questionnaire that only collected the demographic information. This experiment also allows us to examine the 

interaction between the two treatments. The analysis focuses on differences in response rates between respondents in 

the different treatment groups, but is extended to subgroup comparisons in some cases using demographic data 

available on the sample frame. 

Background 

The National Household Education Surveys Program (NHES) is the primary household survey sponsored by the 

National Center for Education Statistics (NCES). The NHES uses rotating topical survey modules to collect 

information on a variety of topics that are difficult to address in school-based surveys such as homeschooling, early 

childhood care and education, school choice, and adult education. The NHES was conducted as a landline random 

digit dial (RDD) survey using computer-assisted telephone interviewing (CATI) approximately every two years 

from 1991 through 2007. Following the 2007 data collection, the NHES underwent a methodological redesign due

to declining response rates1 and concerns about population coverage using the landline telephone frame.2

As part of the NHES redesign, the data collection mode changed from an RDD CATI survey to an address-based 

sample (ABS) with data collection using self-administered surveys conducted through the mail. Cognitive testing 

1

 Response rates to the first phase of the survey fell from 81% in 1991 to 53% in 2007 (Van de Kerckhove et. al. 2008). 

2

 In 2007 the percentage of households with cell phone only telephone service was 15.8 percent and in 2012 that percentage had 

increased to 38.2 percent. (Blumberg and Luke 2009 and Blumberg and Luke 2013).

This paper is intended to promote the exchange of ideas among researchers and policy makers. The views 

expressed during discussion and in this presentation are part of ongoing research and analysis and do not 

necessarily reflect the position of the U.S. Department of Education. 

pap 

This paper is intended to promote the exchange of ideas among researchers and policy makers. The views 

expressed during discussion and in this presentation are part of ongoing research and analysis and do not 

necessarily reflect the position of the U.S. Department of Education. 
2

was conducted on the questionnaires and mailing materials during initial development, followed by a feasibility test 

of the new design in 2009 and a field test in 2011. The results of these tests provided guidance on the design for the 

full scale NHES:2012 data collection. The NHES:2012 was a two-phase mail survey. In the first phase, called the 

“screener” phase, sampled households were mailed a brief screener survey designed to identify households with 

eligible children for the second phase of the survey, referred to as the “topical” phase. Households were asked to 

enumerate children age 20 or younger living in the household and provide basic demographic information about 

each child (including age, sex, and school enrollment status). In households that returned the screener survey and 

contained at least one eligible child, one child was sampled for a topical follow-up survey. The topical survey was a 

longer survey that collected more detailed information about the early care and education of the focal child. Children 

age six or younger who were not yet in kindergarten were eligible for the Early Childhood Program Participation 

Survey (ECPP) and children age 20 or younger who were enrolled in public or private school or homeschooled for 

kindergarten through twelfth grade were eligible for the Parent and Family Involvement in Education Survey (PFI). 

Children’s eligibility for either topical survey was determined by the information provided on the screener survey. 

Parents of children sampled for the topical survey were mailed the survey and asked to respond. At both the screener 

and topical phase, up to three nonresponse mailings after the initial mailing were sent to households in an effort to 

maximize response rates. 

The NHES:2012 contained a screener questionnaire experiment designed to look at ways to improve coverage and 

response rates in future administrations of the survey. The screener experiment used a 2x2 factorial design. The first 

treatment varied the “branding” used for the survey. The second treatment varied whether or not the screener survey 

included a request for respondents to provide children’s names on the survey. Households were randomly assigned 

to different treatment groups. The following paragraphs describe the experimental treatments in more detail. 

Prior to 2012, the NHES had been conducted by Westat on behalf of ED. As noted previously, research literature 

has suggested that highlighting government sponsorship of a data collection improves response rates over private 

sector sponsorship (Heberlein and Baumgartner 1978, Dillman 2007). For this reason the past NHES materials 

emphasized ED’s sponsorship of the study. In 2012, data collection for the NHES moved to the U.S. Census 

Bureau. This presented the opportunity to highlight the Census Bureau’s role in data collection. The first 

experimental treatment involved random assignment of households to receive survey materials featuring the Census 

Bureau as the primary brand or survey materials using the Department of Education as the primary brand. Elements 

of the primary survey brand included the use of the primary brand agency’s official seal on respondent materials and 

greater prominence of the primary brand agency’s name on respondent materials. Materials with the Census Bureau 

as the primary brand contained references to the Department of Education as the sponsor of the survey and materials 

with the Department of Education as the primary brand indicated that the Census Bureau was collecting data for the 

Department of Education. However, these references were much less prominent (in the text of a letter or in the 

authorization and confidentiality text on the survey forms) than the primary brand’s name and seal. 

We hypothesized that the Census Bureau could have greater name recognition as a result of contacting every 

household in the U.S. for the Decennial Census, and many households in the U.S. for the American Community 

Survey (ACS), and that this would generate higher response rates compared to surveys with the Department of 

Education as the primary brand. Alternatively, given the survey’s focus on education and the target population of 

households with children, using the Department of Education as the primary brand may increase the perceived 

legitimacy of the survey request and generate higher response rates. The basic content of the survey materials was 

the same regardless of the branding and the screener survey questions did not vary by primary brand. Exhibit 1 

shows the advance letters with the Department of Education as the primary brand or the Census Bureau as the 

primary brand. Letters with ED as the primary brand were printed on Department of Education letterhead and were 

signed by the NCES Commissioner. Letters with the Census Bureau as the primary brand were printed on Census 

Bureau letterhead and were signed by the Director of the Census Bureau. The mailing envelopes differed by brand 

as shown in exhibit 2. The envelopes for materials with the Department of Education as the primary brand had a 

large image of the Department of Education’s seal printed in light grey below the address window, while the 

envelope for materials with the Census Bureau as the primary brand contained the Census Bureau’s official logo in 

small print along the bottom of the envelope. Both types of envelopes were printed on identical paper stock, had the 

same return address, and had text stating that they were for official business only. Exhibit 3 shows the cover page of 

the screeners with the Department of Education and Census Bureau as the primary brand. The cover pages of the 

surveys included either the Department of Education seal or the Census Bureau seal and the respective agency’s 

name in a banner across the top. The inside of the questionnaire of both primary brands (shown in exhibit 4) 
3

contained a banner across the top left with the following text: National Household Education Survey. On screeners 

with the Department of Education as the primary brand, U.S. Department of Education National Center for 

Education Statistics was printed on the top right of the questionnaire. Screeners with the Census Bureau as the 

primary brand had the statement “conducted for:” placed above the U.S. Department of Education line (this 

difference can be seen in exhibit 4). 

 

Exhibit 1. NHES:2012 advance letters, by primary brand 

Department of Education Census Bureau 

Department of Education Census Bureau 

Exhibit 2. Outgoing screener envelopes, by primary brand 
4

Exhibit 3. NHES:2012 screener covers, by primary brand 

Department of Education Census Bureau

Exhibit 4. Inside pages of NHES:2012 screeners, by primary brand 

Department of Education

Census Bureau 
5

The second experimental treatment tested the effect of requesting each child’s name, initials or nickname on the 

screener survey compared to not requesting this information. Exhibit 5 shows the NHES:2012 screener surveys with 

and without the request for the child’s name. In addition to the potential request for the child’s name, the screener 

survey contained questions asking for each child’s age, sex, school enrollment status, and grade, if the child was 

enrolled in school. Collecting each child’s name in the household at the screener allows the topical survey materials 

to be personalized. If the child’s name is available, the child’s name is used in the topical cover letters and printed 

on the topical questionnaire to remind parents to focus on a particular child when responding to the survey. A 

similar experiment was conducted during a 2011 field test. At the time, there appeared to be a slightly lower 

response rate at the screener level when name was collected but a higher response rate at the topical level when 

name was used. Unfortunately, that experiment was confounded with a request for telephone number which may 

have been perceived as sensitive by respondents (McPhee and Mamedova 2012). Parents may perceive the request 

for the child’s name as sensitive, particularly in combination with other characteristics about the child collected in 

the screener. This may discourage parents from responding and depress response rates. 

Exhibit 5. NHES:2012 screeners, with and without children’s names 

Screener with name 

Screener without name
6

Data 

The NHES:2012 data collection was conducted from January through August 2012 using an address-based sample 

of approximately 159,990 addresses in the United States. The NHES:2012 sample was a two-stage stratified sample. 

In the first stage, residential addresses were sampled and sent a brief screener survey to determine if the household 

had an eligible child for the second stage topical surveys. Households returned completed questionnaires to the 

Census Bureau and in the second stage an eligible child was selected based on information provided on the screener 

survey. Households with an eligible child were mailed the appropriate topical survey. Census tracts with a high 

percentage of Black and Hispanic households were oversampled in the NHES:2012 in order to provide more reliable 

estimates by race and ethnicity. Addresses were assigned to the following three strata: (1) Census tracts with 25 

percent or more Black persons (Black stratum), (2) Census tracts with 40 percent or more persons of Hispanic origin 

(Hispanic stratum), and (3) all other tracts (All Other stratum). Twenty percent of the sample was allocated to the 

Black stratum, 15 percent of the sample was allocated to the Hispanic stratum, and 65 percent of the sample was 

allocated to the All Other stratum. 

 

This paper uses data available at the screener level of the survey. The information available at the screener level is 

primarily from the sample frame. In addition to basic address information, the sample frame vendor, Marketing 

Systems Group (MSG), appended demographic data to the addresses included in the sample. These variables were 

drawn from commercial databases to provide supplemental information about the addresses included in the sample. 

The characteristics examined in this paper from the appended frame data are whether or not the address is associated 

with a Hispanic surname, whether or not a phone number was associated with the address, the education level of the 

head of the household, the home ownership status of the address, household income, and gender of the head of the 

household. 

Findings 

This paper focuses on the results of these experimental treatments at the screener level because it is where they are 

expected to be the most salient for respondents. For the branding treatment, the screener phase is where the survey 

sponsorship is established. If survey sponsorship influences respondent decisions regarding whether or not to 

participate, we would expect the effect to start at the screener level where respondents are first introduced to the 

survey. For the name/no-name treatment, the screener contact is where the request for this potentially sensitive 

information is made. If respondents are making decisions about whether or not to participate in the survey based on 

the perceived sensitivity of the information requested in the initial contact, we would expect to see an effect at the 

point where the request is first made. 

Table 1 presents screener response rates and the number of cases in different response status categories by screener 

treatment. Respondents are cases that returned a non-blank screener form. Within the primary branding 

experimental treatment, a higher percentage of cases were assigned to receive materials with the Department of 

Education as the primary brand (approximately 80%). This allocation was used because only Department of 

Education materials had been tested as part of the development work for the NHES:2012. Given that these materials 

had achieved acceptable response rates and Census Bureau materials had not been tested in this context, a higher 

proportion of cases were allocated to receive materials with the Department of Education as the primary brand in 

case materials with the Census Bureau as the primary brand received lower response rates. A higher proportion of 

cases within the name/no name treatment were assigned to receive surveys requesting the child’s name 

(approximately 80%). In a previous field test, there appeared to be a slightly lower response rate at the screener level 

when name was collected but a higher response rate at the topical level when name was used. However, that 

experiment was confounded by the request for a phone number on the screener form (McPhee and Mamedova 

2012). Due to the expected higher response rate at the topical level, a higher proportion of cases were assigned to 

receive screeners that asked for the child’s name. The overall base-weighted response rate to the screener was 71 

percent. The response rate did not vary according to whether or not the screener included a request for the child’s 

name. The screener response rate did differ by primary questionnaire brand. Forms that had the Census Bureau as 

the primary brand had a five percentage point higher response rate than forms with the Department of Education as 

the primary brand (75% Census compared with 70% ED). 
7

Table 1. Screener response rates and response status, by screener treatment 

Screener Type Response Status 

 

Response 

Rate Responded Total 

Total 71 99,570 159,990 

 Primary Brand 

 

 Department of Education 70 79,620 129,980 

 Census Bureau 75* 19,950 30,020 

Level of PII 

 

 No name 71 18,760 30,020 

 With name 71 80,810 129,980 

* Significant at p<.05 between Department of Education and Census Bureau primary branding. 

Note: Response rates calculated using base weights. Sample sizes rounded to the nearest 10. Detail may not sum to 

total due to rounding. 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Surveys Program of 2012 (NHES:2012). 

Table 2 presents screener response rates and the number of cases in different response status categories by the 

interaction of the two treatments, questionnaire branding and request for the child’s name. Within the surveys that 

had the Department of Education as the primary brand, the screener that did not include the request for the child’s 

name had a slightly higher response rate compared to the forms that did request name. The response rate did not 

vary according to whether or not the screener included a request for the child’s name within forms that had the 

Census Bureau as the primary brand. 

Table 2. Screener response rates and response status, by screener treatment interaction 

 

Screener Type Response Status 

Response Rate Responded Total 

Total 71 99,570 159,990 

 Department of Education – No

name 

71* 15,060 24,390 

Department of Education – With 

name 

70 64,560 105,590 

Census Bureau – No name 75 3,700 5,630 

Census Bureau – With name 76 16,250 24,390 

* Significant at p<.05 between the name and no name conditions with Department of Education as the primary 

brand. 

Note: Response rates calculated using base weights. Sample sizes rounded to the nearest 10. Detail may not sum to 

total due to rounding. 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Surveys Program of 2012 (NHES:2012). 

Table 3 looks at the percentage of respondents with and without eligible children by screener type. Eligible children 

are defined as children age 20 or younger who were enrolled in public or private school or homeschooled for 

kindergarten through twelfth grade or not yet enrolled in kindergarten. The percentage of respondents with eligible 

children to the form with the Department of Education as the primary brand was one percentage point higher than 

the percentage of respondents with eligible children to the form with the Census Bureau as the primary brand. The 

percentage of respondents with children to the without name screener form was one percentage point higher than the 

percentage of respondents with children to the with name form. When the interaction between primary screener 

brand and name is examined, the only statistically significant difference is that a higher percentage of respondents to 
8

the survey with the Department of Education as the primary brand that did not request the child’s name had eligible 

children compared to respondents to the form with the Census Bureau as the primary brand that did request the 

child’s name. All of the statistically significant differences noted were relatively small in magnitude, one to two 

percentage points, but there is some evidence that materials with the Department of Education as the primary brand 

yielded a slightly higher percentage of respondents in the target population of interest, those with eligible children. 

Table 3. Percentage of respondents with and without eligible children, by screener type 

 

 

Screener Type Percentage of respondents 

No eligible children 

reported 

At least one eligible child 

reported 

Total 66 34

Primary Brand 

 Department of Education 65 35* 

 Census Bureau 66 34

 Level of PII 

 

 No name 65 35*

 With name 66 34

Primary Brand and Level of PII Interaction 

 Department of Education – No name 65 35*

 Department of Education – With name 66 34

 Census Bureau – No name 66 34

 Census Bureau – With name 67 33

* Significant at p<.05 for respondents with eligible children. Differences are between primary brands, level of PII 

requested, and for the treatment interaction group between the Department of Education no name request and 

Census with a name request. 

Note: Percentages calculated using base weights. 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Surveys Program of 2012 (NHES:2012). 

Next, we look at differences in response rates to the different screener types by selected address frame

characteristics. Table 4 presents screener response rates by whether or not the screener survey requested children’s 

names and selected frame characteristics. Across the frame characteristics examined, the only differences in 

response rates observed were that the response rate was higher for the screener survey that did not request children’s 

names in the stratum composed of census tracts with 25 percent or more Black persons, among households where 

the head of the household had a bachelor’s degree, and households headed by males. 

 
9

Table 4. Base-weighted screener response rates, by presence of name request and selected frame characteristics 

Characteristic With name Without name 

Total 71 71

Stratum 

 Tracts with 25% or more Black persons 61 62* 

 Tracts with 40% or more Hispanic persons 62 63

 All other tracts 74 74

Hispanic surname 

 Yes 63 64

 No 72 72

Phone number available 

 Yes 79 80

 No 64 65

Education level of head of household 

Less than high school 67 67

 High school 76 75

 Some college 72 73

 Bachelor's degree 79 81* 

 Graduate degree 82 82

 Missing on frame 65 66

Home ownership status 

 Own 78 78

 Rent 62 62

 Missing 56 57

Household income 

 $0-$10,000 63 65

 $10,001-$20,000 63 64

 $20,001-$30,000 65 64

 $30,001-$40,000 68 68

 $40,001-$50,000 71 72

 $50,001-$60,000 73 75

 $60,001-$75,000 75 75

 $75,001-$100,000 77 78

 $100,001-$150,000 81 81

 $150,001 or more 82 84

 Missing 55 56

See notes at end of table. 
10

Table 4. Base-weighted screener response rates, by presence of name request and selected frame characteristics – 

Continued 

Characteristic With name Without name 

Sex of head of household 

 Male 75 76* 

 Female 70 70

 Missing 56 56

* Significant at p<.05 between with name and without name. 

Source: U.S. Department of Education, National Center for Education Statistics, National 

Household Education Surveys Program of 2012 (NHES:2012). 

Table 5 shows that the screener with the Census Bureau as the primary brand had a significantly higher response rate 

across all of the demographic groups we looked at. This suggests that using the Census Bureau as the primary brand 

had broader appeal than using ED as the primary brand. 

Table 5. Base-weighted screener response rates, by primary survey brand and selected frame characteristics 

Characteristic Department of Education Census Bureau

Total 70 75* 

 Stratum 

 Tracts with 25% or more Black persons 60 66* 

 Tracts with 40% or more Hispanic persons 61 66* 

 All other tracts 73 78* 

Hispanic surname 

 Yes 62 67* 

 No 71 76* 

 Phone number available 

 Yes 78 83* 

 No 63 69* 

 Education level of head of household 

 Less than high school 65 72* 

 High school 74 80* 

 Some college 72 77* 

 Bachelor's degree 78 83* 

 Graduate degree 81 86* 

 Missing on frame 64 69* 

Home own ership status 

 

 Own 77 82* 

 Rent 61 67* 

 Missing 56 61* 

See notes at end of table. 

 
11

Table 5. Base-weighted screener response rates, by primary survey brand and selected frame characteristics 

 Characteristic Department of Education Census Bureau 

Household income 

 

 $0-$10,000 62 70* 

 $10,001-$20,000 62 68* 

 $20,001-$30,000 63 69* 

 $30,001-$40,000 67 73* 

 $40,001-$50,000 70 76* 

 $50,001-$60,000 72 78* 

 $60,001-$75,000 74 80* 

 $75,001-$100,000 76 81* 

 $100,001-$150,000 80 85* 

 $150,001 or more 82 87* 

 Missing 54 60* 

Sex of head of household 

 

 Male 75 80* 

 Female 68 75* 

 Missing 55 60* 

* Significant at p<.05 between Department of Education and Census Bureau primary branding across characteristics. 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Survey Program of 2012 (NHES:2012). 

 

Discussion 

In this paper we looked at the impact of two naming conditions on response to a general population survey. The first 

condition is the impact of highlighting sponsorship by different Government agencies on response. Past literature 

has shown that government sponsorship can increase response rates to a survey, but has not explored whether this 

difference is impacted by the government agency sponsoring the study (Heberlein and Baumgartner 1978, Dillman 

2007). Often this is because it is not possible to change the sponsorship of the study. The NHES is sponsored by 

the U.S. Department of Education and had previously been conducted by Westat, a private research firm. In 2012, 

the NHES was conducted for ED by the U.S. Census Bureau. Transitioning to the Census Bureau provided an 

opportunity to look at the relative impact of highlighting ED sponsorship of the study compared with the Census 

Bureau role in data collection. The results showed that materials utilizing the Census Bureau as the primary survey 

brand consistently achieved a 5 percentage point higher response rate than those emphasizing ED as the primary 

survey brand. The increased response held across all groups which were looked at using extant data provided by the 

frame vendor. There are many possible explanations for the difference. In preparation for the 2010 Decennial 

Census, the Census Bureau undertook a 140 million dollar advertising campaign to promote3 the Census. In 

addition to this campaign, the Census Bureau received support from organizations and state and local governments 

in raising awareness about the data collection. It is possible that these efforts had a lasting effect on encouraging 

response to Census Bureau data collections. Another explanation may lay in the branding itself. In a test of Census 

materials that compared different designs of a Decennial questionnaire, Leslie (1997), found that a more official 

looking design yielded a significantly higher response rate than one designated as “respondent-friendly.” This 

difference occurred in spite of the fact that both forms indicated that the Census Bureau was conducting the study. It 

is possible that differences in elements of the Census Bureau branding, particularly the different envelope design, 

compared to elements of the ED branding impacted the response rate. The ED materials were developed through 

many years of qualitative testing and two rounds of field tests. However, it is possible that an alternate ED design 

may result in a higher response rate as occurred in the comparison of Census designs described by Leslie (1997). 

We plan to conduct additional analysis at the topical level to look at the impact of branding on nonresponse bias. 

 

3

 See: http://www.census.gov/2010census/news/pdf/advertising_budget.pdf
12

 

The second experimental treatment looked at the impact of collecting children’s names as part of a two-stage selfadministered data collection. Prior testing with smaller samples on the NHES yielded mixed results and we wanted 

to better understand the impact of collecting names on the survey. The results indicate that at the screener level, 

collecting child’s name did not impact the overall response rate and had less than a 2 percentage point impact on 

response rates among a few subgroups (the stratum composed of census tracts with 25 percent of more Black 

persons, among households where the head of the household had a bachelor’s degree, and households headed by 

males). This suggests that within the context of a screener survey that does not collect sensitive information like 

telephone numbers, asking child’s name is not problematic from a response rate perspective. 

References 

Blumberg S.J., Luke J.V. (2013) Wireless substitution: Early release of estimates from the National Health 

Interview Survey, July–December 2012. National Center for Health Statistics. Available from: 

http://www.cdc.gov/nchs/nhis.htm.

Blumberg S.J. and Luke J.V. (2009) Wireless substitution: Early release of estimates from the National Health 

Interview Survey, January-June 2009. National Center for Health Statistics. Available from: 

http://www.cdc.gov/nchs/nhis.htm. 

. 

De Leeuw, E., Callegaro, M., Hox, J., Korendijk, E. and Lensvelt-Mulders, G. (2007). “The Influence of Advance 

Letters on Response in Telephone Surveys: A Meta-Analysis.” Public Opinion Quarterly 71(3): 413-443 doi: 

10.1093/poq/nfm014 

Dillman, D. (2007). Mail and Internet Surveys: The Tailored Design Method, 2007 Update with New Internet, 

Visual, and Mixed-Mode Guide. Hoboken, New Jersey: John Wiley & Sons. 

Heberlein, T.A. and R. Baumgartner (1978) "Factors affecting response rates to mailed questionnaires: a quantitative 

analysis of the published literature." American Sociological Review 43:447-62.

Jones, W.H. (1979). Generalizing Mail Survey Inducement Methods: Population Interactions with Anonymity and 

Sponsorship.” Public Opinion Quarterly 43 (1): 102-111 doi:10.1086/268495 

Leslie, T.F. (1997) “Comparing two approaches to questionnaire design: official government versus public 

information design.” Proceedings of the Survey Research Section of the American Statistical Association. Available 

at: http://www.amstat.org/sections/srms/Proceedings/

McPhee, C. and Mamedova, S. (2012). “Return to Sender: Improving Response Rates in Two Stage Mail Surveys.” 

Paper presented at the 2012 Federal Committee on Statistical Methodology Research Conference. Washington, DC. 

Paper available at: http://www.fcsm.gov/12papers/Mamedova_2012FCSM_II-C.pdf

Van de Kerckhove, W., Montaquila, J.M., Carver, P. R., and Brick, J.M. (2008). An Evaluation of Bias in the 2007 

National Households Education Surveys Program: Results From a Special Data Collection Effort (NCES 2009–

029). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

Washington, DC 
13

Appendix A. Standard Error Tables 

 

Table A-1. Standard errors for table 1: Screener response rates and response status, by screener treatment 

Screener Type Response Status 

Response Did not Responded Ineligible Total Rate respond 

Total 0.1 † † † † 

 

Primary Brand 

 Department of Education 0.1 † † † † 

 Census Bureau 0.3 † † † † 

Level of PII 

 No name 0.3 † † † † 

 With name 0.1 † † † † 

† Not applicable. 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Surveys Program of 2012 (NHES:2012). 

Table A-2. Standard errors for table 2: Screener response rates and response status, by screener treatment interaction 

Screener Type Response Status 

Response Did not Responded Ineligible Total Rate respond 

Total 0.1 † † † † 

 

Department of Education – No name 0.3 † † † † 

Department of Education – With name 0.2 † † † † 

Census Bureau – No name 0.6 † † † † 

Census Bureau - With name 0.3 † † † † 

† Not applicable 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Surveys Program of 2012 (NHES:2012). 
14

Table A-3. Standard errors for table 3: Percentage of respondents with and without eligible children, by screener 

type 

Screener Type Percentage of respondents 

No eligible children 

reported 

At least one eligible child 

reported 

Total 0.2 0.2 

 

Primary Brand 

 Department of Education 0.2 0.2 

 Census Bureau 0.3 0.3 

Level of PII 

 No name 0.4 0.4 

 With name 0.2 0.2 

Primary Brand and Level of PII Interaction 

 Department of Education – No name 0.4 0.4 

 Department of Education – With name 0.2 0.2 

 Census Bureau – No name 0.4 0.4 

 Census Bureau – With name 0.8 0.8 

Note: Percentages calculated using base weights. 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Surveys Program of 2012 (NHES:2012). 

 
15

Table A-4. Standard errors for table 4: Base-weighted screener response rates, by presence of name request and 

selected frame characteristics 

Characteristic With name Without name 

Total 0.1 0.3 

Stratum 

Tracts with 25% or more Black persons 0.3 0.7 

Tracts with 40% or more Hispanic persons 0.4 0.8 

All other tracts 0.2 0.3 

Hispanic surname 

Yes 0.5 0.9 

No 0.1 0.3 

Phone number available 

Yes 0.2 0.4 

No 0.2 0.4 

Education level of head of household 

Less than high school 0.4 0.9 

High school 0.3 0.7 

Some college 0.3 0.7 

Bachelor's degree 0.4 0.7 

Graduate degree 0.4 0.9 

Missing on frame 0.2 0.5 

Home ownership status 

Own 0.2 0.3 

Rent 0.3 0.7 

Missing 0.3 0.7 

Household income 

$0-$10,000 0.9 1.8 

$10,001-$20,000 0.6 1.2 

$20,001-$30,000 0.5 1.0 

$30,001-$40,000 0.5 .9

$40,001-$50,000 0.4 .9

$50,001-$60,000 0.4 .9

$60,001-$75,000 0.4 .8

$75,001-$100,000 0.4 .7

$100,001-$150,000 0.4 .7

$150,001 or more 0.5 1.0 

Missing 0.4 0.8 

See notes at end of table. 
16

Table A-4. Standard errors for table 4: Base-weighted screener response rates, by presence of name request and 

selected frame characteristics– Continued 

Characteristic With name Without name 

Sex of head of household 

Male 0.2 0.3 

Female 0.3 0.5 

Missing 0.4 0.8 

Source: U.S. Department of Education, National Center for Education Statistics, National 

Household Education Surveys Program of 2012 (NHES:2012). 

 
17

Table A-5. Standard errors for table 5: Base-weighted screener response rates, by primary survey brand and selected 

frame characteristics 

Characteristic Department of Education Census Bureau

Total 0.1 0.3 

 

Stratum 

Tracts with 25% or more Black persons 0.3 0.7 

Tracts with 40% or more Hispanic persons 0.4 0.7 

All other tracts 0.2 0.3 

 

Hispanic surname 

Yes 0.5 0.9 

No 0.1 0.3 

 

Phone number available 

Yes 0.2 0.2 

No 0.3 0.4 

 

Education level of head of household 

Less than high school 0.4 0.8 

High school 0.3 0.6 

Some college 0.3 0.6 

Bachelor's degree 0.4 0.7 

Graduate degree 0.5 0.8 

Missing on frame 0.2 0.5 

 

Home ownership status 

Own 0.2 0.3 

Rent 0.2 0.7 

Missing 0.3 0.7 

 

Household income 

$0-$10,000 0.9 1.7 

$10,001-$20,000 0.6 1.2 

$20,001-$30,000 0.5 1.0 

$30,001-$40,000 0.5 0.9 

$40,001-$50,000 0.4 0.9 

$50,001-$60,000 0.4 0.8 

$60,001-$75,000 0.4 0.7 

$75,001-$100,000 0.4 0.7 

$100,001-$150,000 0.4 0.7 

$150,001 or more 0.5 0.9 

Missing 0.4 0.8 
18

Table A-5. Standard errors for table 5: Base-weighted screener response rates, by primary of survey brand and 

selected frame characteristics – continued 

Characteristic Department of Education Census Bureau

Sex of head of household 

Male 0.2 0.3 

Female 0.3 0.5 

Missing 0.4 0.8 

Source: U.S. Department of Education, National Center for Education Statistics, National Household Education 

Surveys Program of 2012 (NHES:2012).