# Census 2000 Testing, Experimentation,

and Evaluation Program September 24, 2003

## Topic Report Series, No. 6

Evaluations of the Census 2000 Partnership and Marketing Program

### Final Report

Quality assurance procedures were applied throughout the creation of
this report.

This topic report integrates findings and provides context and
background for interpretation of results from Census 2000 evaluations,
tests, and other research undertaken by the U.S. Census Bureau. It is
part of a broad program, the Census 2000 Testing, Experimentation, and
Evaluation program, designed to assess Census 2000 and to inform 2010
Census planning.

W. Sherman Edwards

and Michael J. Wilson

> Westat

![](media/image1.wmf){width="1.95in" height="0.29305555555555557in"}

The Census 2000 Testing, Experimentation, and Evaluation Program
provides measures of effectiveness for the Census 2000 design,
operations, systems, and processes and provides information on the value
of new or different methodologies. The results and recommendations from
these analyses provide valuable information crucial to planning the 2010
Census. By providing measures of how well Census 2000 was conducted,
this program fully supports the Census Bureau\'s strategy to integrate
the 2010 planning process with ongoing Master Address File/TIGER
enhancements and the American Community Survey. The purpose of the
report that follows is to synthesize results from related Census 2000
evaluations, experiments, and other assessments to make recommendations
for planning the 2010 Census. Census 2000 Testing, Experimentation, and
Evaluation reports are available on the Census Bureau\'s Internet site
at:
[[http://www.census.gov/pred/www/]{.underline}](http://www.census.gov/pred/www/).

**CONTENTS**

1\. INTRODUCTION AND BACKGROUND 1

1.1 The Partnership and Marketing Program 1

1.2 PMP Evaluation Activities 4

1.3 Previous Evaluations of Marketing and Outreach Efforts 5

1.4 Other Related Surveys 5

2\. Summary and Assessment of Evaluation Activities 7

2.1 Description of the Evaluation Studies 8

2.2 Findings of the Evaluation Studies 9

2.2.1 The reach of marketing activities 9

2.2.2 Awareness and exposure 11

2.2.3 Attitudes towards the Census 12

2.2.4 Intended cooperation 14

2.2.5 The overall relationship between census communications\
and actual cooperation 16

2.2.6 Disaggregating the effects of PMP component activities 17

2.3 Discussion of Individual Evaluation Studies' Strengths and

Limitations 18

2.3.1 The Survey of Partners 19

2.3.2 Census in Schools Evaluation 19

2.3.3 PMPE Survey 19

2.3.4 Census Monitoring System 20

2.4 Cross-study Evaluation Topics 20

2.4.1 Experimental design 20

2.4.2 Timing of the evaluation activities 21

2.4.3 Civic activities 22

2.4.4 Survey nonresponse 23

2.4.5 Language spoken at home 25

3\. Summary of Results 27

4\. Recommendations 29

References 31

**LIST OF TABLES**

1\. Diverse America likelihood spectrum: attitudes and role of
advertising by segment 2

2\. General advertising plans by phase 2

3\. Significant differences (Chi-square) in census beliefs by recent
awareness\
of the census 13

**LIST OF TABLES (Continued)**

4\. Significant differences (Chi-square) in self-reported census
participation\
by whether positive beliefs reported 15

5\. Final return rates for PMPE Wave 2 survey respondents by sample and\
for corresponding groups in the general population 24

**FIGURE**

1\. Schedule of Census 2000 Activities APPENDIX

1.  **INTRODUCTION AND BACKGROUND**

**1.1 The Partnership and Marketing Program**

In response to declining mail return rates (down from 87 percent in
1970, the first census with a large scale mailout/mailback operation, to
74 percent in 1990 [20, p52, R1]{.mark}), the U. S. Census Bureau
implemented a number of changes in design and operations for Census
2000. These included a greatly expanded outreach and promotion campaign,
called the Partnership and Marketing Program (PMP), which for the first
time included paid advertising and an enhanced Partnership Program, in
an attempt to increase public awareness of the Census, to promote
positive attitudes about the Census, and to increase or at least slow
the decline in mail return rates, particularly among segments of the
population traditionally more difficult to enumerate. Two primary
concerns about the mail return rate made the expanded PMP appear
worthwhile: (1) followup of nonresponse to the mail Census is very
expensive and (2) responses received through the mail appear to be more
complete and accurate than those obtained in followup efforts. PMP
activities were also intended to increase the level and rate of
cooperation with the Nonresponse Followup (NRFU) phase of Census
2000[19, p v, R84]{.mark}. Ultimately, a goal of the PMP was to help
reduce the differential undercount across population groups[19, p ii,
R13]{.mark}.

The PMP included the following components:

-   A paid advertising campaign;

-   The Partnership Program;

-   Promotions and Special Events;

-   A media relations program[19, p i, R85]{.mark}; and

-   The Direct Mail Pieces component[29, p. 1, R2]{.mark}.

Each of these components was new, expanded, or significantly modified
from 1990. The paid advertising campaign, developed by Young and Rubicam
(Y&R), was based on a likelihood to respond model of the United States
population, called the Likelihood Spectrum^TM^ . [19, p 2.1,
R86]{.mark}.Table 1 shows the relationship between the model and the
goals of the advertising campaign. Y&R took as a proxy measure for this
likelihood the number of civic activities an individual engaged in: most
likely to respond are those participating in five or more civic
activities, undecided or passive are those with one to four activities,
and least likely are those with no civic activities.

[29, p. 1, R3]{.mark}

**Table 1: Y&R Likelihood Spectrum**^TM^**: attitudes and role of
advertising by segment**

**Least likely to respond Undecided/Passive Most Likely to Respond**

Attitudes Towards Fear Apathetic Familiar

Census Distrust Not very familiar Intend to participate

Completely unaware

Role of Advertising Lower resistance to pave Provide information
Reinforce positive behavior

way for community Provide reason to Instill sense of urgency

programs complete Motivate

Motivate Motivate Remind

Educate Educate

Remind Remind

Adapted from Wolter et. al., 2002 [29, p. 3, R4]{.mark}

The Y&R campaign was further segmented by race and ethnic group, in
particular targeting traditionally harder to enumerate populations:
African-Americans, Hispanics, Asians, American Indians/Alaska Natives,
and Native Hawaiians and other Pacific Islanders. The primary slogan for
the campaign, selected to promote beliefs of personal and community
benefits and stimulate return of the census form[26, p. 2, R22]{.mark},
was: "This is your future. Don't leave it blank." There were variations
of this slogan for different race and ethnic groups. [19, p. 2.6,
R5]{.mark}

The advertising campaign was divided into three phases, as shown in
Table 2. Each phase was intended to have its own set of messages, in
keeping with the goals shown in Table 2. A primary focus was to
demonstrate the benefits to the individual and community of
participation, and the cost of not participating. (U.S. Census Bureau,
Decennial Management Division, undated) [19, p. 2.1, R6]{.mark}

**Table 2: General media plans by phase\***

**Education phase Motivation phase Nonresponse followup phase**

Vehicles Print Print Radio

Radio Radio Television

Television Television

Out of home\*\*

Time Period November 1 to January 30 February 28 to April 9 April 17 to
June 5

Activity Weeks Broadcast: 9 weeks Broadcast: 6 weeks Broadcast: 7 weeks

Print: 2 months Print: 2 months

Out of home: 2 months

\*For the Diverse America audience, those most likely to respond were
not targeted during the education and nonresponse followup phases.

\*\*"Out of home" media included posters, outdoor advertisements, and
transit advertisements [13, p. 19, R7]{.mark}

Adapted from Wolter et. al., 2002 [29, p. 3, R8]{.mark}

The Partnership Program, greatly expanded for Census 2000, involved
Census Bureau partnership specialists working with state, local and
tribal governments, community groups, nongovernmental organizations,
local media, and private sector industries. The objectives, as for the
PMP overall, were (1) to increase the overall response rate for Census
2000, (2) to reduce the undercount of historically hard-to-enumerate
populations, and (3) to communicate a consistent message to all
Americans that re-enforced the paid advertising message and, in effect,
closed the sale. (U.S. Census Bureau, Decennial Management Division,
undated [19, p. ii, R13]{.mark}) The Partnership Program employed some
690 partnership specialists around the country, working with about
140,000 partner organizations. [28, p. 1, R14]{.mark}

The Census Bureau provided materials to partners to help publicize the
census and to educate and motivate partners' constituents, including
posters and fact sheets, videos, articles for newsletters, press
releases, sample forms, graphics, and promotional items. The Census
Bureau made materials available in a number of different languages; in
addition, many partners developed in-language educational and
informational materials for their constiutents. The Census Bureau also
provided instructional manuals for partners in a variety of settings to
help them design programs to meet shared goals. Census Bureau staff also
participated in partner activities. Partners, in turn, helped to
publicize the census through a variety of media, organized educational
and motivational community activities, and provided assistance in some
census operations (Westat, 2001 [28, p. 1, R15]{.mark}).

The Census in Schools (CIS) Program was a significant component of
Promotions and Special Events, with the goal of teaching students about
the census. [25, p. iv, R16]{.mark} A variety of teaching materials were
made available to teachers, with the intention that students would take
materials home and/or communicate with their parents about the
importance of participating in the census (Macro International, 2002).
[25, p. iv, R17]{.mark} Other major components of Promotions and Special
Events were "How America Knows What America Needs," which assisted local
elected officals in encouraging their communities to participate in the
census, and the Census 2000 Road Tour, in which twelve Census Bureau
vehicles traveled around the country during February through April 2000,
setting up exhibits in local "high traffic" areas. [26, p. 3,
R23]{.mark}

The Census Bureau also focused on media relations during the census
period to complement the other components of the PMP. The goal was to
ensure that positive and educational stories about the census would
appear in print and electronic media. [26, p. 3, R24]{.mark}

The Direct Mail component incorporated some significant changes from
1990 to 2000, based on methodological research in the intervening years.
For Census 2000, both the Mailout/Mailback and Update/Leave universes
received advance letters, telling them that the Census 2000
questionnaire would be coming; 1990 census operations did not include an
advance letter [personal communication from Census Bureau staff]{.mark}.
A 1992 Implementation Test of the effects of sending advance letters,
including a stamp on the return envelope, and mailing reminder post
cards found that each of these additional mail contacts with households
resulted in higher overall response rates -- 6.4 percent, 2.6 percent,
and 8.0 percent respectively[22, p. 45, R87]{.mark} -- and that the
effects were additive within the test samples[22, p. 45, R88]{.mark}.
The improvements for 1990 low response rate areas were somewhat smaller
-- 4.2 percent, 1.6 percent, and 5.7 percent [22, p. 43, R89]{.mark}
(Clark et. al., 1993). Both the 1990 and 2000 censuses included reminder
post cards; the stamp was used in neither, although both included
prepaid return envelopes. [personal communication from Census Bureau
staff]{.mark}

Another major change between 1990 and 2000 was prompted by a 1993 test
of the effects of various kinds of motivational messages on response
rates. Including the statement "Your Response is Required by Law" in a
box on the outer envelope increased response rates by 9 to 11 percentage
points overall as compared with approaches not using that phrase on the
envelope, and 7 to 8 percentage points in 1990 low response areas
(Dillman et. al., 1996) [24, p. 385, R91]{.mark}. This statement was
included on the Census 2000 outer envelope; it had not been used
previously in a decennial census.

Finally, the census questionnaire itself was redesigned to be more
"respondent-friendly," using generally accepted design principles and
focus group testing. This redesign was made possible by the use of the
new technologies of optical scanning and character
recognition.[Statement added by a Census Bureau reviewer]{.mark} A
Simplified Questionnaire Test in 1992 found that the respondent-friendly
design increased the return rate by 3.4 percent overall as compared with
the 1990 Short Form, and by 7.5 percent in low response areas (Dillman
et. al., 1993) [23, p. 298, R142]{.mark}.

**1.2 PMP Evaluation Activities**

The Census Bureau commissioned three major research evaluations of PMP
activities:

-   The Partnership and Marketing Program Evaluation (PMPE), a series of
    three general population surveys conducted and analyzed by the
    National Opinion Research Center (NORC), which was intended to
    evaluate the effects of most of the PMP components;

-   The Survey of Partners, a sample survey of organizations enlisted as
    partners for Census 2000, focusing on the Partnership Program and
    conducted and analyzed by Westat; and

-   An evaluation of the Census in Schools Program, based on a survey of
    primary and secondary school teachers, conducted and analyzed by
    Macro International.

Another important evaluation study was the Census Monitoring Survey
(CMS), a weekly survey of the general population conducted just before
and during the Census 2000 mailout/mailback by InterSurvey (now known as
Knowledge Networks). The CMS was privately commissioned and funded.

Figure 1 in the Appendix shows the timing of Census 2000 operations,
paid advertising, and evaluation study activities. The advertising
campaign was timed so as to achieve the objectives outlined in Table 1,
and the PMPE and CMS surveys were timed to assess the effects of the
advertising campaign (as well as other PMP activities). The PMPE survey
Wave 1 was fielded before the education phase of the advertising
campaign to measure "baseline" awareness and attitudes; Wave 2 was
largely between the education and motivation phases, and the start of
Wave 3 coincided with the NRFU phase of both Census 2000 operations and
the advertising campaign. The CMS was conducted essentially between
Waves 2 and 3 of the PMPE survey.

Partnership activities had a longer time frame than the advertising
campaign. The planning and education phases, focusing on developing the
partnerships, stretched from late 1996 through late 1999. Motivation
activities began in late 1999 or early 2000, reached a peak between the
mailout and Census Day (April 1), and continued through the NRFU. The
Partnership Evaluation field period was October 2000 through March
2001[28, p. 7, R105]{.mark}, considerably after PMP activities had
concluded.

The Census in Schools Program conducted mailings to teachers and
principals March through September 1999. All schools received at least
one teaching kit, and invitational packets were sent to elementary
school teachers and secondary school math and science teachers in
historically hard-to-enumerate (HTE) areas [19, p. 4.2, R18]{.mark}.
During the census period, take-home packets were mailed to all
elementary school teachers and middle school social studies teachers.
The CIS survey was conducted in the Spring of 2000.

The advertising campaign was tested during the Census 2000 Dress
Rehearsal in 1998, in Sacramento CA, Columbia SC and eleven surrounding
counties, and Menominee WI. A pre-/post-test survey to assess the
effects of the campaign in Sacramento and South Carolina was conducted
by Westat and analyzed by Roper-Starch.

Besides these major evaluations by contractors and other non-Census
groups, there were a variety of other evaluation activities conducted
both by Census Bureau and contractor staff that looked at specific
components of the PMP. Some of these activities will be cited in the
remainder of the report.

**1.3 Previous Evaluations of Marketing and Outreach Efforts**

The PMPE and Census 2000 Dress Rehearsal surveys follow the general form
of two previous "pre/post" evaluation surveys: the Knowledge, Attitudes,
and Practices (KAP) Survey in 1980, and the 1990 Outreach Evaluation
Survey (OES). Each was designed to assess the effects of census
marketing and outreach efforts on the American public's awareness,
knowledge and attitudes, and behavior with regard to the Decennial
Census.

This report will cite findings from these two surveys by way of
historical comparison, where comparable items were used. It should be
noted that there are significant differences in the timing of survey
waves across the various evaluation studies, so interpretation of
differences in results must be viewed cautiously. The last two rows of
Figure 1 show the approximate dates of the 1980 KAP and 1990 OES.

Wave 1 (the "pre" phase) of each of these surveys occurred in late
January and early February. Wave 1 of the PMPE was much earlier because
the advertising campaign in 2000 included an education phase that
happened much earlier than such efforts in 1990 or 1980. While the three
Wave 1 surveys are roughly comparable in their relationship to planned
outreach and publicity efforts, they may not be comparable for some
measures because of the differences in timing and because of external
events such as news reports related to the census.

Wave 2 of the 1980 KAP occurred after the outreach and publicity
campaign was well under way, but concluded just before the
mailout/mailback operation. In contrast, Wave 2 of the 1990 OES was
conducted after the mailout/mailback and during the NRFU. The 2000
PMPE's Wave 2 was conducted before the mailout/mailback, so is somewhat
comparable to Wave 2 of the 1980 KAP, although it is not as close to the
mailout/mailback operations. Timing of Wave 3 of the PMPE is comparable
to that of Wave 2 of the 1990 OES.

**1.4 Other Related Surveys**

Besides the surveys conducted to evaluate marketing and outreach
efforts, the Census Bureau commissioned other research to examine issues
of attitudes and behaviors related to participation in the census. In
1999, NORC conducted a Knowledge, Attitudes, and Perceptions Survey,
known as KAP-1. During the 1990s, there were several surveys measuring
public attitudes about privacy and confidentiality around the census,
reviewed in a companion Topic Report by Eleanor Singer. The Survey of
1990 Census Participation (SCP), conducted by NORC in June and July
following census operations, was designed to assess the reasons for the
decline in response from 1980 to 1990. A roughly comparable survey was
the 1980 Applied Behavior Analysis Survey (ABAS), conducted in April
during census operations. Limited results from these surveys will be
cited to demonstrate historical trends or provide reinforcement or
contrast to findings from the 2000 evaluation studies.

**2. Summary and Assessment of Evaluation Activities**

By most outcome measures, Census 2000 was a success. The mail return
rate, defined as the number of mail returns received before the cutoff
date for the NRFU divided by the number of occupied housing units in
mailback areas, was 74.1 percent, almost identical to that of the 1990
Census (1990 rates cited in Stackhouse and Brady, 2002 [1, p. v,
R9]{.mark}) and ending the sharp decline between 1970 and 1990. The
final mail response rate, defined as the percentage of the NRFU-eligible
households returning forms, was 67 percent, up from 65 percent in 1990
and well above the expected rate of 61 percent [33, p. 1, R141]{.mark}
(U.S. Census Bureau,
[http://rates.census.gov](http://rates.census.gov/)). The NRFU effort
finished almost two weeks ahead of schedule. [2, p. v,
R140]{.mark}Finally, in 1990 the net undercount of the U.S. population
was estimated at 1.6 percent overall, and up to 5 percent for various
racial and ethnic groups (U.S. Census Bureau,
<http://www.census.gov/dmd/www/pdf/underus.pdf>). For 2000, various
estimates (U.S. Census Bureau, 2003) indicate a net overcount of 0.36 to
1.12 percent, with no undercount of a racial or ethnic group larger than
2.5 percent. [31, p. 11, R13]{.mark}

However, the mail return rate for the long form (63.0 percent) was
considerably lower than that for the short form (76.4 percent). These
rates compare with 70.4 percent for the long form and 74.9 percent for
the short form in 1990 [1, p. v, R10]{.mark}. The gap of 13.4 percentage
points overall between short and long form return rates was greater
among all non-White races than among Whites by at least 1.5 percentage
points [1, p. 19, R11]{.mark}. The gap between long and short forms was
smaller by the final return rate (9.6 percentage points overall), but
the gap was reduced less among all non-White races, except Asians, than
among Whites (Stackhouse and Brady, 2002) [1, p. 20, R12]{.mark}.

PMP evaluation studies were intended to measure the effectiveness of PMP
components and activities -- to try to attribute the contribution of
each to the relative success, as it turns out, of Census 2000. Without
an experimental design, it was not easy, and perhaps not possible, to
measure these contributions directly, so the evaluation analysis
strategy relied on a simple behavioral model, the one underlying the Y&R
advertising strategy: in order to participate, individuals must first be
aware of Census 2000, they must have positive attitudes about it, and
they must be motivated to fill out the Census 2000 form. Attitudes and
motivation, in turn, are a function of the information individuals have
about the Decennial Census. The PMP attempted to convey the right
message, to the right people, at the right time to convince them to
respond to the census.

As we shall see, the evaluations largely support the links between
awareness and positive attitudes or beliefs, between positive beliefs
and intended or reported participation, and between intended or reported
and actual participation. Evaluation data indicate that the presence and
strength of these links vary by population group. Taken as a whole, the
evaluation study data suggest certain conclusions with regard to the
effectiveness of the PMP, but are far from conclusive on any front. An
attempt to put all of the pieces together in multivariate models did not
show any significant main effect of PMP activities on actual mail return
behavior.

In the remainder of this chapter, we describe the Census 2000 evaluation
studies and discuss their results. Then, we discuss the strengths and
weaknesses of the study designs and implementations. Chapter 3
summarizes the discussions and make recommendations for evaluating
future Decennial Census Partnership and Marketing Programs.

**2.1 Description of the Evaluation Studies**

The PMPE survey was conducted in three waves, combining telephone and
in-person interviews. The survey was intended to capture census
awareness and other factors thought to be associated with cooperation,
as well as exposure to messages about the census from a wide variety of
sources. Wave 1, with 3,002 completed interviews from a
random-digit-dial sample frame, occurred (largely) before the education
phase of the advertising campaign. Wave 2 comprised 2,716 completed
interviews with a sample selected from the Decennial Master Address File
(DMAF) and was conducted during the motivation phase and before the
mailout of Census 2000 forms. Wave 3 was conducted after the mailout,
during and after the NRFU; the sample was also selected from the DMAF,
and 4,247 interviews were completed [29, pp. 6-7, R24]{.mark}. The
design of each wave included oversamples of hard-to-enumerate population
groups: Hispanics, non-Hispanic African-Americans, Asians, Native
Hawaiians, and American Indians. Survey instruments included items on
"media use; awareness of government agencies and programs; awareness of
community agencies and programs; recall of exposure to the mass media;
recall of exposure to partnership-sponsored activities; recall about
sources of information; knowledge and attitudes about the Decennial
Census; aided recall of specific advertising; aided recall of specific
partnership activities; Census 2000 mailback form receipt, handling, and
mailback behavior; and demographic information" (Wolter et. al., 2002).
[29, pp. 6-7, R25]{.mark}

The 2000 CMS was conducted weekly during the motivation phase of the
advertising campaign and through the mailout/mailback period. It was
intended to provide immediate feedback on marketing and mailout
activities. The sample was drawn from the InterSurvey (now known as
Knowledge Networks) panel, and was conducted through interactive Web TV.
The five weekly surveys had sample sizes (completed interviews) of 993,
973, 719, 1,004, and 948 respondents, respectively. [32, p 1,
R26]{.mark}Survey content included exposure to and reaction to
advertising, census attitudes and awareness, perceptions of, experience
with, and action taken with respect to the Census 2000 mailback form,
reasons for nonresponse, response intentions, NRFU experience, and
recognition of ads played back during the interview (Nie and Junn,
2000).

The Survey of Partners was conducted well after PMP and Census 2000
activities were over, relying on the recall of designated contacts at
partner organizations. It was intended to assess the success of the
partnership efforts and identify the benefits and contributions of
partners and the Census Bureau. The survey was conducted through the
mail, with telephone followup, of a stratified random sample of partner
organizations drawn from the May 2000 Contact Profile and Usage
Management System (CPUMS), including: national and local Federal
government organizations; media organizations, sub-classified by
race/ethnicity of their target audiences; national and all other
for-profit private businesses; national non-government organizations
(NGOs); local NGOs, sub-classified by race/ethnicity of their
constituencies; and state, local and tribal government organizations[28,
pp 5-6, R27]{.mark}. A total of 9,057 interviews was completed[28, p 7,
R28]{.mark}. The survey instrument included items on: materials the
partners received and used, including how helpful the materials were and
the timeliness of their receipt; the relative importance of various
partnership goals; the kinds of activities partners engaged in; the
kinds of assistance Census staff provided; costs associated with Census
2000-related activities; and characteristics of the partner
organization[28, Appendix A, R106]{.mark}.

The CIS survey was conducted during the Spring of 2000, at the end of
the school year in which teachers would have used materials they
received as part of the program. The survey focused on whether and how
teachers learned about the CIS Program, obtained materials, and used
those materials in class, and asked teachers to assess the program and
its materials. The survey was conducted through the mail, with telephone
followup, of a sample of some 4,000 primary and secondary school
teachers. The sample was selected from two sources: a commercial list of
teachers and a list of teachers who ordered CIS Program materials. The
sample was stratified to allow separate estimates for HTE areas and
non-HTE areas. Some 1,046 surveys were completed and accepted for
analysis. [25, p 2, R29]{.mark}

**2.2 Findings of the Evaluation Studies**

This section will summarize the findings of the evaluation studies by
topic, following the behavioral model of participation described
earlier. We will begin by describing findings with regard to the reach
of the PMP activities that were evaluated, then discuss awareness of
Census 2000 and exposure to PMP activities, then describe findings with
regard to attitudes towards the Census and government in general, and
finally cover findings with regard to intended/reported and actual
participation. For each topic, we will describe results for the general
population and also for the targeted race and ethnic groups. Finally, we
will discuss findings with regard to the relative effects of different
PMP components and activities across these topics.

**2.2.1 The reach of marketing activities**

Several evaluation and operational reports describe aspects of the reach
of PMP activities. The Partnership Program, the Census in Schools
Program, the paid advertising campaign, the media relations campaign,
Promotions and Special Events and the Direct Mail Pieces component all
helped to spread the word about Census 2000. Several of these components
particularly targeted historically hard-to-enumerate population groups
and/or geographic areas.

The Census Bureau enlisted about 140,000 organizations in its
Partnership Program[^1]. Most of these were local in scope, with almost
40 percent operating at a city level and another 26 percent at a county
level[28, p. 14, R107]{.mark}. The partners were about equally divided
between governments and non-governmental organizations. Partnering
governments included local, state, and tribal entities[28, p. 16,
R108]{.mark}. Partnering non-governmental organizations included
community-based organizations (29 percent), businesses (22 percent),
religious organizations (16 percent), and educational organizations (15
percent) [28, p. 17, R109]{.mark}. More than half of partners did not
target any specific race or ethnic group, while 23 percent targeted
Hispanics, 22 percent African-Americans, 11 percent Asians, 7 percent
American Indians/Alaska Natives, and 4 percent Native Hawaiians/Pacific
Islanders (Westat, 2001) [28, p. 18, R110]{.mark}.

At least 70 percent of partner organizations reported conducting one or
more kinds of activities to publicize Census 2000, educate and motivate
constituents, or support Census 2000 operations[28, p. 28, R111]{.mark}.
Almost one-quarter reported conducting more than ten different kinds of
activities[28, p.28, R112]{.mark}. On average, each state[^2] had more
than 1,100 active and about 400 very active partner
organizations[derived from previous statements]{.mark}. It is not
possible to estimate with any precision how many individuals may have
been reached by these partners' activities, but the number of active
partners is substantial.

About 16 percent of partner organizations spent non-Census funds, either
their own or from another source, to promote Census 2000. While there
was considerable nonresponse in the Survey of Partners on questions
asking for the amount of funds expended, respondents reported some \$168
million spent, which is probably a lower bound given the item
nonresponse. About one-third of partner organizations reported making
in-kinds contributions, such as staff time, office space, and equipment
usage. The dollar value of these contributions was estimated at about
\$374 million (Westat, 2001).[28, Tables 8 and 9]{.mark}

According to the CIS Evaluation report, some 56 percent of teachers
nationally had heard of the CIS Program [25, p. 4, R19]{.mark}. Among
elementary school teachers in HTE areas, a targeted group, 68 percent
had heard of the CIS Program, compared with 62 percent in other areas
[25, p. 8, R20]{.mark}. Among secondary school social studies and math
teachers, 44 percent in HTE areas had heard of the CIS Program, compared
with 59 percent in other areas (Macro International, 2002 [25, p. 8,
R21]{.mark}). Thus, it appears that the mailing of invitational packets
directly to teachers in HTE areas increased awareness of the CIS Program
among elementary school teachers, but not among secondary school social
studies and math teachers. About 37 percent of teachers in HTE areas and
34 percent of those in other areas reported actually receiving
materials[25, Appendix C, unweighted frequencies, p. 119, R113]{.mark}.
Elementary school teachers were much more likely to report receiving
materials (46 percent) than were secondary school social studies and
math teachers (30 percent)^3^[25, Appendix C, unweighted frequencies, p.
120, R114]{.mark}.

An important goal of the CIS Program was to reach families through their
children, increasing awareness and knowledge of the census, and
ultimately increasing participation. Take-Home Packets were sent to all
elementary school teachers and to middle school social studies teachers.
About 19 percent of elementary school teachers and 8 percent of
secondary school social studies and math teachers sent materials home
with their students[25, Appendix C, unweighted frequencies, p. 153,
R116]{.mark}. Teachers in HTE areas were about as likely (14 percent
versus 12 percent) to send materials home as those in other areas[25,
Appendix C, unweighted frequencies, p. 153, R115]{.mark}.[^3]

A Census Bureau assessment of the "How America Knows What America Needs"
campaign (Sha and Collins, forthcoming) reported that local and national
media coverage of Census 2000 more than doubled that of the 1990 census
(in terms of sheer number of news stories) [7, p. 10, R117]{.mark}.
However, according to an independent media analysis commissioned by the
Census Bureau (Douglas Gould and Co, 2001), print media coverage of
Census 2000 across nine major outlets was down from the level of the
1990 Census. The Gould report speculated that the decline occurred
because Census 2000 was less controversial than the 1990 census and
because of interest in the Presidential campaign[9, p. 2, R118]{.mark}.
Important exceptions to the "less controversial" observation were
partisan wrangling over adjustment and comments by some politicians
about the intrusiveness of the long form. The discrepancy between the
Census findings and those of the Gould report may indicate greatly
increased coverage of Census 2000 at the local level. The Gould report
also noted that opinion pieces were largely positive[9, p. 2,
R119]{.mark}, and also that outreach efforts were the subject of 19
percent of the sample articles overall and 36 percent of pre-Census Day
articles, up from only 5 percent in 1990[9, p. 3, R120]{.mark}.

2.2.2 **Awareness and exposure**

It is clear that awareness of Census 2000 rose dramatically throughout
the marketing campaign, at least well into the NRFU phase. During Wave 1
of the PMPE survey, about 65 percent of respondents reported having
heard nothing about Census 2000 and fewer than 10 percent reported
having heard a great deal. In Wave 2, about 25 percent reported hearing
nothing and about the same percentage reported hearing a great deal. By
Wave 3, only about 15 percent of respondents reported having heard
nothing, and almost half reported having heard a great deal, almost a
complete reversal from Wave 1 [29, p. 27, R30]{.mark}. Awareness
increased significantly between each wave for each of the oversampled
populations, except that awareness levels for Hispanics may have leveled
off between Wave 2 and Wave 3 [29, p. 28, R31]{.mark}. (Wolter et. al.,
2002)

Despite these dramatic increases in awareness measured during Census
2000, peak awareness (percentage having "heard recently") was lower than
during the 1990 Census among Hispanics and non-Hispanic Whites, although
it was higher among non-Hispanic African-Americans. Awareness as
measured in Wave 2 of the PMPE and the 1980 KAP were fairly comparable,
although again higher in 2000 among non-Hispanic African-Americans.
Comparing the Wave 1 results from 2000 with those from 1980, the 2000
PMPE found somewhat lower levels among non-Hispanic African-Americans
and Whites, but higher levels among Hispanics (Wolter et. al., 2002,
drawing data from Bates and Whitford, 1991, and Moore, 1982) [29, p.
118, R34]{.mark}. The latter difference may be due to early education
efforts aimed at Hispanics in 2000. Wave 1 awareness was much higher in
1990 than either 1980 or 2000[29, p. 118, R34]{.mark}, which may in part
be attributable to the 1990 Census Awareness and Products Program, which
started earlier than similar efforts in 1980 [18, p. 10, R121]{.mark}
(Fay et. al., 1991) and to the timing of Wave 1 in 1990 as compared with
2000. It may, of course, also be related to other factors, such as a
higher level of news coverage of census issues.

The 2000 Dress Rehearsal survey reached a level of "heard recently"
similar to that of the PMPE (more than 80 percent[16a and 16b, p. i,
R122]{.mark}), but the pre-campaign 2000 Dress Rehearsal survey
awareness levels were lower (28 and 29 percent in Sacramento and South
Carolina, respectively[16a and 16b, p. i, R122]{.mark}) than in Wave 1
of the PMPE survey (35 percent[29, p. 27, R30]{.mark}) (Roper-Starch,
1999). It seems reasonable that ambient information about the census
would be lower two years before the census.

The CMS, conducted essentially between Waves 2 and 3 of the PMPE survey,
showed that the proportion of respondents reporting they had seen or
heard "a lot" about Census 2000 from TV commercials rose from 30 percent
during the first week of March to 70 percent 3 weeks later[12, p. 19,
R92]{.mark}. Smaller percentages, but similar proportionate increases,
were reported for radio and newspaper advertising [12, p. 19,
R93]{.mark} (Nie and Junn, 2000). Clearly, the intense motivation phase
campaign and the mailout had a substantial effect on awareness.

**2.2.3 Attitudes towards the Census**

Overall, it appears that positive attitudes towards the census increased
significantly during the PMP campaign, but that this increase was
tempered by receipt of the census forms, particularly among those who
received the long form. Among race and ethnic groups other than
non-Hispanic Whites, those who had recently heard about the census were
more likely to hold positive beliefs about it than those who hadn't. The
following paragraphs provide details of these findings.

The PMPE survey included eight items asking about respondents' beliefs
about the census [29, p. 71, R35]{.mark}. Using factor analysis to
construct and analyze a composite of these items, Wolter et. al. (2002)
showed that there was some significant movement of beliefs in the
positive direction between Wave 1 and Wave 2, but not between Wave 2 and
Wave 3 [29, p. 72, R36]{.mark}. Every race and ethnic group examined
except American Indians showed some increase in positive beliefs over
the three waves [29, p. 74, R37]{.mark}.[^4]

The PMPE survey findings are similar to those of the 1980 KAP survey and
1990 OES. In 1980 (comparable to the PMPE Wave 1 to Wave 2 comparison),
favorable responses to three attitude items included in both 1980 and
1990 increased between Wave 1 and Wave 2[18, p. 15, R124]{.mark}. In
1990, there was mixed movement between Wave 1 and Wave 2 across six
items, with some items increasing and others decreasing[18, p. 15,
R125]{.mark} (Fay et al, 1991a). Because recent awareness of the census
was relatively high in Wave 1 of the OES, it may be that the appropriate
comparison is with Wave 2 to Wave 3 of the PMPE, where there was no net
movement.

The CMS included five belief items. The level of agreement with three
positive items stayed reasonably constant over five weeks of
interviewing, while agreement with two negative items -- the census is
an invasion of privacy, and my answers could be used against me --
increased (that is, beliefs moved in a negative direction) over that
period[12, p. 23, R94]{.mark}. Martin (2000) modeled the association
between hearing about the controversy and negative beliefs, and
concluded that the controversy did increase negative beliefs, as did
receipt of the long form, and that the effects of these two factors were
largely independent[33, p. 23, R140]{.mark}.

About 44 percent of respondents in week 5 of the CMS had heard of the
controversy, and 9 percent of those said that it made them feel less
like returning their Census 2000 form[12, p. 25, R95]{.mark}. Virtually
all of this 9 percent were people who had received the long form, it
appears from data presented by Nie and Junn -- almost half of those
receiving the long form said that the controversy made them feel less
like returning their form[12, p. 25, R96]{.mark}[^5]. It seems very
likely, then, from the findings of both Martin (2000) and Nie and Junn
(2000) that the long form controversy had a negative effect on census
returns, at least among those receiving the long form, although neither
source links hearing about the controversy directly with behavior.

The Census 2000 Dress Rehearsal study included nine belief items, with
some overlap with both the PMPE and CMS. Eight of the 9 items showed
some increase in positive views from before the marketing campaign to
after, most significantly, in both Sacramento and South Carolina.
Notably, there was no change in either site in the proportion agreeing
strongly or somewhat that the census is an invasion of privacy[16a p.
63, 16b p. 73, R123]{.mark}. This finding suggests two
interpretations: 1) that this negative belief was not addressed or was
not addressed convincingly in the marketing campaign; 2) that receipt of
the census mailing alone may not have been the cause of the increase in
agreement with the statement found in the CMS[^6]. Non-Hispanic
African-Americans were significantly more likely than non-Hispanic
Whites to agree with each of the three negative beliefs after the
marketing campaign, in both sites[16a pp. 68-70, 16b pp. 78-80,
R127]{.mark}. In Sacramento, Hispanics were more likely to agree with
two of the three negative beliefs than non-Hispanic Whites[^7][16b pp.
78-80, R127]{.mark}.

**Table 3. Significant differences (Chi-square) in census beliefs by
recent awareness of the census**

  ---------------------------- ------- ---------- ------------------ ------- ------- ---------- ----------
  Belief                       Total   Hispanic   African-American   White   Asian   American   Native
                               Pop.                                                  Indian     Hawaiian

  Filling out the census will  \*                 \*                 \*      \*                 \*
  let the government know what                                                                  
  my community needs [29, p.                                                                    
  76, R38]{.mark}                                                                               

  The census counts citizens           \*                                                       \*
  and non-citizens alike [29,                                                                   
  p. 76, R39]{.mark}                                                                            

  It is important for as many                     \*                                            
  people as possible to                                                                         
  participate in the census                                                                     
  [29, p. 77, R40]{.mark}                                                                       

  My answers to the census             \*         \*                         \*                 \*
  could be used against me                                                                      
  [29, p. 77, R41]{.mark}                                                                       

  Answering and sending back           \*         \*                                 \*         
  the census matters for my                                                                     
  family and my community [29,                                                                  
  p. 78, R42]{.mark}                                                                            

  The Census Bureau promise of                    \*                                            
  confidentiality can be                                                                        
  trusted [29, p. 78,                                                                           
  R43]{.mark}                                                                                   

  I just don't see that it                        \*                         \*                 \*
  matters much if I personally                                                                  
  fill out the Census or not                                                                    
  [29, p. 79, R44]{.mark}                                                                       

  Sending back your census                                                   \*      \*         
  form could personally                                                                         
  benefit or harm you in any                                                                    
  way[29, p. 79, R45]{.mark}                                                                    
  ---------------------------- ------- ---------- ------------------ ------- ------- ---------- ----------

Based on findings reported by Wolter et. al., 2002

\*p \<.10

For each of the eight PMPE survey items, among some (but not all) race
and ethnic groups, people who had recently heard about the Census were
significantly more likely to hold positive views than those who hadn't
recently heard [29, p. 76, R38]{.mark}. Table 3 summarizes the
significant differences in percentage of respondents holding positive
beliefs about the census by whether they reported recent awareness.
While the recently aware had more positive beliefs among almost all
groups for almost all items, the patterns of statistical significance
vary interestingly by race and ethnic group. Non-Hispanic
African-Americans showed the largest number of statistical differences,
and non-Hispanic Whites the fewest.

Three of the belief items were included in the 1990 OES, and two of
these were in the 1980 KAP survey. The two common items (whether the
census promise of confidentiality can be trusted and that the census
data can not be used against you) showed increases over time within all
three surveys; the largest increases (12 and 8 percentage points) were
between Wave 1 and Wave 2 in 2000[29, p. 119, R90]{.mark}.

**2.2.4 Intended cooperation**

The PMPE survey allows comparison of the change in intended cooperation
from before the start of the paid advertising campaign (Wave 1) to after
the campaign's education phase (Wave 2). Among all groups except
Hispanics (who started at a high level and stayed there) and American
Indians (who started at a low level and rose slightly), the level of
intended cooperation rose significantly between Wave 1 and Wave 2 [29,
p. 65, R46]{.mark}. The correlation between awareness of Census 2000
communications and intended cooperation rose among all groups between
Wave 1 and Wave 2 [29, p. 65, R47]{.mark},[^8] suggesting beneficial
effects from the PMP.

**Intended, self-reported, and actual cooperation.** Wave 3 of the PMPE
survey asked whether the respondent's household had returned the census
forms, and for both Waves 2 and 3 information on actual return of the
census form was obtained. The correlation between intended and
actual[^9] cooperation was low, ranging from 0.05 (all correlation
values are weighted) for American Indians to 0.27 for Hispanics [29, p.
90, R48]{.mark}. Reported cooperation in Wave 3 was more highly
correlated with actual cooperation, ranging from 0.30 for
African-Americans to 0.42 for American Indians [29, p. 90, R49]{.mark},
but the correlations are still fairly low. About three-quarters of Wave
2 respondents who said they "definitely will" return the form actually
did so [29, p. 90, R50]{.mark}, and about the same proportion of Wave 3
respondents who said they had returned the form had their claim verified
[29, p. 90, R51]{.mark}.

**The relationship between beliefs and cooperation.** The CMS asked
respondents whether they felt particular messages were "persuasive
reasons to fill out the census form." Allocation of federal dollars to
communities was viewed as persuasive by about three-quarters of
respondents, peaking the week of March 23[12, p. 23, R97]{.mark}.
Allocation of Congressional seats as a persuasive reason also peaked the
same week, at 60 percent of respondents[12, p. 23, R98]{.mark}.
Identification of mandatory participation as a persuasive reason doubled
from March 3 to March 23 (after the mailout, which had that message on
the envelope), reaching a peak of 46 percent in the week of April 7[12,
p. 23, R99]{.mark}[^10]. Analysis of the 1990 OES and SCP found that
knowing that the census was required by law was the only knowledge item
tested that was a significantly better predictor of return than
others[18, p. 16, R128]{.mark} (Fay et. al., 1991a).

Table 4 summarizes the relationship between beliefs about the census and
self-reported participation in Wave 3 of the PMPE survey. For most race
and ethnic groups, there is a similar pattern of association between
beliefs and self-reported behavior. For Hispanics, however, the only
belief significantly associated with self-reported participation is that
the census counts both citizens and non-citizens. For American Indians,
the relationship between the belief that answers to the census "won't be
used against me" and self-reported cooperation is highly significant,
while it is not significant among any other group.

**Table 4. Significant differences (Chi-square) in self-reported census
participation by whether positive beliefs reported**

  --------------------------- ------- ---------- ------------------ ------- ------- ---------- ----------
  Belief                      Total   Hispanic   African-American   White   Asian   American   Native
                              Pop.                                                  Indian     Hawaiian

  Filling out the census will \*                 \*                 \*      \*      \*         \*
  let the government know                                                                      
  what my community needs                                                                      
  [29, p. 82, R55]{.mark}                                                                      

  The census counts citizens  \*      \*                            \*      \*                 
  and non-citizens alike [29,                                                                  
  p. 82, R56]{.mark}                                                                           

  It is important for as many \*                 \*                 \*      \*      \*         \*
  people as possible to                                                                        
  participate in the census                                                                    
  [29, p. 83, R57]{.mark}                                                                      

  My answers to the census                                                          \*         
  could be used against me                                                                     
  [29, p. 81, R53]{.mark}                                                                      

  Answering and sending back  \*                                    \*      \*      \*         \*
  the census matters for my                                                                    
  family and my community                                                                      
  [29, p. 83, R58]{.mark}                                                                      

  The Census Bureau promise                                                                    
  of confidentiality can be                                                                    
  trusted [29, p. 80,                                                                          
  R52]{.mark}                                                                                  

  I just don't see that it    \*                 \*                 \*      \*      \*         \*
  matters much if I                                                                            
  personally fill out the                                                                      
  Census or not [29, p. 84,                                                                    
  R59]{.mark}                                                                                  

  Sending back your census                                                                     
  form could personally                                                                        
  benefit or harm you in any                                                                   
  way[29, p. 81, R54]{.mark}                                                                   
  --------------------------- ------- ---------- ------------------ ------- ------- ---------- ----------

Based on findings reported by Wolter et. al., 2002

\*p \<.10

Note that the three items related to privacy and confidentiality in
Table 4 ("My answers to the census could be used against me," "The
Census Bureau promise of confidentiality can be trusted," and "Sending
back your census for could personally harm you in any way \[sic\]") show
no significant association with participation, with one
exception---American Indians were less likely to report returning the
census form if they believed the census could be used against them. Fay
et. al. (1991a) examined the relationship between an index of three
privacy/confidentiality items and return of the census form using the
1990 OES and 1980 KAP. In 1990, those with positive attitudes on all
three items were significantly more likely to return the census form (by
both self-report and matching actual behavior, while there was no
difference in 1980 (only self report was examined). Similar findings
were reported by Kulka et. al. (1991). Martin (2000), in an analysis of
2000 CMS and 1990 OES data, found "several indications that Census 2000
engendered more sensitivity and a more diverse privacy reaction than the
previous census." Martin noted, as had Fay et. al. (1991b), that privacy
and confidentiality are multi-faceted concepts for the public, with
complex inter-relationships that may change over time and in response to
particular stimuli (such as receipt of the census form or hearing about
the long form controversy in 2000). Thus, the relationship between these
concerns and return of the census form in 2000 may warrant further
exploration before simply accepting the findings for these three items
as summarized in Table 4.

**2.2.5 The overall relationship between census communications and
actual cooperation**

Wolter et. al. (2002) fitted a series of multivariate models in an
attempt to show the relationship between PMP activities and actual mail
return behavior. They concluded, "the . . . data are consistent with the
hypothesis that mass media and community-based communications had no
effect on the odds of a mail return for the Asian, American Indian, and
Native Hawaiian populations. . . . The data support a conclusion that
census communications were less effective for the other-language
population than for the English population." [29, p. xxii,
R15]{.mark}These conclusions could certainly be termed disappointing.
However, the incremental increase in return rates overall due to the PMP
is likely to be relatively small (on the order of 5 percentage points or
less), there are many confounding factors that were unable to be
included in the models (e.g., news stories on the census, conversations
with friends and family, and the innumerable influences on individuals'
mood and behavior that arise every day), and measurement of the
explanatory factors may be subject to various types of error (e.g.,
comprehension of survey questions, recall of messages or behavior,
identification of the correct household respondent). Thus, the
multivariate analysis should not necessarily be taken as evidence that
there is not a relationship between mail return behavior and PMP
activities, just as an indication that this particular method of
exploring the relationship (assuming it exists) is not finely tuned
enough to detect it.

Bentley (2003) conducted an auxiliary evaluation of the relationship
between PMP activities and Census 2000 return rates. Using county-level
return rates, a variety of indicators of PMP activities by geographic
area, and other control and explanatory variables, Bentley constructed a
series of statistical models in the same spirit as those developed by
Wolter et. al.. Essentially replicating the Wolter et. al. results, he
detected no relationship between the intensity or nature of PMP
activities and county-level return rates[21 p. iv, R131]{.mark}. Bentley
cites a number of limitations of his analysis, notably the lack of an
experimental design in the PMP. Again, the fact that no relationship was
detected does not demonstrate that a relationship does not exist.

Neither of these modeling efforts could include consideration of the
effects of the advance letter and mandatory notice on the questionnaire
outer envelope, since these features were included for all households.
Tortora et. al. (1993), commenting on the likely effects on return rates
in Census 2000 of the various mail piece innovations, noted:

> "In the past, the Census Bureau has obtained somewhat lower response
> rates in noncensus years than in census years . . . The usual
> explanation for this difference is 'census climate', a succinct
> explanation of the combination of media attention, advertising, and
> cultural sense of participation that seems to build during each census
> year . . . We do not know whether the existence of a 'census climate'
> will substitute for the effects of these elements or add to the
> response likely to be obtained in a census year[27, p. 123,
> R132]{.mark}."

In fact, this is the question that remains unanswerable. The modeling
efforts of Wolter et. al. and Bentley tried to correlate return rates
and different levels of intensity of "census climate" as stoked by the
PMP, and could not detect a relationship. The mail piece innovations may
be the key elements that stemmed the ebbing tide of return rates, but
there is no direct statistical evidence to support that contention. One
small argument for the mail pieces is presented by Dillman et. al.
(1996). The 1994 field test combined three motivational appeals: a
statement that response was mandatory, statements about the benefits of
participation, and varying levels of assurance of the confidentiality of
responses. The latter two kinds of appeal could be thought of as
elements in the general category of "census climate" features, although
weak ones. The effects of the mandatory statement dwarfed any effects
the other appeals had on return rates[24, p. 385, R91]{.mark}.

**2.2.6 Disaggregating the effects of PMP component activities**

If a direct link between the PMP and mail return behavior cannot be
demonstrated statistically from the available evaluation data, one would
certainly not expect to find evidence of the direct effects of
individual PMP components. The evaluation studies do provide some
indirect insights into the relative success of the components, however.

The PMPE measured awareness of census information from a variety of mass
media and community-based sources in each of its three waves. Awareness
of mass media (television, magazine, radio, newspaper, and billboard)
messages increased significantly each wave among most race and ethnic
groups, with the exception of magazine awareness, which did not increase
significantly between Wave 2 and Wave 3[29, pp. 43-44, R60]{.mark}.
Television had the highest mean awareness among all groups, with radio
and newspapers next[29, pp. 43-44, R60]{.mark}. The Census 2000 Dress
Rehearsal study found the same ranking among mass media in respondent
awareness. In the CMS, more than twice as many people reported seeing or
hearing "a lot" about the census in TV commercials as in radio or
newspaper advertisements[12, p. 22, R100]{.mark}. Reported exposure more
than doubled for each medium between the weeks of March 3 and March 31,
then leveled off for the final week[12, p. 22, R100]{.mark}.

The PMPE did not distinguish between advertising and news when asking
about mass media, but the CMS did, although it is not clear whether
respondents would consistently be able to make this distinction.
Exposure to news stories about the census on television, on the radio,
and in newspapers was somewhat lower (in the case of television almost
half) than to advertising in the same sources. As with advertising, news
exposure increased steadily for the first four weeks, then leveled off
for the final week[12, p. 22, R101]{.mark}.

In the PMPE, community-based sources that reached about the same levels
of awareness as some of the mass media were informal conversations,
census job announcements, signs or posters inside buildings, and
articles[29, p. 46, R61]{.mark}. Awareness of each of these sources
increased between Wave 1 and Wave 2, but only informal conversations and
signs or posters increased significantly between Wave 2 and Wave 3[29,
p. 46, R61]{.mark}. Very similar patterns of awareness were found among
all race and ethnic groups[29, pp. 46-49, R62]{.mark}. None of these
sources was included in the CMS. Speeches by government or local leaders
had the highest exposure rate among the community-based sources included
in the CMS, and their exposure increased steadily over the survey
period[12, p. 23, R102]{.mark}. Religious groups, local community or
government organizations, "things children brought from school," and
school-related activities had lower and fairly constant levels of
exposure during the survey period[12, p. 22, R103]{.mark}. While
awareness of messages from these sources was also at lower levels in the
PMPE survey, awareness increased significantly between Wave 1 and Wave 2
for each of them, less so between Wave 2 and Wave 3[29, p. 46,
R61]{.mark}. The remaining sources included in the PMPE survey,
conference exhibit booths, the Internet, paycheck or utility bill
inserts, and participation on complete-count committees, had the lowest
levels of awareness throughout and exhibited some increases across
waves[29, p. 46, R61]{.mark}.

In Wave 3 of the PMPE survey, about 12 percent of respondents said they
had heard "a little" about Census 2000 from materials their children
brought home from school, and about 3 percent reported hearing "a lot."
[29, p. 36, R63]{.mark} While it is impossible to compare estimates of
the reach of the CIS Program between the PMPE and the CIS Evaluation
with any precision, these numbers are not inconsistent with the CIS
Evaluation figures cited earlier. In the Census 2000 Dress Rehearsal
survey, non-Hispanic African-Americans (in South Carolina) and
Asians/Pacific Islanders (in Sacramento) reported more exposure to
school-based sources (their own or their children's schools) than did
non-Hispanic Whites[16a, p.51, 16b, p. 53, R133]{.mark}. In the PMPE
survey, all other race and ethnic groups reported more exposure to
school-based sources than did non-Hispanic Whites, although the
statistical significance of the difference cannot be determined from the
published report[29, pp. 46-49, R62]{.mark}.

Wolter et. al. also compared the correlation between intended
participation and awareness of mass media and community-based
communications in Wave 1 and Wave 2. For mass media, the correlations
increased significantly between waves for the general population,
Hispanics, non-Hispanic African-Americans, non-Hispanic Whites, and
Native Hawaiians[29, p. 66, R64]{.mark}. For community-based
communications, the correlations increased for all of the same groups
except Hispanics[29, p. 66, R65]{.mark}. The correlations between
awareness of both kinds of communication and intended participation rose
for Asians, but not significantly, and did not rise for American
Indians[29, p. 66, R64 and R65]{.mark}. The CMS asked whether the
advertising (or news) exposure "make me feel more like taking part" in
the census. For each, "yes" responses rose sharply between March 3 and
March 10, then dropped off slowly. Overall, about half of respondents
said "yes" for advertising and for news reports. [12, p. 22,
R104]{.mark}

The Survey of Partners identified what activities partnering
organizations conducted, but most of these activities cannot be directly
related to items in the PMPE survey or CMS. Vehicles for communication
with target audiences included printing and distributing materials (37
percent of partners), using print media (34 percent), holding public and
in-house meetings (33 percent), distributing census promotional items at
meetings and events (30 percent), sponsoring local media coverage (19
percent), and including messages in utility bills, phone cards, etc. (13
percent) [28, p. 29, R134]{.mark} (Westat, 2001).

Finally, Wolter et. al. examined the association between awareness of
various information sources and actual return behavior across the 1980
KAP survey, 1990 OES, and the PMPE survey's Wave 2 and Wave 3. They
found no consistent pattern of association across the three studies and
four data points, although there was variation within each[29, p. 92,
R66]{.mark}.

**2.3 Discussion of Individual Evaluation Studies' Strengths and
Limitations**

The studies commissioned to evaluate the Partnership and Marketing
Program for Census 2000 represent the most comprehensive effort of this
kind to date. The three major studies complement each other well. The
PMPE provides the largest amount of information related to the final
outcomes of interest -- increasing or at least slowing the decrease in
mail return rates, and reducing the differential undercount of various
race and ethnic groups. The privately-funded CMS provides a
"pulse-taking" look at what happened during the peak period of the PMP
week by week, which the PMPE was not designed to do. The Survey of
Partners supports more of a process evaluation for one component of the
PMP than an outcome evaluation. We will discuss each study in turn. The
following section discusses evaluation issues that span the studies.

**2.3.1 Survey of Partners**

Of the three Census-commissioned evaluations, this study was most
process-oriented. It provides a good quantitative assessment of the
number and level of involvement of active partners. The Survey of
Partners did not attempt to measure person-level outcomes in terms of
awareness, beliefs, or participation in Census 2000, even though the
ultimate goals of the Partnership Program were very much linked with
those of the advertising campaign and other PMP activities. In fact, one
of the objectives of the PMP model was to reduce resistance to the
census among the hardest to enumerate to "pave the way for community
programs," which would likely have come via the Partnership Program. The
decision not to attempt to measure such outcomes retrospectively was
probably a good one. However, it might be possible in future evaluation
efforts to measure the "reach" of partners prospectively, using a
community case study approach.

**2.3.2 Census in Schools Evaluation**

This survey-based evaluation provided more detail on the effectiveness
of one major component of the Partnership and Marketing Program, an
attempt to introduce Census 2000 into the classrooms of the country's
elementary and secondary schools. Like the Survey of Partners, it too is
more of a process than an outcome evaluation, useful for assessing the
implementation of the program rather than its effect on Census 2000
return rates.

**2.3.3 PMPE Survey**

The three-wave (baseline, pre-census, post-census) survey design was
employed to address the target populations in manners most appropriate
to their circumstances. A mixture of telephone and in-person survey
modes was used to ameliorate the difficulties in sampling and surveying
some hard-to-count populations. In the first wave, the expected sample
frame, the Decennial Master Address File (DMAF), was not available so a
random-digit-dial (RDD) approach was implemented for surveying the
general population and oversamples of Hispanics, non-Hispanic African
Americans, and native Hawaiians[29, p. 8, R67]{.mark}. As described in
more detail below, the Wave 1 response rate was lower than the Wave 2
and 3 administrations, which used the DMAF[29, p. 10, R68]{.mark}.

Limitations of the design include issues of sampling error, nonresponse,
frame undercoverage, and response error. Appropriate adjustments and
caveats were used in the analysis and reporting, and are noted in this
report where applicable. For one of the more problematic populations,
American Indians, the conclusion drawn is that undercoverage "...should
be comparable to that achieved for this population in the Census 2000
itself." (Wolter et. al., 2002, p. 10) [29, p. 10, R32]{.mark}

One limitation of the evaluation was particularly highlighted by the
authors: the inability to clearly and separately quantify the effects of
the mass-media campaign and effects of the partnership program[29, p.
11, R69]{.mark}. Due to the variety of potential influences possible
from the two programs, as well as other census-related stimuli (e.g.,
news reports, etc.), the time periods covered, and known problems in
respondent recall precision, the separation of effects was seen to be a
daunting task. For this quantification, an experimental approach was
recommended in future evaluations.

Some specific comments on the PMPE (others will be incorporated in topic
areas below):

-   While the PMPE survey included questions specific to messages in the
    advertising campaign, analysis of these items was not included in
    the report. There were also intended to be somewhat different
    messages in different phases of the campaign---it is not clear from
    the materials provided what these variations were, nor whether the
    variations were covered in the PMPE questionnaire, which had the
    same content regarding messages in each Wave.

-   It would have been instructive for the analysis of the relationship
    between beliefs about the census and reported participation to have
    been extended to (or replaced by) the same assessment using actual
    participation.

-   The factor analysis of PMPE belief items and the use of a composite
    belief variable was illuminating. However, for comparison with
    previous research further item-by-item analysis would have been
    helpful. It is also not clear whether the lack of movement in the
    belief composite between Wave 2 and Wave 3 could have been due to
    the performance of particular items, notably the negative beliefs
    stimulated by the negative long form publicity.

**2.3.4 Census Monitoring Survey**

While not a census-sponsored evaluation activity, this study provided
very interesting and useful week-by-week snapshots of the populations'
reactions to the PMP and the mailout/mailback operation. The CMS
provided the flexibility to respond to breaking events, such as the
controversy over the long form, perhaps its most valuable contribution
in 2000. Unfortunately, two planned components of this effort,
additional rounds of interviewing into the NRFU period and collection of
actual return behavior, apparently did not happen. These features would
have provided additional useful insights.

The CMS is a complement to the PMPE, and an entirely different kind of
survey. While the effective response rates are somewhat lower to
considerably lower than those of the PMPE survey waves, its temporally
fine-grained view of the effects of the advertising campaign can't be
replicated in the PMPE style of survey design without compromising the
response rate, which depends on repeated contacts over a period of
several weeks.

**2.4 Cross-study Evaluation Topics**

**2.4.1 Experimental Design**

The studies evaluating the effects of the Partnership and Marketing
Program and the Census 2000 Dress Rehearsal were observational. The
limitations of observational designs in the establishment of cause and
effect are clearly stated in the NORC report on the PMPE[29, p. 11,
R69]{.mark} (also see Cook and Campbell, 1979). (NORC's final
recommendation is to include an experimental design in future
evaluations.) From an evaluative perspective, it certainly would be
easier and less ambiguous if it were possible to partition people into
real-world groups receiving measured doses of advertising and/or
exposure to partnership activities. Classical (e.g., agricultural)
experimental designs have the power of controlling sources of variation
and so allow the establishment of causation and the quantification of
effects. An experiment might eliminate possible confounding of mass
media and Partnership activities. Indeed, people included in such an
experiment would not even have to report exposure to such activities.
Activity and exposure levels would be set and known.

We do not think, however, that it would be feasible to design a
meaningful experiment for the evaluation of the effects of mass media on
awareness and behavior. By its very nature, mass media is "out there"
for all to see so the control of its reach and frequency for selected
groups is at least problematic if not actually impossible. It is
difficult to imagine successful control of media outlets in a manner
that would allow selection of differential doses for selected groups in
the same way that soil Ph and moisture might be controlled in a
classical experiment.

Designing an experiment evaluating the effects of the Partnership
Program is more conceivable, although any such design would likely still
be confounded by the mass media campaign. Partnership activities could
be restricted to selected geographic areas so that experimental and
control groups could be established. This would not establish the dose
or mix of treatments received, however, for surveyed individuals (and
recall, as noted above, can be a poor indicator). Although perhaps
somewhat more tractable, it would also be difficult to select initially
comparable experimental groups for treatments. At the least, such groups
would have to have comparable baseline attitudes and knowledge of the
census and similar patterns of census form return.

**2.4.2 Timing of the evaluation activities**

The PMP was carefully designed to build over time to a peak just before
and during the mailout/mailback operation, and to re-energize for the
NRFU operation. The evaluation activities were designed around the PMP
and Census 2000 operations. Wave 1 and Wave 2 of the PMPE were well
positioned to capture the effects of the Education phase of the paid
advertising campaign. Wave 3's position does not allow such a clean look
at a particular part of the PMP: between Wave 2 and Wave 3 was the
Motivation phase of the advertising campaign and the mailout/mailback
operation. The NRFU effort and the associated advertising campaign were
active while Wave 3 was in the field. The quantity and volume of ambient
information about the census also peaked during the period between Wave
2 and Wave 3, and into the Wave 3 field period. Finally, the Wave 3
field period extended almost a month after the end of NRFU activities,
which may have resulted in increased recall error for the later
interviews. It is thus impossible for the PMPE to disentangle the
effects of these different events, but it does provide useful
information on the cumulative effects of all of the influences on census
behavior.

The CMS fit nicely into the gap between the PMPE's Wave 2 and Wave 3 to
offset the limitation just described. The week-by-week design allowed
tracking during the peak period of PMP activity. Unfortunately, since
the planned last two weeks of the CMS, during the NRFU, apparently did
not happen, there is no way to isolate the effects of the NRFU phase of
the advertising campaign.

The major shortcoming of the Survey of Partners was the timing of the
field period, many months after Census 2000 operations. Some contact
persons at partner organizations had left the organization, and others
had difficulty recalling details of their involvement. Thus, the survey
probably underestimates somewhat both the number of active partners and
the level of involvement of active partners. About one-quarter of
sampled organizations from the Census Bureau's database were never
contacted during the survey, despite multiple attempts both through the
mail and over the telephone. The timing of the survey undoubtedly
contributed substantially to this low rate of contact.

**2.4.3 Civic activities**

Because the Y&R model of response likelihood was based on the assumption
that general civic participation would be correlated with returning the
census form, it might have been useful for the PMPE to have put somewhat
more emphasis on civic participation in its design and analysis. The
PMPE questionnaire for each wave included seven questions on civic
participation. Given the distribution of responses in the PMPE survey
(Wolter et. al., 2002, Figure 16[29, p. 23, R70]{.mark}) compared with
the Y&R estimates (Wolter et. al., 2002, Figure 1[29, p. 2, R71]{.mark})
it seems likely that the Y&R model was based on more items. Thus, it is
probably not appropriate to compare the two sets of estimates. Two
points are worth noting, however. First, the level of no civic
participation in the general population in the Y&R model (17 percent[29,
p. 2, R71]{.mark}) is almost identical to that found in the PMPE survey
in Wave 1 and Wave 2[29, p. 23, R70]{.mark}. Second, the distribution of
amount of civic participation drops in Wave 3, with those with no
activities increasing to about 25 percent and those with three or more
activities dropping from about 24 percent to about 10 percent[^11][29,
p. 23, R70]{.mark}. The decrease in civic participation (of those
reporting at least one activity) between Wave 1 and Wave 3 is
statistically significant for the general population, for non-Hispanic
African-Americans, and for non-Hispanic Whites[29, p. 86, R72]{.mark}.
The PMPE report does not explore any possible relationship between this
difference in civic participation across waves and any of the findings
with regard to awareness, beliefs, or behavior.

The few findings presented in the PMPE report relating to civic
participation provide only very limited support for the Y&R likelihood
model. There were significant differences (in the expected direction) in
recent census awareness by level of civic participation for the general
population, for non-Hispanic African-Americans, for Asians, and for
American Indians. There were similar but non-significant differences for
Hispanics, non-Hispanic Whites, Asians, and Native Hawaiians[29, p. 59,
R73]{.mark}. None of the final multivariate models described in the
report found a significant relationship between civic participation and
intended or actual participation (when controlling for other factors).
While there may have been a positive correlation between civic
participation and returning the census form, other factors would appear
to explain that correlation if it exists.

**2.4.4 Survey nonresponse**

The response rates for PMPE Wave 1, Wave 2, and Wave 3 were 48.4
percent, 64.5 percent, and 67.7 percent, respectively[29, p. 10,
R68]{.mark}. Wolter et. al. acknowledge that nonresponse is a potential
source of bias in the findings. Generally, nonresponse bias is more
likely when variables of interest are correlated with the propensity to
respond to the survey. Intuitively, one would expect a correlation
between responding to a telephone or in-person survey and mailing back
the census form. Thus, there is reason to be concerned about nonresponse
bias in the PMPE survey and others that seek to evaluate the effects of
PMP or other activities on the census return rate.

Table 5 shows the final weighted and unweighted return rates for
participants in Wave 2 and Wave 3 of the PMPE survey and corresponding
final return rates[^12] for the population as a whole. For Whites and
American Indians, the rates are reasonably comparable, but for
Hispanics, African-Americans, and Asians the survey respondent return
rates are 10 or more percentage points higher than the population rates.
Some of this difference may be due to the effects of the survey, in
essence acting as a motivating factor. However, by extrapolation, within
these race and ethnic groups survey nonrespondents had a 50 percent or
lower final census return rate, indicating that there is some other
systematic difference between respondents and nonrespondents in these
groups. While it is impossible to know what the bias would be, we
surmise that some of the differences between the White and other
populations in awareness, beliefs, etc., would have been even greater
had there been no survey nonresponse. Note that the weighted return
rates for survey participants are generally higher than the unweighted
rates, so the weighting does not ameliorate the apparent bias.

**Table 5. Final return rates for PMPE Wave 2 survey respondents by
sample and for corresponding groups in the general population**

  ------------------ -------------- ------------- ---------- ------------- ----------------------- ------------------
                     Final return                                                                  Difference^HH^
                     rate                                                                          between

                     Unweighted\*   Weighted\*    Census     PMPE Survey   Estimated final return  respondents\' and
                                                  2000 final response      rate for survey         non-respondents'
                                                  return     rate\*^H^     non-respondents\*\*\*   final return rates
                                                  rate\*\*                                         

  **Wave 2**         [29, p 88,     [29, p 88,               [29, p 161,                           
                     R76]{.mark}    R76]{.mark}              R74]{.mark}                           

  Total Population   81.1%          84.4%                    57.8%                                 

  Hispanic           81.9%          81.7%         69.2%                    51.8%                   30.1

  African-American   74.8%          76.6%         64.3%                    49.9%                   24.9

  White              87.1%          87.1%         86.8%                    86.4%                   0.7

  Other              79.0%          79.3%                                                          

  Asian              88.5%          89.8%         74.6%      72.2%         38.6%                   49.9

  American Indian    72.3%          74.7%         70.7%      71.2%         66.8%                   5.5

  Native Hawaiian    76.8%          79.2%         59.4%      73.0%                                 

                                                                                                   

  **Wave 3**                                                 [29, p 163,                           
                                                             R75]{.mark}                           

  Total Population   80.1%          80.9%                    64.6%                                 

  Hispanic           83.5%          89.2%         69.2%                    47.6%                   35.9

  African-American   73.5%          74.6%         64.3%                    50.4%                   23.0

  White              83.9%          81.7%         86.8%                    91.1%                   -7.2

  Other              79.0%          60.9%                                                          

  Asian              88.3%          85.3%         74.6%      55.6%         35.5%                   52.9

  American Indian    69.2%          71.1%         70.7%      74.6%         74.3%                   -5.1

  Native Hawaiian    76.1%          78.0%         59.4%      72.6%                                 
                                                  ^HHH^                                            
  ------------------ -------------- ------------- ---------- ------------- ----------------------- ------------------

\*Source: Wolter et. al. (2002)

\*\*Source: Stackhouse and Brady (2002)

^H^"Unweighted response rate 3" from Appendix B

\*\*\*Calculated from unweighted return rate, Census 2000 final return
rate, and PMPE survey response rate

^HH^ Difference in percentage points

^HHH^ U.S. population value is for Pacific Islanders

Martin (2001) discussed the response rates and potential for bias in the
CMS. The response rates for the five weekly CMS surveys ranged from 58
percent to 83 percent. However, the sample frame for these surveys was
the InterSurvey panel, recruited using an RDD sample design. The
combined response rate thus averaged around 30 percent, considerably
lower than the rates for the PMPE. Martin made the following
observations about the potential for bias in the CMS:

-   The demographic composition of the samples match reasonably well
    with that of the Current Population Survey, except that the CMS
    under-represents those with less than a high school education and
    over-represents voters;

-   Likely biases are in Census 2000 participation rates (participation
    is highly correlated with voting) and concerns about privacy
    (nontelephone households are not included in the CMS samples, and
    households with unlisted numbers are under-represented);

-   Any biases that are present are more likely to affect estimates of
    levels, and less likely to affect trend estimates, since the bias
    should be relatively constant over the surveys[32, pp. 1-2,
    R135]{.mark}.

The Survey of Partners achieved an overall response rate of 68 percent.
The refusal rate was only about 7 percent; most nonresponse was due to
difficulty in contacting partner organizations[28, p. 7, R136]{.mark}.
As noted earlier, this difficulty was likely due in part to the timing
of the survey, and probably resulted in under-estimating partner
participation in Census 2000 activities, since, for example, some
contact staff responsible for census-related activities were no longer
with the partner organization.

  ------------ --------- -------- ---------- ---------- ---------- ----------
                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   

                                                                   
  ------------ --------- -------- ---------- ---------- ---------- ----------

\*Source: Wolter et al (2002)

\*\*Source: Stackhouse and Brady (2002)

Nonresponse could be a serious issue for the CIS Evaluation survey,
although it perhaps is of less concern from a process evaluation
perspective than from the perspective of producing population estimates.
Only 28 percent of the sample returned questionnaires, and 26 percent
were ultimately included in the analysis[25, p. 2, R137]{.mark}. As
noted by the report's authors, "Our expectation is that teachers who did
not hear of the program or did not use the CIS materials were less
willing to respond to the survey. [25, p. 2, R138]{.mark}" This
intuitively reasonable observation suggests that the survey may
over-estimate awareness of the CIS Program and use of its materials. The
fact that the response rate among teachers sampled from the orderers'
list (32 percent) was only a little higher than that among those sampled
from the commercial list (27 percent) [25, Appendix A, p. 11,
R139]{.mark} provides some reassurance about this potential for bias,
however. Nonetheless, one should interpret level estimates with caution.
It is less clear how non-response may have affected comparisons, such as
between HTE and non-HTE areas.

**2.4.5 Language spoken at home**

One important area where nonresponse may be an issue is with
linguistically isolated individuals. Of the various studies evaluating
the PMP, only the PMPE survey has much to say about such people. The
survey was conducted in English and Spanish. Some number of households
were not screened, and some number of interviews were not conducted,
because of language barriers.[^13] We cannot tell whether translators or
proxy interviews were allowed for languages other than Spanish, but some
number of interviews were conducted where a language other than English
was spoken at home.

Total nonresponse due to language and incapacity in Wave 1 ranged from
about 2.5 to 3.5 percent in the core, American Indian, and Asian
samples, and was about 9 percent in the Native Hawaiian sample[29, p
158, R77]{.mark}. In Wave 2, it was between 2 and 2.5 percent for the
core, Asian, and Native Hawaiian samples[29, p 160, R78]{.mark}. In Wave
3, it was about 2.5 percent for the core, 4 percent for the Native
Hawaiian sample, and 9 percent for the Asian sample[29, p 162,
R79]{.mark}. There was virtually no nonresponse for these reasons in the
American Indian sample in Waves 2 and 3[R78 and R79]{.mark}.

"Language spoken at home" was used several times as an explanatory
variable in the PMPE report, broken down as English, Spanish, and Other.
The findings include:

-   Awareness of census communications increased significantly for all
    languages between Wave 1 and Wave 3, but less so for Spanish and
    Other than for English[29, p 51, R80]{.mark};

-   The correlation between awareness of census communications
    generally, and of mass media communications, and intended
    participation increased from Wave 1 to Wave 2 about the same for
    English, Spanish, and Other language spoken at home[29, p 69,
    R81]{.mark};

-   The correlation between awareness of community-based communications
    and intended participation did not increase significantly for Other,
    while it did for English and Spanish in the core sample and for
    English in the Native Hawaiian sample[29, p 70, R82]{.mark};

-   In the multivariate analyses, households speaking an Other language
    at home were significantly more likely to return their census forms
    than were English-at-home households[29, p 116, R83]{.mark}.

The last finding in particular is counter-intuitive; one would expect
linguistically isolated households to have a lower return rate. The
authors discount the findings about Other-language households because of
small sample sizes. However, nonresponse bias may be particularly acute
here---truly linguistically isolated households (other than those
speaking Spanish) would apparently not have been able to complete the
survey.

**3. Summary of Results**

Through its continuing program of methodological research and innovation
to improve the census, the Census Bureau was able in Census 2000 to
reverse the downward trend in mail return rates and reduce the
differential undercount. The combination of several major evaluation
efforts for the Census 2000 Partnership and Marketing Program provides
much insight into the effects of the PMP, and the connection between the
PMP and the other successes is intuitively compelling. However, the
grand prize of the evaluation activities, a direct connection between
the PMP interventions and return rates, remains elusive.

Here is what we believe we can say with confidence from the evaluation
data we have examined:

-   The mandatory notice on the questionnaire's outer envelope had a
    positive effect on return rates.

-   The Partnership Program and the Census in Schools Program were
    relatively successful in reaching out to hard-to-enumerate
    populations, as evidenced by the kinds of constituencies active
    partners reported in the Survey of Partners and by the levels of
    awareness and use of materials reported in the CIS evaluation
    survey, although it is not possible to quantify their impact in
    terms of number of individuals reached or increases in participation
    rates.

-   The PMP, and the paid advertising program in particular,
    dramatically increased awareness of the census among the general
    population, and among certain traditionally hard-to-enumerate race
    and ethnic groups.

-   Print media coverage of Census 2000 nationally was much broader than
    in 1990, and probably more positive in tone overall.

-   The PMP increased the proportion of positive beliefs about the
    census up to the time of the mailout.

-   The effects on proportion of positive beliefs vary by belief and by
    race and ethnic group.

-   Some positive beliefs about the census are associated with increased
    reported participation, and the association varies somewhat by race
    and ethnic group.

-   Organizations including many targeting traditionally
    hard-to-enumerate populations, conducted a large number of
    activities in support of Census 2000, with cash and in-kind
    contributions exceeding \$500 million.

The following statements are indirectly supported by the evaluation data
or other research:

-   News stories about politicians saying that the long form is an
    invasion of privacy had a negative effect on return rates for the
    long form; CMS data are persuasive on this point.

-   The advance letter is likely to have had a positive effect on
    response rates.

-   The respondent-friendly questionnaire design is also likely to have
    had a positive effect, and the effect may have been greater in
    hard-to-enumerate areas.

-   Receipt of the mailout package, particularly the long form, may have
    increased some negative beliefs about the census.

-   Nonresponse to the PMPE and CMS probably results in an underestimate
    of the differences between the non-Hispanic White population and the
    Hispanic, non-Hispanic African-American, and Asian populations.

-   Nonresponse to the PMPE and CMS also results in an underestimate of
    the differences between English-speaking and linguistically isolated
    households in awareness and behavior with regard to the census.

**4. Recommendations**

It is very hard, if not impossible, from the statistical evidence to
attribute the relative success of Census 2000 to the PMP or its
components. The fact remains that Census 2000 was much more successful
than predicted, and more successful than the 1990 Census. It seems
reasonable to take the view that a number of changes were made, many
incorporated into the PMP, and overall they were successful. Thus, it
seems reasonable that this successful strategy should be continued, with
refinements that appear warranted in the spirit of continuous
improvement.

It is very likely that including the message that participation is
required by law on the mailout envelope had a positive effect on return
rates, and should be continued. An experimental test of this message in
a noncensus year increased returns by about 10 percentage points
overall. Otherwise, we do not feel that the data warrant recommendations
about continuing specific aspects of the PMP or not.

The lack of an experimental design was a principal reason for not being
able to associate the return rate success of Census 2000 with PMP
components. It does not seem sensible to embed an experimental design in
a decennial census, but it may be possible to do so in a dress rehearsal
or field test carried out in discrete geographic locations. In
particular, it would be instructive to vary the PMP or "census climate"
factors as a whole against different features of the mail pieces.

The data accumulated over three censuses should be able to provide
guidance on what beliefs about the census (a) are associated with return
propensity and (b) can be influenced by marketing and communication
efforts, differentially by race and ethnicity. We recommend that the
next marketing and communication campaign review these findings to
identify particular messages to include for particular segments of the
population.

Given that general civic participation does not seem to be a primary
factor in return propensity (although the two may be correlated), we
recommend a review of the Y&R model to see how its segmentation strategy
might be different if other factors were used as the basis.

We recommend a review of PMP activities, including the segmentation
strategy, to assess how they might better reach younger adults and those
who speak languages other than English and Spanish at home. New
formative work for these groups may also be warranted.

Regarding evaluation activities for the next census, we recommend
continuing PMPE- and CMS-type surveys. We recommend extending the CMS to
go into the NRFU period, assuming that activities targeted to the NRFU
remain a part of the PMP, and obtaining actual participation information
on CMS households to include in the analysis. It would be desirable for
Wave 3 of the PMPE to coincide more closely with the NRFU period (i.e.,
rather than extending beyond it), although a truncated field period
would undoubtedly lower the response rate somewhat.

While desirable, it is unlikely that the next survey would be any more
successful than the 2000 PMPE at connecting the PMP or its components
directly with return behavior. We recommend that the analysis of the
next PMPE survey focus more on the marketing strategy itself and its
messages, with an eye toward continuous improvement.

The Survey of Partners was a useful process evaluation, and should
provide some guidance in planning and implementing a future Partnership
Program. If the survey is repeated, it should be integrated into the
overall schedule so as to be fielded as soon after partner activities
subside as possible. It would also be helpful to provide partners with
some of the evaluation questions during the Program activities, so more
(and more reliable) quantitative information would be available for the
survey.

If funds are available, new kinds of evaluation activities might be
useful. For example, it may be possible to do more comprehensive
evaluations at a local level, using ethnographic and/or other
qualitative methods as well as surveys and other quantitative
methods[^14]. A locally-based "case study" approach could begin to
quantify, for example, the reach of local and national partner
activities or local media relations efforts. It could also assess
influences on the behavior of linguistically isolated households more
efficiently than a national survey. Combining qualitative and
quantitative methods locally would also provide more depth to the
analysis, helping to make causative connections not possible with broad
quantitative data alone.

**References**

Bates and Whitford (1991), "Reaching Everyone: Encouraging Participation
in the 1990 Census," *Proceedings of the Social Statistics Section*,
American Statistical Association, 507-512.

Bentley, M. (2003), *Census 2000 Auxiliary Evaluation: Evaluation of
Partnership and Marketing on Improving Census 2000 Mail Return Rates*,
Washington, D.C., U.S. Census Bureau, April 3. [Reference 21.]{.mark}

Citro, C.F., D.L. Cork, and J.L. Norwood, Eds. (2001), *The 2000 Census:
Interim Assessment,* National Academy Press, Washington DC. [Reference
20.]{.mark}

Clark, J.R., D.A. Dillman, and M.D. Sinclair (1993), "How Prenotice
Letters, Stamped Return Envelopes and Reminder Postcards Affect Mailback
Response Rates for Census Questionnaires," *Annual Research Conference
Proceedings*, Bureau of the Census, 37-48. [Reference 22.]{.mark}

Cook, T.D. and Campbell, D.T. (1979). *Quasi-Experimentation - Design &
Analysis Issues for Field Settings*, Rand McNally College Publishing
Company, Chicago.

Dillman, D.A., M. Sinclair, and J.R. Clark (1993), "Effects of
Questionnaire Length, Respondent-friendly Design, and a Difficult
Question on Response Rates for Occupant-addressed Census Mail Surveys,"
*Public Opinion Quarterly* 57: 289-304. [Reference 23.]{.mark}

Dillman, D.A., E. Singer, J.R. Clark, and J.B. Treat (1996), "Effects of
Benefits Appeals, Mandatory Appeals, and Variations in Statements on
Confidentiality on Completion Rates for Census Questionnaires," *Public
Opinion Quarterly* 60: 376-389. [Reference 24.]{.mark}

Douglas Gould and Co. (2001), "Media Analysis of Census 2000,"
Washington, D.C., U.S. Census Bureau, February 28. [Reference 9.]{.mark}

Fay, R.E., N. Bates, and J. Moore (1991a), "Lower Mail Response in the
1990 Census: A Preliminary Interpretation," *Annual Research Conference
Proceedings*, Washington, D.C., U.S. Census Bureau, 3-32. [Reference
18.]{.mark}

Fay, R. E., Carter, W., Dowd, K. (1991b) \"Multiple causes of
nonresponse: Analysis of the Survey of Census Participation.\"
*Proceedings of the Social Statistics Section*, American Statistical
Association, 525-530.

Kulka, R. A., Holt, N. A, Carter, W., and Dowd, K. (1991) \"Self-reports
of time pressures, concerns for privacy, and participation in the 1990
mail census.\" *Annual Research Conference Proceedings*, Washington,
D.C., U.S. Census Bureau, 33-54.

Macro International (2002), *Census 2000 Evaluation D.2: Evaluation of
the Census in Schools Program: Materials and Distribution*, Washington,
D.C., U.S. Census Bureau Planning, Research, and Evaluation Division,
July 10. [Reference 25.]{.mark}

Martin, E. (2000), \"Changes in public opinion during the census.\"
Paper presented at the Census Advisory Committee of Professional
Associations, Virginia, Oct. 19, 2000. [Reference 33.]{.mark}

Martin, E. (2001), "Privacy Concerns and the Census Long Form: Some
Evidence from Census 2000," *Proceedings of the Section on Survey
Methods Research*, American Statistical Association. [Reference
32.]{.mark}

Martin, E., and E. Rivers (2001), "A Look at Some Preliminary Results of
Evaluating the Census 2000 Integrated Marketing Strategy," paper
prepared for presentation at the American Association for Public Opinion
Research. [Reference 26.]{.mark}

Martin, B., J. Moore, N. Bates, H. Woltman, D. Hubble, and E. Rivers
(undated), "Project Overview---Census Tracking Study," Washington, D.C.,
U.S. Census Bureau, unpublished document. [Reference 12.]{.mark}

Moul, D.A. (2002), *Census 2000 Evaluation H.5: Nonresponse Followup
forCensus 2000*, Washington, D.C., United States Census Bureau.
[Reference 2.]{.mark}

Nie, N., and J. Junn (2000), "Census Monitoring Study Summary Findings,"
press release, May 4, 2000. InterSurvey, Inc. [Reference 12.]{.mark}

Roper-Starch (1999), *Promotion Evaluation: Evaluating the Effectiveness
of Paid Advertising in the Census 2000 Dress Rehearsal in Sacramento and
South Carolina: Survey Findings*, report prepared for the U.S. Census
Bureau. [Reference 16.]{.mark}

Sha, B., and L.V. Collins (forthcoming), "Communicating the Importance
of Civic Participation in Census 2000: *How America Knows What America
Needs*," draft chapter for L.L. Kaid, D.G. Bystrom, M.S. McKinney, and
D.B. Carlin, eds., *Communicating Politics: Engaging the Public in
Democratic Life*, in press. [Reference 9.]{.mark}

Stackhouse, H.F., and S. Brady (2002), *Census 2000 Evaluation A.7.b:
Census 2000 Mail Return Rates*, Washington, D.C., United States Census
Bureau. [Reference 1.]{.mark}

Tortora, R.D., S.M. Miskura, D.A. Dillman (1993), "Onward Towards a 2000
Census Design: Research Results," *Proceedings of the Section on Survey
Research Methods,* American Statistical Association, 120-128. [Reference
27.]{.mark}

Westat (2001), *Census 2000 Evaluation D3: Report of Survey of
Partners*, Washington, D.C., U.S. Census Bureau Planning, Research, and
Evaluation Division, November 19. [Reference 28.]{.mark}

Wolter, K., B. Calder, E. Malthouse, S. Murphy, S. Pedlow, and J. Porras
(2002), "Census 2000 Evaluation: Partnership and Marketing Program
Evaluation," Washington, D.C., U.S. Census Bureau Planning, Research,
and Evaluation Division, July 17. [Reference 29.]{.mark}

U.S. Census Bureau (2001), *Regional Partnership Report: Portrait of
America*. Washington, D.C., U.S. Census Bureau. [Reference 30.]{.mark}

U.S. Census Bureau (2003), "Decision on Intercensal Population
Estimates," March 12, 2003, Washington, D.C., report. [Reference
31.]{.mark}

U.S. Census Bureau, Decennial Management Division (undated), "Census
2000 Partnership and Marketing Program: Program Master Plan,"
Washington, D.C., unpublished document. [Reference 19.]{.mark}

![](media/image2.wmf)

[^1]: The Census Bureau counted some 140,000 partners enlisted
    throughout the Partership Program. The Survey of Partners estimated
    that almost 86,000 organizations were eligible for the survey after
    accounting for duplicates in the master list and organizations
    reporting that they were not partners[28, p. 8, R14]{.mark}. Note
    that the survey occurred some months after partner activities had
    ended, so the survey estimate may be affected by changes in some
    partner organizations in the interim.

[^2]: Included were the 50 states, the District of Columbia, and Puerto
    Rico.

[^3]: These proportions were not presented in the CIS Evaluation report;
    the authors calculated them from tables presented in the report's
    Appendix.

[^4]: The report includes no item-by-item analysis of movement over
    time. In retrospect, it would have been useful to have this
    information available.

[^5]: In the 1990 OES, almost twice as many long form as short form
    recipients thought that the census was an invasion of privacy[18, p.
    17, R126]{.mark}.

[^6]: The level of agreement is also interesting. In the Census 2000
    Dress Rehearsal survey [16a p. 63, 16b p. 73, R123]{.mark}and the
    1999 KAP1[14, p. 34, R126]{.mark}, the levels of agreement were at
    20 percent or higher. Early in the CMS they were at 10 percent and
    went to 20 percent[12, p. 23, R94]{.mark}, the "historical norm."
    Published findings of the 1990 OES show only the percentage giving
    positive responses, which declined between Wave 1 and Wave 2 [18, p.
    15, R125]{.mark} (after receipt of the mailout package).

[^7]: Neither the PMPE nor the CMS report includes comparable
    information by race/ethnicity.

[^8]: The increase was statistically significant for all groups except
    American Indians.

[^9]: That is, returning a census form by April 18.

[^10]: In the Dress Rehearsal survey, knowledge that the census is
    mandatory also doubled between the pre- and post-campaign survey
    rounds, and the levels of knowledge at each period were very close
    to the early and late levels in the CMS[16a p. 72, 16b p. 82,
    R129]{.mark}. The 1999 KAP also found that about 22 percent of
    people knew the census was mandatory[14, p. 32, R130]{.mark}.

[^11]: Wolter et. al. suggest that seasonality may explain this
    difference, perhaps with a reduction in PTA participation as the
    summer approached, but were unable to test this hypothesis. Since
    the questionnaire asks about participation in the past 12 months,
    this explanation seems unlikely.

[^12]: We use final return rates rather than the rates through April 18
    because of discrepancies in the difference between the 4/18 and
    final rates in the Wolter et. al. report and for the U.S.
    population.

[^13]: Language problems and "incapacitation" were counted together in
    the PMPE report, Appendix B.

[^14]: Census 2000 activities included ethnographic research, but not
    specifically to help evaluate the PMP.