##### U.S. Census Bureau Responses to the

##### Recommendations of the 

**Census Advisory Committees of Professional Associations**

**Made as a Result of the Meeting on April 19-20, 2007**

# 

# Recommendation 1a 

**Chief Economist Update**

**[American Economic Association]{.underline}**

"The Committee commends the Census for its work with the Research Data
Centers and the significant progress that has been made with this
program. Of particular note is the progress that has been made in
reducing the turn-around time for processing applications by outside
researchers to do research in the centers. We also regard as a very
favorable development the January 2007 memorandum from the Census Bureau
Director acknowledging the value of scientific research taking place at
the Research Data Centers to advancing the mission of the Census Bureau.

The Committee was very interested to hear that raw data from the 1958
Census of Manufactures (CM), has been recovered and that there is a
discussion about converting this into computer data form to append to
the Longitudinal Research Database (LRD). The original LRD, constructed
in the mid 1980s linked the Census of Manufactures from the 1963, 1967,
1972, 1977, and 1982 Censuses. Since that time, as new Census data has
been collected, it has been added forward, 1987, 1992, etc. By adding
the 1958 CM, the Census would build onto the LRD by extending it
backward.

If this project goes forward, priority should be given to obtaining the
basic variables like receipts, employment, payroll, location, industry,
etc. In addition, it is important to be able to link the records to the
1963 file, so whatever records would be needed for this purpose
(facility name, etc.) need to be collected. (The methodology used to
link 1964 with 1967 should be revisited to see what variables were used
in this linking process.)

Having the 1958 data added would be of most value for research projects
that explicitly take into account particular events, e.g., macroeconomic
events or policy changes, or industry events (strikes, etc.) that
occurred between 1958 and 1963. For any such analysis, having the 1958
data would obviously be critical. In addition, any study wanting to
compare things before 1970 with after 1970 would also benefit, because
the "before 1970" case would now have three sets of observations (1958,
1963, 1967) rather than just two.

Having the 1958 file would only be of limited value to studies that are
not interested in historical issues. There are already nine Census years
in the LRD. The marginal contribution of adding the 1958 CM simply to
get more observations is limited.

The Committee commends the Census for producing a series of papers on
the retail sector. The retail sector is important yet has received
little attention as compared to manufacturing. The Census has rich
information on this sector that has not been fully exploited. This
series of papers represent a promising step in harvesting this
information and suggesting directions for future research."

**[Census Bureau Response]{.underline}**

The Census Bureau thanks the committee for its recognition of the
advances made to improve the Research Data Center (RDC) network, and it
will continue to make additional improvements. For example, since the
Advisory Committee meetings, we have reached agreement with the Agency
for Healthcare Research and Quality to offer its Medical Expenditure
Panel Survey household data through the RDCs, joining the National
Center for Health Statistics in offering its data.

Plans are unfolding for the capture of selected data from the 1958 CM.
The current proposal, should funds become available, is first to have
examples of completed survey forms scanned from microfilm and indexed by
form type (i.e., Annual Survey of Manufactures (ASM), or one of the over
200 industry-specific CM forms). We would focus initial data capture
efforts on ASM cases, which include the largest establishments. In most
cases, the ASM forms also contain annual data from 1954-1958, increasing
the returns to focusing on these cases. We propose to capture as much
data from these ASM forms as resources allow, including identifying
information (name, address, Census File Number, Employer Identification
Number), industry, geography (state and county), salary and wage
variables, employment data, value of shipments, cost of materials and
energy, and capital expenditures. The keying of more "exotic" ASM data
items will depend on resources available. The keying of other data from
the CM forms is a much larger and more costly endeavor and, thus, is
unlikely at this time.

# Recommendation 1b 

#### The Role of Retail Chains: National, Regional, and Industry Results {#the-role-of-retail-chains-national-regional-and-industry-results .unnumbered}

# American Economic Association

"This paper reports useful facts about trends in average firm and store
size and patterns of entry and exit. It makes use of the longitudinal
micro-data to classify firms by their geographic extent.

The authors ask for suggestions from the Committee for other aspects of
the data that would be useful to consider. The Committee offers the
following suggestions:

A. While average establishment size overall in the sector has increased,
it is worth noting that the traditional department store segment has had
very large establishments for a long period of time and that this sector
is in decline. This is perhaps an offsetting force to the general trend
up in average establishment size. It would be useful to report
information about what is happening to the share of employment in
establishments with more than 1,000 employees. We conjecture this share
might actually be declining. Also, it is worth noting that the decline
in large department stores is relatively more important when looking at
the co-worker mean (i.e., employee-weighted) than the unweighted mean
which is the main focus of this paper.

B. We encourage the researchers to exploit the geographic data at a
richer level of detail than the county. When more detailed geographic
information becomes available (e.g., longitude and latitude) it will be
possible to conduct the analysis at a richer level than the county. For
example, stores can be put into Census block groups. The block groups
can be categories by population density (e.g., number of people within a
one mile or two mile radius). This would make it possible to determine
the relationship between retail structure and population density.

C. In looking at the role of geography, the authors might want to take
into consideration that there are important differences across the
retailing sector in the role that distance plays. For convenience
stores, obviously the relevant market is very narrow geographic areas.
In contrast, for goods bought online like books, geography increasingly
is less relevant.

D. Franchising plays an important role in the retail sector. In future
work, the authors may consider incorporating this in some way.

E. An important development in retail is increasing vertical integration
where retail and wholesaling operations are conducted in the same firm.
In future work, the researchers might consider exploiting the micro data
to examine this issue. For ideas, the researcher can look at a recent
census project of Chad Syverson and Ali Hortascu, who are working with
manufacturing micro data. One approach would be to classify a store as
vertically integrated if there is a warehouse owned by the same firm
within some distance (e.g., 200 miles).

F. In examining entry and exit, the researchers might consider how these
relate to the change in population. Of course net entry will be
positively connected to population growth. But what about gross entry
and exit?

G. The panel noted that technological change has and will continue to
play a major role in accounting for the changes in the structure of
retail over the past century. It is well understood that the advent of
the car led to fundamental changes in the way we shop. There are also
strong arguments that changes in information technology have affected
the structure of retail. These changes in information technology include
recent advances with computers and bar codes but also include earlier
advances like the telephone."

######### 

######### Census Bureau Response

The Census Bureau is grateful to the Census Advisory Committee of
Professional Associations (CACPA) for their comments on our paper "The
Role of Retail Chains: National, Regional, and Industry Results." We
believe these are valuable suggestions and agree that there are many
additional aspects of the data to be analyzed. We will focus on those
related to geography first. Specifically, we plan to extend our work
using latitude and longitude information. We agree with the Committee's
recommendation (C) and believe that it is possible to develop
store-specific measures of a local market by looking at store
agglomeration and distance to stores of similar characteristics.
Following recommendations (B) and (F), we will include demand-side
controls, such as population density, as well as population growth. We
are also trying to locate a dataset on the development of the interstate
highway system as another potential improvement to our measure of
demand. We believe that a paper that seeks to develop refined measures
of local markets will be an important contribution to the empirical
literature in this area and a guide for future microdata projects at
Center for Economic Studies (CES) and the RDCs.

We also agree that franchising, vertical integration, and technology can
all be measured using data already collected by the Census Bureau and
possibly incorporated into future work. Also of interest to us is the
Committee's recommendation (A). We agree that we should pay additional
attention to describing co-worker means and the offsetting potential of
the largest establishments. We believe examination of these particular
statistics and populations will contribute to our understanding of the
trends we observe in retail markets. Following on this theme, we also
believe it would be interesting to examine the correlations between
entry and exit rates for stores of different types (single units vs.
chains) across particular markets.

# Recommendation 1c

The Evolution of National Retail Chains: How We Got Here

# American Economic Association

"Like the previous paper, this paper reports useful facts about trends
in average firm and store size and how turnover relates to store and
firm characteristics. A very interesting finding is that job destruction
rates are much lower in large chain stores (Mega stores by in the
paper's terminology). Given the growth of the large chain segment, this
has very interesting macroeconomic implications.

We offer the following suggestions for future work:

A. The researchers might consider classifying stores on the basis of
whether they are vertically integrated following the discussion above.

B. The panel found the discussion of the link between entry of Mega
stores and the entry and exit of small stores to be very interesting. We
encourage the researchers to consider down this line. There has been
work on this issue with the publicly available County Business Patterns
data. However, the publicly available version cannot be linked over
time. So only net flows are determined. And even new flows are not
measured that cleanly. So working on this with the Longitudinal Business
Database micro data would be an important advance. We do note there is
one limitation with the micro data in that it is then impossible to
focus on the effects of entry by a few large firms. (For example, with
the public data one can analyze the effects of Wal-Mart entry; this is
impossible with the micro data). We believe the gain of doing this kind
of work with the micro data to address this question well offsets this
cost.

C. The Committee noted that the retail industry is not in steady state.
It may be that down the line, the current patterns of job destruction
may not be sustained."

**[Census Bureau Response]{.underline}**

We greatly appreciate your comments and plan to incorporate many of them
into our project. Due in part to the committee's suggestions, we will
focus more intensively on the entry and exit of "Mega" stores and
Single-Unit (SU) stores. We will try to determine if Mega stores and SUs
are substitutes or complements, while accounting for industry
differences and the demographic characteristics by geography (either by
block groups or county). We will keep in mind the Committee's
recommendation that we consider vertical integration as an important
characteristic in determining differences in dynamics. We are also
intrigued by the suggestion that certain types of SUs may enter near
Mega stores, and will look closely at the characteristics of these
stores. Finally, like several committee members, we are interested in
what happens to "pure" churn in areas as Mega firm stores account for
bigger shares of the local market and plan to look closely at this part
of the story. We recognize that the retail industry is not necessarily
in steady-state, and, therefore, the patterns found in the paper may not
persist into the future.

# Recommendation 1d 

#### Supersize It: The Growth of Retail Chains and the Rise of the "Big Box" Retail Format {#supersize-it-the-growth-of-retail-chains-and-the-rise-of-the-big-box-retail-format .unnumbered}

# American Economic Association

"This paper shows that retailers with many stores tend to cover a broad
scope of activities and provides convincing evidence of a link. The
magnitude of the link is relatively small. The paper also provides a
nicely worked out theoretical model.

The paper makes extensive use of the product-line data. This data is
rarely used for research purposes and we are pleased to see researchers
using it. We encourage the researchers to continue down this line. Also,
the authors note that the work is preliminary. We thank them for giving
us the opportunity to provide comments at this early stage.

A. When aggregating the product information from the establishment level
up to the firm level, the researchers do the following: They ask whether
a particular product (e.g., eyeglasses) is carried by 25% of the stores,
50%, etc. An alternative procedure would be to take a simple average of
the store-level data. Suppose there are two product lines, widgets and
food. Suppose 49% of the stores carry only widgets and the remaining 51%
carry widgets and food. If the 50% threshold is used, then this firm has
one product line. But if now 51% of the stores carry food, the firm is
classified as having two product lines. So a small difference leads to a
major reclassification. If instead, we took the simple average,
initially the firm would have an average of 1.49 lines and this would
increase to 1.51 lines.

> B. It would be useful to make greater use of the product-line
> information beyond just counts. For ideas, the authors might want to
> look at related research. For example, in the international trade
> literature, researchers are examining the link between plant size and
> the number of countries a plant sells to. (In particular, work by
> Eaton, Kortum, and Kramarz.) There is an analogy between a count of
> number of products a store sells and the number of countries a
> manufacturer sells to. The work in the international trade literature
> examines whether there is a hierarchy of countries. Analogously, there
> might be a hierarchy of products. So all mass merchandisers sell
> hardware and clothing. Maybe only the bigger ones sell groceries, or
> tires.
>
> C. The theoretical analysis is interesting but it is short on
> empirical predictions. The Committee feels that it would be useful if
> there were a tighter link between the theory and the empirical work."

**[Census Bureau Response]{.underline}**

We appreciate the committee's comments. The first suggestion ― to use
the average number of lines at the firm's establishments ― will be
investigated. However, we are concerned that implementing the suggestion
may also prove problematic when we examine entry into (and exit) from
different lines, since two stores selling a different line (not food or
widgets) within a 100-store firm would contribute to the average measure
(but not the current measure) in the same way as two stores adding food.
To be consistent with the average measure, would this line have to be
measured as entry? We very much like the idea of looking at the related
international trade literature and appreciate the suggestion. We agree
that the theoretical model needs additional work and should be closely
aligned with the empirical section of the paper.

# 

# Recommendation 2 

#### Promoting Business Response to the 2007 Economic Census {#promoting-business-response-to-the-2007-economic-census .unnumbered}

# American Economic Association

"We highly commend Census for the approach it is taking to increase the
response rate to the 2007 Economic Census. We agree with the two main
approaches as outlined in the paper, but we recommend that the Census
emphasize the legal requirement in all direct communications, rather
than the private benefits from participating. The Census Account Manager
program, in particular, should help ensure that large businesses respond
in a timely and accurate way. In response to your specific questions, we
reemphasize the following points.

A. Encouraging timely response, particularly by large businesses:

> \- Earlier and more emphatic communication of the legal requirement to
> respond especially to large businesses. Do this through direct
> channels, not the news media. Perhaps a single-page letter to CEO/CFO
> in the next few months from the Office of General Counsel. This
> complements the advance notification package already distributed.
>
> \- Attempt to enlist a CEO- or CFO-oriented association in marketing,
> and address these forums as soon as possible. Specifically, attempt to
> address the Business Round Table of CEOs (perhaps through Rob Wescott)
> as to the issues (declining response rate) and importance. You might
> also consider enlisting a high-profile recently retired CEO as a
> communicator.
>
> \- While most businesses will respond to the legal request, they also
> will appreciate knowing that the effort they expend also will result
> in better information. Explain the benefits to businesses of full and
> accurate information.

B. Steps to raise intermediary interest:

\- Other associations that may give you a link to a large number of
companies include the National Federation of Independent Businesses
(NFIB - 600,000 small company members) and the Conference Board (larger
organizations). These two organizations cut across many industries and
may give you a different avenue than the typical trade association. NFIB
may give you a communication channel to many of your "problem children."

-   
-   

\- Contact the news media for story placement. Specifically, try to
reach the business news channels to get story placement (may be achieved
through some of the many economic commentators, analysts, etc.).

> \- Communicate with the National Association for Business Economics
> (NABE) for other past presidents with high visibility in
> local/regional areas (Alan Greenspan obviously is the most prominent,
> but perhaps Diane Swonk in Chicago, Dick Berner in New York, and
> others might be willing to assist).

C. Promote the Business Help Site:

\- Talk to Google and Yahoo about placing ads with potential
business-related queries.

D. Persuasive themes that Census may use in various communications,
along with the main message, which is that the Census is not selling a
service, it is trying to elicit a response to a legally required report
from business that also may benefit business:

-   Unlike most required government reporting, businesses only have to
    provide Census data once every five years.

> \- We underscore your finding that informing intermediaries about the
> broad programs supported by the Economic Census will assist in
> enhancing intermediary communication of the Census message.
>
> \- Census is investing as much as it can to make reporting as
> convenient as possible. Third party preparation offers a potential tax
> deduction as a business cost and a new line of business for the third
> parties. It may be worth informing the "problem children" and/or
> smaller businesses of this option, because they may not see any
> potential economic benefit. It may also be worthwhile to alert third
> parties to this opportunity to expand their offerings."

**[Census Bureau Response]{.underline}**

We thank the American Economic Association (AEA) for reviewing our
communication plans and affirmation of our methods.

We intend to continue emphasizing the mandatory requirements, in
addition to advising respondents of the private benefits arising from
their response. Our research indicates, and business behavior confirms,
that businesses are more likely to respond when they are aware of the
census-reporting requirement. We will include clear messages about the
reporting requirement in the questionnaire package, on our Internet
pages, and in our news releases and articles.

We are exploring development of a new letter to CEOs relating to the
Economic Census. As you know, we sent letters and copies of the Advance
Information to about 800 CFOs. As the census moves forward, we will be
able to assess the effectiveness of this approach. Corresponding with
CEOs may reinforce the other messages. We also will work with NABE and
other organizations to explore the role of past presidents as
spokespersons.

We recently launched the new Web site
\<[business.census.gov]{.underline}\>. We will use the Public
Information Office and other channels to let the business community know
about this new resource that will help demonstrate the importance of the
census to business decision making. We also are in discussions with
Google to integrate census messages and information into their
information and mapping resources. Our options are limited, however, as
there are no resources for paid advertising.

We have begun meeting with organizations that have a broad reach through
the business community, including the American Payroll Association, the
U.S. Chamber of Commerce and the NFIB, and plan to open discussions with
the Business Round Table and other groups. We also are preparing
tailored messages for associations geared to industries with lower
response, including a range of formats and media, such as journal
articles, e-newsletters, Web links, and Weblogs.

Recommendation 3

**Plans for Improving Services Data...Or What Would We Do for \$8
Million?**

**[American Economic Association]{.underline}**

"The Committee commends the Census for undertaking this important effort
to expand coverage of the burgeoning service sector.

A. The Committee recommends that the Census define the goals of each
stage of development of the services data. For example, if the goal is
to enhance current analysis of economic conditions then the share of
volatility at business cycle frequencies (say 2 to 5 years) may be more
important than share of GDP. This should guide the prioritization.
Relying on the BEA and other statistical agencies that use the data for
guidance is appropriate, but it is also useful to define the goals
implicit in these decisions. Also are there any other important users?
What are their needs? If there are no other users, why not?

B. The Committee also recommends that Census investigate means of
shortening the lag between end of quarter and release of QSS data.

C. The Committee agrees that product line data for the service sector is
very important if we are ever to understand the service sector as well
as we do the manufacturing sector. The Committee recommends that Census
define measures of success for product line definitions and data and
then base decisions on the basis of which product line definitions are
most likely to be successful.

D. The Committee suggests that Census consider collecting R&D
expenditures."

**[Census Bureau Response]{.underline}**

The Census Bureau will continue to work with our key data users to
determine their data needs and priorities, and understand how our key
data users use Census data. In addition, the Census Bureau is meeting
with trade associations and attending conferences to determine the needs
of our 'other' data users in addition to the Bureau of Economic
Analysis.

Census will investigate shortening the lag between the end of quarter
and release of Quarterly Services Survey (QSS) data. The investigation
will be completed by January 1, 2008, and will concentrate on the effect
that a shorter data collection and analysis time will have on the
quality of the estimates. It should be noted that it is common for data
of large contributors to be received on the last days of the collection
period.

Census will determine which products to include on the SAS forms by
analyzing the results of the 2007 Economic Census product data. For this
analysis, criteria to measure the success of the Economic Census product
data collection will be developed.

Since Research and Development (R&D) is already collected on the Survey
of Industrial Research and Development form and the addition of R&D
expenditures to the SAS form would increase respondent burden, the
Census Bureau will not pursue adding R&D expenditures to the SAS or QSS
forms. For more information on the Survey of Industrial Research and
Development go to \<<http://www.nsf.gov/statistics/industry/>\>.

[Recommendation 4]{.underline}

Effectiveness of the 2002 Survey of Business Owners Questionnaire and
Possible Additions to the 2007 Questionnaire

[American Economic Association]{.underline}

"The Committee commends the Census for its careful revision of the 2007
SBO Survey Form and for adding questions to elicit further information
about business owners and their activities. The Committee thinks these
efforts have led to improved survey. The additional information elicited
about business owners and business characteristics will increase the
value of the survey.

A. The Committee recommends considering additional questions to discover
business owners' pervious occupations or business activity as well as
questions to discover more about the businesses' customer base. A
related issue is determining the extent to which the individual's
"business" masks what is in reality an employee-employer relationship
that is set up as a business for tax reasons on some other regulatory
reasons.

> The Committee does not have a recommendation for eliminating any
> specific questions from the survey.

B. The Committee agrees that the new topics added to the survey will
elicit valuable information. The questions on governance and management
practices are interesting and potentially important and may be worth
further refinement and development going forward. We recommend that the
Census consider inquiring about the level of outsourcing activity and
operations outside the U.S. in addition to inquiring about changes in
those levels. The Committee does not understand the distinction between
a venture capitalist firm and a business angel in the survey."

**[Census Bureau Response]{.underline}**

Due to the lack of available space on the questionnaire and the lack of
time to seek approval from the Office of Management and Budget to test
additional questions, the Committee's suggestion to add additional
questions to the 2007 Survey of Business Owners (SBO) form to solicit
more information from the individual business owners will have to be
deferred until the 2012 SBO. Also, the Committee's suggestion to
consider inquiring about the level of outsourcing activity and
operations outside the United States, as well as changes in those
levels, will also be considered for the 2012 SBO.

The 2007 SBO form is currently designed to collect information from the
top four principal owners of a business using a single-page per owner
format. The third and final round of cognitive testing of the
questionnaire was completed on June 8, 2007. During testing, companies
responded positively to the overall design of the form and found the
one-page per owner format simple to follow. No specific item(s) were
found to be problematic to warrant elimination from the questionnaire.
Only one significant change was made to the form that was given to the
committee for review and commentary on April 19. Item 76 (techniques
used to attract, develop, and/or retain workers) was replaced with a
question on employee benefits paid totally or partly by the business
such as: health insurance; contributions to retirement plans, including
401(k), Keogh, etc.; profit sharing and/or stock options; paid holidays,
vacation, and /or sick leave; or none of the above.

The Committee's comment on the lack of distinction between the venture
capitalist firm and a business angel was addressed in the third and
final round of testing. The categories listed under sources of capital
for start-up or acquisition and for expansion or capital improvements
were combined into one category (investment by venture capitalists).
Feedback from respondents to this change was favorable.

**[Recommendation 5]{.underline}**

### Multi-Year Estimates from the American Community Survey

**[American Marketing Association]{.underline}**

"The AMAAC recommends that single estimates be provided by default for
casual users when multi-year estimates are available from the ACS.
Selection of a single default estimate by the Census Bureau will help
minimize the confusion, conflict and potential misuse of multi-year
estimates. Researchers will, of course, demand multi-year estimates and
all users should be able to access multi-year estimates.

The AMAAC recognizes the difficulty of selecting a single default value.
For some applications, a 1-year period estimate will be best, and for
others a 3-year or 5-year estimate will be more appropriate. Providing
decision criteria for selecting the most appropriate estimate should be
a long-term research goal of the Census Bureau. Until such criteria are
developed, the use of a naïve decision rule, such as selecting the
longest period estimate, has much to recommend it."

**[Census Bureau Response]{.underline}**

We agree that users will need further guidance on which estimate is
appropriate for a particular need. These issues are being investigated
in our evaluations of the Multi-Year Estimates Study. One of the main
objectives of these evaluations is to assess and answer questions about
usability of the multi-year estimates. The goal is to develop clear user
guidelines in the form of case studies on specific applications for
single versus multi-year estimates. In addition to evaluations planned
by Census Bureau staff, we are working with several external researchers
who are also considering these questions and issues from a data user
perspective.

**[American Statistical Association]{.underline}**

A. "The ASA/AC commends the Census Bureau for developing an improved
small-area estimation method by combining data over time. The estimation
method used is a direct one, but uses an implicit model. A natural next
step would be to put in some effort to check the implicit model involved
and to consider alternates."

B. "The ASA/AC recognizes that near-term tract estimates must
necessarily be based on rolling 5-year averages. In preparation for the
future, when more years of data are available, we recommend that the
Census Bureau initiate research into statistical methods that utilize
tract-level historical data to obtain a single-year estimate for each
tract. Such estimates would be more sensitive to discrete changes in
local conditions than 5-year averages and provide more flexibility in
analyzing ACS data over time for small areas or areas with boundaries
that are not coincident with county and state boundaries.

> One year, 3 year, and 5 year period estimates are important products
> of the ACS program. While the Census Bureau has done much to advance
> the discussion of these important estimates, much more remains to be
> done to insure the proper understanding of these estimates. It is
> difficult to know what information needs to be conveyed to users to
> ensure the informed use and interpretation of the ACS estimates, but
> active and timely steps need to be taken prior to the release of
> multi-year ACS data."

C. "The inclusion of the GQ data in multi-year ACS estimates has the
potential to cause substantial differences in yearly estimates at low
levels of geography. The ASA/AC would appreciate more information on the
design and estimation procedures for the GQ, and the implications for
stable multi-year estimates."

D. "The Census Bureau faces an enormous challenge in releasing huge
numbers of estimates at local levels every year. We recommend that the
Census Bureau simulate this process in advance to prepare for the types
of inquiries that will be generated by local communities that will use
these data critically [every year]{.underline} over time."

E. "The ASA/AC further recommends the Census Bureau develop a software
system for the automatic production of control charts (e.g., quality
control charts and charts of change) for cross-sectional statistics and
estimates of change. For the Quality Control Charts, software should be
developed for control charts of cross-sectional unit nonresponse rates,
coverage rates, and item nonresponse rates for published and, possibly,
unpublished geographic areas. The rates would be plotted versus area
identifiers. The charts would be used to identify areas that are out of
acceptable quality bounds, as determined by the Census Bureau. The use
of these charts would be to help identify data collection or processing
problems. For the Charts of Changes between time periods, software
should be developed to plot sizes of changes between time periods for
key analytic variables for published areas versus area identifiers.
These plots would be used to identify areas that have unusually large
positive or negative changes. Some of these changes will be perfectly
legitimate, but others may be due to data errors or other problems that
should be investigated."

F. "The ASA/AC also recommends that the Census Bureau establish an ACS
staff to develop and implement a program of training, education and
outreach to users and potential users of ACS data. The staff should
develop data user relationships with key state and local staff to help
determine how best to convey and interpret the ACS estimates and use
their advice to develop outreach programs. The estimates are complicated
and subject to misinterpretation by the large and diverse body of ACS
data users. Passive programs**,** allowing users to contact the Census
Bureau with comments and questions through the ACS web site and other
media, should also be instituted."

> **[Census Bureau Response]{.underline}**

A. The current implicit model used for combining data over time is for
each year to contribute approximately an equal amount to the period
estimate to the extent that the area's size has not changed
dramatically. One benefit of this model is that it produces lower
variances than, for example, models that assign unequal weights to the
data as a function of the collection year. For instance, a model that
weights the recent years more heavily than the older years in the period
would result in higher variances. A second benefit of the existing model
relates to the ease that such a model can be explained to data users.
Once the first set of 5-year period estimates are released and users get
experience with the data, we will consider experimenting with other
implicit models taking into account data user input.

B. The Census Bureau agrees that research into alternative methods for
producing small area estimates is desirable. While it is not a current
priority, we hope to pursue this type of research once we have
accumulated five years of data. We would be happy to discuss specific
alternatives further with the Committee at the appropriate time.

> We agree with the recommendation of the American Statistical
> Association Advisory Committee (ASA/AC) and are developing an active
> education and outreach program. The goal is to have this program in
> place prior to the release of the first set of multi-year estimates in
> the summer of 2008.

C. The Census Bureau welcomes the opportunity to share the documentation
of the research that went into the design and the development of the
estimation procedures used for the Group Quarters population. We will
provide documentation to the Committee under separate cover. The
documentation shows the series of alternatives that were investigated
and explains the rationale for selecting the current design both for its
implications relating to the single- and multi-year estimates.

D. The Census Bureau appreciates the enormous challenge in releasing a
large volume of estimates at the local levels every year. One way we are
preparing for this is the recent release of the Multi-Year Estimates
Study data. The objectives of the study were to test production methods
planned for the 2008 release, and to help data users learn more about
the characteristics of multi-year estimates. The Census Bureau released
data profiles for 14 data sets created using data from 1999 to 2005:
three 5-year estimates, five 3-year estimates, and six 1-year estimates.
Based on this release of study data, we are soliciting comments and
questions via the Web at \<<acso.web.staff.list@census.gov>\> and will
use the information when preparing for the 2008 release of the first
official multi-year estimates. In addition to comments and questions
received from the general public, we are working with several external
researchers who will provide feedback to us based on their experience
working with the data.

E. We agree with the Committee\'s recommendation on a software system
for the automatic production of control charts, and are considering the
development of a more formal set of methods to analyze data we collect
in real time about such quality measures as unit and item nonresponse.
Our goal is to develop a prototype system and to have a functional,
basic system in place in 2008.

# F. The Census Bureau concurs with the Committee\'s recommendation to establish an American Community Survey (ACS) staff to develop training and outreach, which corresponds to the Committee on National Statistics (CNSTAT) recent recommendation concerning the need for increased outreach to and education for a variety of data users and other interested parties. Currently the ACS Communication, Information, and Education Staff is developing plans to partner with organizations that will assist end users, including the State Data Centers (SDCs), Federal State Cooperative Program for Population Estimates, Census Information Centers, and many other organizations and groups. These plans supplement ongoing training, education, and outreach efforts currently underway.

# 

# Population Association of America 

A. "While the Census Bureau has done its best to make the transition
from the decennial census long form to the ACS smooth, the
implementation of the ACS represents a sea change for data users. User
reorientation will not occur as a result of issuing new documentation or
a new Web site. A serious educational effort is needed. The Census
Bureau must provide concise rules and a set of cogent best practices
that are well illustrated and include meaningful examples. An outline
for such a plan has been developed by the Committee on National
Statistics (CNSTAT) Panel on the Functionality and Usability of the ACS.

> The Population Association of America Advisory Committee (PAA/AC)
> recommends that the education and outreach approach taken by the
> Census Bureau should be modeled on Chapter 7 of the CNSTAT Report
> "Using the American Community Survey: Benefits and Challenges (2007)."

B. "The Census Bureau needs to develop an educational effort in close
cooperation with key members of the data user community --
intermediaries who are responsible for assisting data users with the
ACS. While the staff of Census Bureau Regional Offices would be involved
in any Census Bureau educational effort, the Bureau must reach out, far
and wide, to a number of organizations, including: the State Data
Centers; Local and Regional Councils of Governments; secondary and
post-secondary educational institutions; the Federal State Cooperative
Program for Population Estimates; Census Information Centers; and a
whole host of professional associations and user groups, such as the
Committee on Population Statistics of the PAA. After introducing the new
concepts employed in the ACS, the goal would be to educate data users
about the new perspectives they must apply to make effective use of
these data. The goal should be to help data users grasp those key
elements that make ACS different. The most important of these include
the change from point-in-time to period estimates of characteristics,
the increase in the size of standard errors for many estimates, and the
opportunities and challenges that will arise with annual data.

> The PAA/AC recommends that, in order to encourage the responsible use
> of data, the Census Bureau needs to establish a Census Bureau ACS
> users' staff devoted to education and outreach that would cultivate a
> network of trained intermediaries who would assist with providing a
> basic ACS education to users."

C. "While the Census Bureau's Web site is full of information on all
aspects of the ACS, this vehicle alone is not sufficient to educate data
users about ACS. The Census Bureau needs to implement a plan with the
objective of building a foundation of knowledge that is meaningful for
those who use and apply these data. Covering the basics of ACS requires
the development of application-oriented documentation and user-friendly
metadata, illustrated with sample applications that can be presented on
paper and/or on the Web in an online tutorial. Historically, the Census
Bureau has provided technical knowledge, but this approach emphasizes
data appropriate to user applications."

D. "The PAA/AC recommends that the Census Bureau consult with major user
groups to create concrete examples for large cities, smaller
governments, rural places, transportation interest groups, research and
educational institutions, and other groups that serve the data user
community. The examples should demonstrate how to use ACS, explain its
strengths and weaknesses, and be written in a user-friendly format."

**[Census Bureau Response]{.underline}**

A.B. The Census Bureau concurs with the Committee\'s recommendation,
which corresponds to

> the CNSTAT\'s recent recommendation concerning the need for increased
> outreach to and
>
> education for a variety of data users and other interested parties.
> Currently the ACS
>
> Communication, Information, and Education Staff is developing plans to
> partner with
>
> organizations that will assist end users, including the SDCs, Federal
> State Cooperative
>
> Program for Population Estimates, Census Information Centers, and many
> other
>
> organizations and groups. These plans supplement ongoing training,
> education and outreach efforts currently underway.

C. The Census Bureau acknowledges that the current information on the
Web site alone is

> not sufficient in educating data users about the ACS. We are
> researching ways to develop new training materials for use by the
> Census Bureau, SDCs, and others on how to understand and work with the
> ACS data. These training materials will be applicable to all users of
> the ACS ---- new to advanced. This research is consistent with the
> recent CNSTAT recommendation concerning the need for increased
> outreach to and education for a variety of data users and other
> interested parties.

D. The Census Bureau agrees with the PAA/AC recommendation that we
should consult

> with major user groups to create concrete examples on how to use the
> ACS.

**[Recommendation 6]{.underline}**

#### Recommendation regarding the National Academy of Sciences Panel {#recommendation-regarding-the-national-academy-of-sciences-panel .unnumbered}

**[American Marketing Association]{.underline}**

"The AMAAC learned in the final session late Friday April 20 that Census
was planning to have representatives from the National Academy of
Sciences (NAS) help evaluate census programs and outcomes. We have since
learned that Census has asked NAS to answer the following question: How
effective was the communication strategy for improving response and
accuracy of the Census including a.) the degree to which the separate
components of the communications (e.g., advertising, partnerships, etc.)
contribute to any improvements; b.) how effective are the targeted
messages in reaching specific audiences; and c.) did the communications
strategy increase awareness of or change attitudes or behaviors
regarding participation in the Census?\
\
AMAAC believes these are important questions to answer but we are unable
at this point to determine whether the NAS efforts will supplement or
substitute for previously planned evaluation efforts. While the AMAAC
members to CACPA have great respect for NAS, we would recommend that the
Census Bureau ensure that its marketing communication efforts be
reviewed by individuals and organizations with appropriate skills and
experience. The AMAAC stands ready to assist the Census Bureau in
providing the requisite skills to answer these vital questions."

**[Census Bureau Response]{.underline}**

The NAS panel was formed to provide recommendations for a program of
evaluation and experiments for the 2010 Census. One part of this
includes identifying the best strategies to evaluate the effectiveness
of the Integrated Communications Program (ICP). In addition to the NAS
panel recommendations, the Census Bureau is requiring the communications
contractor to subcontract with an independent research organization to
test the creative messaging. Funding permitting, the Census Bureau will
also sponsor an omnibus-type survey to validate the findings that
emerged during the October 2006 focus groups (barriers to Census
participation and motivational messaging). The Census Bureau is also
developing a model to segment the population using both macro-level
sociodemographic data and micro-level predispositional/attitudinal data.
An evaluation to assess the overall effectiveness of the ICP
(independent of the communications Contractor) is also planned and will
be designed with the recommendations of the American Marketing
Association (AMA) Advisory Committee and the NAS Panel in mind. The
Census Bureau greatly values the input and expertise that the AMA
Advisory Committee provides as we research and develop the 2010 ICP.

# Recommendation 7 

### Working with Trade Associations to Encourage Response to the Economic Census

# American Marketing Association

"The American Marketing Association Advisory Committee would like to
thank the Census Bureau for its thorough review of communication plans
for the 2007 Economic Census. The current plan to encourage
participation in the Economic Census is well designed and will likely
boost participation to some degree. Some of the planned programs that we
feel will be especially effective are the account representative program
for supporting larger corporations and providing detailed online help
sites.

Another, very important part of the plan is leveraging associations.
Associations not only provide access to member lists, but also use their
association publications and newsletters to represent an endorsement to
their members that will be believable and noticed.

While many associations may be willing to share their lists, others may
be reluctant. The AMAAC recommends creating a win-win situation for
associations to participate in encouraging their members to respond to
the 2007 Economic Census. Associations must understand "what's in it for
them."

The AMAAC recommends using HotReports via DataFerrett as a way to create
this win-win situation. These reports can provide associations targeted
and valuable information about their industries, constituencies, and
other business data that will help them accomplish their respective
missions in a more efficient way. By providing them access to such
reports, the Census Bureau will be giving them something useful in
return for their efforts. In addition, the committee recommends offering
webinars as a way to instruct new users on how to use HotReports and
communicate their value."

**[Census Bureau Response]{.underline}**

We thank the AMA for reviewing our communication plans and affirmation
of our methods.

We agree that partnership with trade and professional associations can
return benefits to agency and association alike. We will pursue the AMAs
recommendations, including membership lists, and also will provide
materials that associations can send through their regular
communications.

We have begun meetings with organizations that have a broad reach
through the business community, including the American Payroll
Association, the U.S. Chamber of Commerce, and the National Federation
of Independent Business. We are discussing a range of formats and media,
such as journal articles, e-newsletters, Web links, and Web logs, and
limited only by the individual associations' offerings. We also are
working with the Public Information Office to develop new formats such
as Webinars.

We agree that HotReports (HR) provide an opportunity to provide
information from Economic Census reports for many industries in an
understandable format. We recently have overcome technical issues that
precluded adoption of HR to disseminate industry snapshots on
\<[business.census.gov]{.underline}.\> We are working with appropriate
staff to design the reports.

**[Recommendation 8]{.underline}**

**Endorsement of HotReports and DataFerrett**

**[American Marketing Association]{.underline}**

"The AMAAC was excited to review the capabilities of the Community
HotReports and DataFerrett technology which, in the view of the
committee, create a greatly expanded utility for Census data and provide
the ability to combine Census data with data from other sources. The
AMAAC was particularly gratified to see the HotReports integrated topic
data from numerous reliable sources both inside and outside the Census
Bureau, rather than focus on results from one source. This approach
gives the user a better view of available knowledge on a topic and
enhances the understanding of business and social environments. While
there may be some concerns about the credibility of mixing Census data
with data from other sources, the value of this program outweighs those
concerns and a disclaimer can appropriately communicate a warning to
users. Given the flexibility and utility of these programs, the
committee recommends that their use be strongly promoted within the
Bureau and to its external data users."

**[Census Bureau Response]{.underline}**

The DataWeb and Applications staff of the Data Integration Division
(DID) and the Customer Liaisons Marketing Services Offices (CLMSO)
appreciate the AMA\'s enthusiastic endorsement of The DataWeb (DW),
DataFerrett (DF) and HR.

CLMSO has plans to continue its educational and marketing efforts on
behalf of the DF program, to strongly promote DF and HR to external data
users and within the Census Bureau.

Sourcing the data is paramount to DF HotReport clients. Through an
ongoing series of meetings and demonstrations, DW staff is working to
find the best way to label sources to ensure that the end user fully
understands the originating source for the data; data limitations are
based upon the collection agency's protocols, and also will provide
contact information for each data source on the DW site. CLMSO and DW
staff will work with data user communities and will test for a suitable
disclaimer statement.

Assuming further external investments and partnerships, the DW
applications staff would be able to integrate topical data from
additional sources to expand the understanding of business and social
environments.

**[Recommendation 9]{.underline}**

Getting Input From User Groups on Uses for Reports (ACS and HotReports)

# American Marketing Association

"The Census Bureau has made great strides in developing data products
that can meet the needs of a variety of data users. In order to be
increasingly responsive to the needs of its users and stakeholders, the
AMAAC strongly recommends:

A. "That the Census Bureau obtains continuing input from users
(especially users of American Community Survey reports and the Community
HotReports) so that these reports can be further enhanced and are
customer-friendly and responsive to user needs."

B. "Create panels of current users via an invitation on the Census
Bureau web site. The panels can be used to evaluate new report concepts,
volunteer ideas for new reports, and prioritize the submitted ideas.
Members of the panel could also function as beta testers in order to
broadly represent the spectrum of end-user communities. The AMAAC is
ready to support the Census Bureau as they evaluate other means of
gaining and prioritizing user input."

C. "Develop an on-line forum where all users could share their charts
and web pages. If the charts were maintained and updated as the base
data was refreshed, then there would be reasons for users to return to
the site. Also, potential new users would have an incentive to
periodically explore the site. Perhaps this approach removes the concern
about favoring one industry over another, because all are free to
participate. It would also promote the Census Bureau as the integrated
statistical repository for the nation. In addition, as the owner of the
site, the Bureau has the capability to promote proper use of the data as
well as learning how customers and users employ the data for various
business and social issues. We believe that the web forum approach will
help alleviate the firewall security problem mentioned above."

D. "Develop a communication package that clearly states what is
available to users, how to access it, and what if any fee structure
exists. This could be a marketing or promotional package that encourages
investigation and use."

E. "Partner with the Research Data Centers to develop and share creative
uses of the data. Five of the RDCs are located in university
environments which would be a good source of creative student talent.
The effort would include developing informative, interactive Web pages
that could be shared with other users. The pages could be
"off-the-shelf" products that less sophisticated users could access when
useful. Efforts like this would be part of the promotional
communications noted above."

######### Census Bureau Response

A. The DW and MSO value AMA's strong recommendation to obtain continuing
input from users. The DW welcomes further insights from the American
Marketing Association Advisory Committee to apply best practices to
collect, evaluate, and prioritize feedback from data users. MSO's
customer research program stands ready to assist in this effort as
appropriate. The AMA recommends continuing input from users to enhance
and make reports user friendly to respond to user needs, including
panels and online forums. The DW, through the Economic Development and
Displacement team, convened a one-and-a-half-day meeting of data users
and community experts to determine the content of the Community Economic
Development HotReport.

B. The DW currently has a couple of online forums, one for beta testing
of HR \<[http://brac.thedataweb.org/forum]{.underline}\> and a second
for users of DF \<[http://www.thedataweb.org/forum]{.underline}\>. The
latter responds to questions about the data, the DF software shows
examples of DF tables and how they were constructed, and also shows
graphic output developed in DF. To date, examples of HR have not been
posted to the forum. The DW will continue to build the online forum and,
as part of a strategic communications plan developed by CLMSO, will work
to attract users to the online site, and also find ways to get them to
return to the site. The DF software has built-in statistical
intelligence to help users avoid improper use of the data and the online
forum works to instill statistical literacy.

C. Through the auspices of the Local Employment and Household Dynamics
Program (LEHD) and the Economic Development and Dislocation (EDD) team,
the DW has been working with MSO for about two years to build innovative
products and forge partnerships within the Census Bureau to foster
integration of disparate data sets. Marketing activities to date have
centered largely on presentations, demonstrations, flyers, and
brochures. Most recently MSO developed a short \"infomercial\" DVD about
DF and HotReports for use in exhibit booths and at training events.
Based on AMA's constructive feedback, MSO is at work aligning future
promotional messages to push the users/uses behind DF and HotReports,
rather than focusing on branding the product name. Ideas include \"The
Power Tool for Data Users\" or \"Empowered by Data." Marketing
opportunities may be explored with the media and media user groups like
Investigative Reporters and Editors (IRE) and National Institute for
Computer-Assisted Reporting (NICAR**).**

D. The DW will work towards developing a pricing model to address
potential data access opportunities. Like other divisions working on
highly customized projects or products, DID will price new work on a
cost recovery basis. Marketing opportunities may be explored with the
media and media user groups like IRE and NICAR. CLMSO stands ready to
participate in this business and pricing model development team.

E. DW and CLMSO will investigate options involving the RDC Program, as
suggested. However, since projects involving tabulations are explicitly
discouraged from being undertaken at RDCs, and also since security
arrangements prevent RDC users from accessing the Internet, it is
difficult to see how such a collaboration would work. DW and CLMSO will
also explore how the Census Bureau's State Data Centers might provide
data analysts and expertise to the development of HotReports.

# Recommendation 10 

# General Recommendation

# 

# American Statistical Association

"The ASA/AC appreciates the update on the review of the Census Bureau's
research program. The ASA/AC urges the Census Bureau to address the
organizational issue for the Statistical Research Division in the Census
Bureau, raised in the report, as expeditiously as possible. We believe
that it is particularly important that research and standards (e.g., the
Associate Director for Methodology and Standards) have a seat at higher
level administrative meetings. We would appreciate an update on this at
the next meeting, including a follow-up on how the ad hoc task force
recommendations to the Statistical Research Division are being addressed
by the Census Bureau."

######### Census Bureau Response

We appreciate the ASA/AC's continued interest in the Census Bureau's
Program of General Methods Research. We value greatly the past, present,
and future contributions of the Statistical Research Division (SRD) to
Census Bureau programs and are looking at alternatives to address the
organizational issue for SRD in the Census Bureau. We will keep the
ASA/AC informed of developments in this regard.

#  {#section-5 .unnumbered}

**[Recommendation]{.underline}** 11

#### Variance Reduction for Sub-County Estimates in the American Community Survey {#variance-reduction-for-sub-county-estimates-in-the-american-community-survey .unnumbered}

# American Statistical Association

"The ASA/AC was very pleased to hear about the research efforts and that
the initial results indicate the potential for reducing the variances of
ACS estimates using administrative records. We believe the focus and
direction of the research is very appropriate. The research is
responsive to many of the issues that our group has raised over the last
few years regarding ACS estimation. We strongly support this research
and recommend that, if the promising results are confirmed in activities
scheduled for this year, an operational plan be developed to incorporate
this method of estimation in time for the planned initial rollout of the
multi-year estimates."

**[Census Bureau Response]{.underline}**

We welcome the interest and review of the ASA/AC on this matter. We will
continue to report the research findings, particularly from the
Multi-Year Estimates Study, in conference papers and internal reports.
We are also beginning to study the operational implications of
implementing the research in the initial rollout of the multi-year
estimates next year.

# Population Association of America 

A. "The Census Bureau should continue research on variance reduction in
ACS estimates, including the use of administrative records, and, in
particular, the ACS MAF extract. However, the PAA/AC recommends that
strict controls be exercised with regard to maintaining the
confidentiality of all the records and that a sharp eye is kept on the
"privacy" horizon."

B. "Although model-assisted estimators are design consistent and their
biases have been found to be generally of lower order than their
sampling variances (Fay, 2006), it may be worthwhile to keep the issue
of bias in mind. As such, the PAA/AC recommends that the Census Bureau
periodically examine the issue of bias as work on variance reduction
progresses."

#### 

#### C. "The PAA/AC recommends that the Census Bureau keep calibration as a secondary goal as research continues on ACS variance reduction." 

D. "In regard to continuing research on ACS variance reduction, the
PAA/AC recommends using principles that are consistent with ones that
the Census Bureau has already developed (the ones for its estimates
program, for example) and applied to the issue of ACS weights. As a
starting point the PAA/AC recommends the following criteria:

1.  Do the methods under consideration meet user needs?

2.  Do the methods have face validity?

3.  What is their ease of application?

4.  What is their ease of explanation? "

E. "One source of increased variance in the ACS stems from the different
sub-sampling rates for CAPI interviews. These different sub-sampling
rates are largely determined by cost factors and the need to retain
adequate sample sizes for census tracts with lower than expected mail
and CAPI responses. The PAA/AC recommends that ways be examined to
specifically reduce the overall variance associated with these
interviews. Model assisted estimation might be one possibility."

> F. "The PAA/AC recommends that work on variance reduction at the tract
> level be exhaustive before work begins at the block group level.
> Having a solid body of experience at the tract level is important
> before proceeding to the block group level."

####  G. "Given the divergence between the ACS MAF extract and the housing unit estimates by the Census Bureau at the national and county level, the PAA/AC recommends that the Census Bureau consider the implications not only of unlinked records but also the growing divergence as the Census Bureau proceeds with research on ACS variance reduction." 

**[Census Bureau Response]{.underline}**

A. We will continue to consider the sensitivity of administrative
records for this and other Census Bureau applications. One question we
brought to the committee was whether to step beyond the current
research, which uses only demographic characteristics estimated from the
administrative records, to the potentially more sensitive use of income
data. The discussion by the committee at the meeting has helped to
dissuade us from planning any research on use of income data from
administrative records for an initial implementation. We note that
income data is not present on the files extracted for use in our initial
research.

B. In our analysis of the Multi-Year Estimates Study, we plan to study
the issue of bias by searching for any systematic effect on the
estimates from the model-assisted estimator compared to estimates
without it, a comparison we can make for the 2001-2005 and 2003-2005
multi-year estimates.

C. We will continue our research with this perspective in mind.

D. We will consider these questions in our review of the Multi-Year
Estimates Study. We believe that the first and third questions can
already be answered in the affirmative. The model-assisted estimation
will simply be one of the several steps in the complex ACS estimation
process, but the effect will be essentially transparent to the user
because the same data products will be produced. With respect to the
third question, the calculations for the model-assisted approach will
require internal resources, but users will not be required to perform
any calculations or to use the ACS products in any new way.

> We will continue to consider the second and particularly the fourth
> questions in our analysis of the Multi-Year Estimates Study. In
> addition to our own staff, we are working closely with a set of
> external researchers who are evaluating the estimates. We will seek
> their advice on an appropriate level of explanation for the methods.

E. As the committee notes, weight variation contributes to the overall
sampling variance. The model-assisted estimation will help to reduce
this and other sources of variance. However, the issue of variance
reduction specifically for the subsampling is not separable from other
aspects of the variance reduction. Because the subsampling factors are
known only for the realized sample but not for the rest of the frame, it
does not appear to be feasible to extend model-assisted estimation to
adjust for subsampling directly.

> The ACS program recognizes the benefits of increasing funding in order
> to increase sample size. A sample size increase can be realized by
> increasing the initial mail-out sample or the nonresponse follow-up.
> The determination of the optimum trade-off is in our sampling research
> agenda.

F. We note the Committee's advice on research strategy. In the
Multi-Year Estimates Study, we are able to observe the average variance
reductions for different geographic types, particularly for the
2001-2005 period estimates. The analysis may motivate alternatives to
the block group in some areas, such as counties with small places or
townships that are of potentially greater interest than block groups.
These questions will arise for the 5-year estimates, but the proposal
for the 3-year estimates, as implemented in the Multi-Year Estimates
Study, uses places or MCD/place pieces instead of tracts.

G. The Census Bureau agrees with the Committee\'s recommendation. The
growing divergence between the ACS MAF extract and the housing unit
estimates should be monitored and its impact considered in ACS variance
reduction research. The Census Bureau will be undertaking major
operations to provide the MAF with updates and address corrections in
time for the 2010 Census. The ACS will use the updated MAF for the 2010
sample selection. Following the 2010 Census, both the ACS MAF and the
housing unit estimates should be in better agreement. The Census Bureau
will also be considering methods to continue updating the MAF after
2010, which may help control divergence between the two sources in the
future.

**[Recommendation 12]{.underline}**

**Reengineered SIPP**

**[American Statistical Association]{.underline}**

"The ASA/AC welcomes the prompt introduction of the DEWS survey and
commends the Census Bureau on reducing the gap between the end of Survey
of Income and Program Participation (SIPP) and the beginning of the new
survey. The Census Bureau should be complimented for proposing a complex
modern system that relies on survey and administrative data. The Census
Bureau has initiated this important redesign project very quickly and,
given the current schedule, there is not a substantial amount of time
prior to implementation. Once implemented, the DEWS system will be an
important part of the federal statistical system, providing important
information on the dynamics of income and program participation.

Nevertheless, the ASA/AC has concerns about quality issues for the new
survey and the difficulties of comparing data from SIPP and DEWS."

A. "The ASA/AC recommends that the Census Bureau conduct a study that
evaluates differences in estimates obtained from SIPP methods (3
interviews per year) and proposed DEWS methods (event history calendar).
An important goal of the study would be to provide quantitative
information to users on the expected difference in estimates associated
with the change in measurement methods. The committee recognizes that it
may not be possible to pursue this within a production environment, and
thus understands that the study may need to be conducted outside of the
production environment.

> As the sample design is developed, we recommend that DEWS build in the
> capacity for states and other groups to request and fund data
> collection on additional sample units in the DEWS."

B. "The ASA/AC recommends that the Census Bureau create a technical
working group to provide external consultation on the statistical and
survey research and evaluation aspects of the DEWS program. The working
group should meet more frequently than the advisory committees and focus
specifically on the technical and evaluations aspects of the DEWS
program."

C. "The ASA/AC recommends that the Census Bureau establish within the
DEWS budget significant funds to continue an ongoing program of survey
and statistical research to evaluate the DEWS data and to develop
methods to improve the quality of the data. Because of the fast startup
and the complex nature of the system, it will be very important for the
long-term development of the system to have an ongoing program of
research and evaluation."

D. "The Census Bureau's outreach program to the SIPP data users to
identify data elements for the DEWS is commendable. At the conclusion of
the outreach program, we recommend that the Census Bureau prepare a
document for the DEWS and SIPP users informing them of the SIPP data
topics that will remain in the DEWS and the topics that will be
deleted."

E. "The ASA/AC recommends that the Census Advisory Committees be
informed regularly of the design and implementation issues the Census
Bureau faces with the initiation of the DEWS system. We are particularly
interested in being kept up-to-date on the progress toward meeting the
substantive, technical, and operational goals of the DEWS system."

> **[Census Bureau Response]{.underline}**

A. Note: Since these meetings, many developments with respect to
reengineered SIPP have taken place. At this time, we don't know exactly
what form the reengineered SIPP will take. We do know that a form of
SIPP will be maintained without a data gap and may, at some future point
in time, be enhanced with improvements, such as an event history
calendar, and processing mechanisms that will make the SIPP a state of
the art survey.

> For the past two decades, the SIPP has been the leading source of data
> on the economic well-being of Americans. The Census Bureau remains
> committed to providing high-quality and relevant data in a timely
> manner for this purpose. The Census Bureau also is committed to fully
> addressing the concerns of the user community, though perhaps at a
> slower testing and implementation pace than originally planned. The
> exact size of the reengineered SIPP, and what upgrades might be
> tested, will be dictated by the funding levels provided by the
> Congress. Until we know those funding levels, we are unable to fill in
> the details.
>
> The Census Bureau and the DOC are actively providing Congress with
> budget figures for a number of different SIPP scenarios. This includes
> what it would cost to continue SIPP at the full 45,000 sample level
> beginning in February 2008, and continued development of improvements.
>
> A major issue in the planning for the reengineered SIPP is the
> validity of use of the Event History Calendar (EHC) and a twelve-month
> recall period for the collection of monthly federal program data. The
> use of an EHC to capture sub-annual transitions is a relatively new
> feature in automated demographic survey instruments. The Panel Study
> of Income Dynamics (PSID) conducted by the University of Michigan, as
> well as the labor surveys of Statistics Canada and Statistics
> Netherlands, are currently using or are planning to use this
> methodology, which ties life events to program participation, health
> insurance, and job transitions, and enables sub-annual estimates to be
> obtained in a single annual interview rather than three interviews at
> four-month intervals as is the current practice with SIPP. Current
> plans include a test of a paper questionnaire using EHC in early 2008.
> The sites of this test would be Illinois and Maryland where
> administrative data for certain programs may be available to the
> Census Bureau. The test would include administering the EHC paper
> questionnaire to respondents of the 2004 SIPP Panel who were in the
> three waves of FY 2007 and to respondents who were part of the sample
> cut for the 2004 SIPP Panel between September and October 2006. The
> results of the paper interviews would be compared to the results of
> the SIPP interviews, as well as the administrative records in order to
> begin an assessment of the impact of 12-month recall with a calendar
> versus a four-month recall with the SIPP instrument. Further, to
> assist in the development of an electronic EHC, evaluation of data
> from an electronic EHC, and comparisons of recall error between these
> results and that of the four-month recall used in the current SIPP, we
> have contracted with the University of Michigan's PSID staff. As a
> result, the University of Michigan will commission a series of
> analytic papers and conduct a two-day workshop on testing EHC
> instruments, evaluating data gathered in an EHC instrument, and
> training of field staff for an EHC instrument, etc., in December 2007.
>
> With some possibility of funding for FY 2008 to allow a 2008 SIPP
> Panel to run in parallel with testing and evaluation of the EHC
> methodology, data quality and comparability concerns could be greatly
> allayed. The ongoing research into better ways to elicit correct
> response, as well as reduction of respondent burden in surveys (such
> as EHC), needs to continue as SIPP continues.
>
> The current plans for content of the instrument also include questions
> that would identify subpopulations of interest to agencies and groups
> for future reimbursable topical surveys. Several requests for
> estimates of the costs associated with such topical surveys have been
> answered to date.

B. The Census Bureau is beginning the process of reconstituting the
membership of the American Statistical Survey/Research Methodology
Advisory Group for SIPP. We are currently deciding on appropriate
membership and agendas in order for the group to advise us on the
technical and evaluations aspects of the reengineered SIPP.

C. We plan to conduct ongoing research into the data quality of
estimates derived from the reengineered SIPP based on full funding for
both SIPP and the reengineered SIPP. We must also, however, plan
contingencies for the case where various levels of sufficient funding
are not available.

D.E. Our plan is to prepare such a document, and this project has been
initiated. We plan to prepare an informational document for users and
regularly update the Advisory Committees.

**[Population Association of America]{.underline}**

"The PAA/AC commends the Census Bureau for its innovative plans to
create a leaner, cleaner version of the Survey of Income and Program
Participation (SIPP). The report presented by David Johnson at the April
2007 CACPA meeting outlines plans for a new system of longitudinal
surveys, DEWS, that promises to include much of the vital data from the
SIPP while also cutting both administrative costs and processing times.
The committee also commends the Census Bureau for consulting carefully
with other government agencies and researchers that rely on SIPP, to
ensure that essential data items are retained in the transition to DEWS.
The Census Bureau's advance planning for ease of use in DEWS data
products is also welcome news.

The CACPA report mentions several differences between SIPP and DEWS that
may affect academic and other research users. The PAA/AC requests
clarification about exactly what these changes are, and about their
expected costs and benefits:

A. "**Periodicity.** The planned change from 3 interviews per year with
4-month retrospective questions in the SIPP, to 1 interview per year
with 12-month retrospective questions in DEWS, will certainly affect the
quantity of data collected and may affect its quality.

> What items will be cut, and which remaining items are most likely to
> be affected by recall errors or nonresponse?"

B. "**Topics**. The CACPA report contains little specific description of
which SIPP items may be cut in the transition to DEWS. With one-third as
many interviews, some important topical modules will presumably be
shortened or eliminated. Which? What is the plan?"

C. "**Sampling Frame**. DEWS will use ACS interviewed cases as its
sampling frame. This change is probably a big cost-saving measure, but
it is unclear how it may affect data quality. The change of frame raises
many technical questions about the MAF, and about reweighting to account
for differences between the ACS sample and the ACS interviewed sample.
The PAA/AC requests a brief explanation of the costs and benefits of
this fundamental change."

D. "**Administrative Records.** The PAA/AC commends the Census Bureau
for its ongoing efforts to use administrative records to streamline,
check, and enhance its data products. It appears that the DEWS project
will use administrative data in two very different ways: (1) as a
quality control device, comparing DEWS responses to other sources of
information on DEWS respondents, and (2) as a means of merging
additional individual-level data (e.g., IRS tax records) onto DEWS data
sets.

> The quality control part of this approach appears to work well with
> Federal administrative data, but the problem is at least 51 times
> bigger when attempting to use state-level program data. Given the
> problems of coordination, timing, and harmonization of data items, is
> state-level matching really a reasonable goal?"

**[Census Bureau Response]{.underline}**

A. Note: Since these meetings many developments with respect to
reengineered SIPP have taken place. At this time we do not know exactly
what form the reengineered SIPP will take. We do know that a form of
SIPP will be maintained without a data gap and may, at some future
point, be enhanced with improvements, such as an event history calendar,
and processing mechanisms that will make the SIPP a state of the art
survey.

> For the past two decades, the SIPP has been the leading source of data
> on the economic well-being of Americans. The Census Bureau remains
> committed to providing high-quality and relevant data in a timely
> manner for this purpose. The Census Bureau also is committed to fully
> addressing the concerns of the user community, though perhaps at a
> slower testing and implementation pace than originally planned. The
> exact size of the reengineered SIPP, and what upgrades might be
> tested, will be dictated by the funding levels provided by the
> Congress. Until we know those funding levels, we are unable to fill in
> the details.
>
> The Census Bureau and the DOC are actively providing Congress with
> budget figures for a number of different SIPP scenarios. This includes
> what it would cost to continue SIPP at the full 45,000 sample level
> beginning in February 2008, and continued development of improvements.
>
> A major issue in the planning for the reengineered SIPP is the
> validity of use of the EHC and a 12-month recall period for the
> collection of monthly federal program data. The use of an EHC to
> capture sub-annual transitions is a relatively new feature in
> automated demographic survey instruments. The PSID conducted by the
> University of Michigan, as well as the labor surveys of Statistics
> Canada and Statistics Netherlands, are currently using or are planning
> to use this methodology, which ties life events to program
> participation, health insurance, and job transitions, and enables
> sub-annual estimates to be obtained in a single annual interview
> rather than three interviews at four month intervals as is the current
> practice with SIPP. Current plans include a test of a paper
> questionnaire using EHC in early 2008. The sites of this test would be
> Illinois and Maryland, where administrative data for certain programs
> may be available to the Census Bureau. The test would include
> administering the EHC paper questionnaire to respondents of the 2004
> SIPP Panel who were in the three waves of FY 2007 and to respondents
> who were part of the sample cut for the 2004 Panel between September
> and October 2006. The results of the paper interviews would be
> compared to the results of the SIPP interviews as well as the
> administrative records in order to begin an assessment of the impact
> of 12-month recall with a calendar versus four-month recall with the
> SIPP instrument. Further, to assist in the development of an
> electronic EHC, evaluation of data from an electronic EHC, and
> comparisons of recall error between these results and that of the
> four-month recall used in the current SIPP, we have contracted with
> the University of Michigan's PSID staff. As a result, the University
> of Michigan will commission a series of analytic papers and conduct a
> two-day workshop on testing EHC instruments, evaluating data gathered
> in an EHC instrument, and training of field staff for an EHC
> instrument, etc., in December 2007.
>
> With some possibility of funding for FY 2008 to allow a 2008 SIPP
> Panel to run in parallel with testing and evaluation of the EHC
> methodology, data quality and comparability concerns could be greatly
> allayed. The ongoing research into better ways to elicit correct
> response as well as reduction of respondent burden in surveys (such as
> an EHC) needs to continue as SIPP continues. The response on items
> retained and cut is addressed in the next section.

B. A proposed main component of the reengineered SIPP system will be an
annual survey collecting data for the previous calendar year (using an
EHC), with content similar to that of the current SIPP core survey, and
following households for a three-year period. While content will be more
limited than in the current SIPP since there are no topical modules
associated with this main component of the system, additional data could
be obtained through reimbursable surveys, separate from the base
product. The goals of both SIPP core and the reengineered SIPP are the
same, that is, to provide a nationally representative sample that can be
used to evaluate the annual and sub-annual dynamics of income, the
movements in and out of government transfer programs, and the
interactions between these items. The current plans for reengineered
SIPP envision an annual interview with an EHC method of interviewing in
order to obtain information that has been historically used for
sub-annual estimates (e.g., means tested social welfare programs). Other
sections of the new basic instrument, previously asked in core or
planned as "hooks," will be asked in a typical interview mode but only
annually (e.g., assets) when it is deemed that sub-annual estimates are
either not critical to program evaluation or not as meaningful on a
monthly basis as on an annual basis. Since field costs have historically
been about one half of the annual program costs, we anticipate a cost
savings of about one-third of previous program costs once the system has
been developed and is operational. Response rates are also expected to
improve since households will be interviewed annually rather that 3
times a year. In order to determine the content of this base survey,
there have been numerous meetings between Census staff and major
stakeholders. Large groups of stakeholders and Census have met for
multiple public presentations of the developing plans for the survey.
Matrices of all content requirements have been submitted by various
major stakeholders to the SIPP Web site
\<[http://www.sipp.census.gov/sipp/dews.html]{.underline}\>. Five
content sessions on specific topics have also been conducted at Census
with stakeholders during the spring of 2007. We continue to tabulate the
results of content matrices received and the results of these topical
meetings with the goal of finalizing instrument content by the beginning
of summer 2007. Once finalized, we anticipate that the core content of
the new survey will remain constant.

> The current plans for content of the instrument also include questions
> that would identify subpopulations of interest to agencies and groups
> for future reimbursable topical surveys. We have had several requests
> for cost and resource estimates for this type of topical survey and
> the estimates for authoring an instrument, fielding the instrument,
> and processing the data from each of these interviews are being
> developed.

C.  Given that the low-income population is of special interest to
    reengineered SIPP, we plan to over-sample for low income. Selecting
    sample from the ACS interviewed cases provides the opportunity to
    use more recent income information in over-sampling, thereby
    improving the efficiencies of that procedure. We expect that using
    the more recent income data, we can increase the percentage of cases
    that are low income from

> 22 percent to about 25 percent (Out of 45,000 cases - 9,900 and 11,500
> are expected to be low income using current methods).
>
> Further advantages to sampling from the ACS cases include having more
> data on both respondents and nonrespondents to the reengineered SIPP
> that can be used in a noninterview adjustment, possibly improving the
> weighting adjustment, thereby reducing nonresponse bias.

The drawbacks to sampling from the ACS include:

A larger survey error due to ACS nonresponse and Master Address File
(MAF) undercoverage of rural addresses (ACS final weights may mitigate
some of this) and;

An insufficient sample in small PSUs (although this can be dealt with by
taking less sample, not oversampling in those areas, or allowing
multiple hits to the same address and increasing their baseweight).

After an analysis of the main criteria for deciding on a sample frame
(reliability of low income estimates, minimum number of Field
Representatives (FRs) required, and cost efficiency for personal visits)
it was decided to:

Sample from ACS in order to improve the reliability of estimates of the
low income - we're able to get a sample with a higher percent of the low
income with the same total sample size;

Cluster the sample into reasonable FR workloads to minimize the number
of FRs needed, thereby, minimizing training costs; and

Keep the same PSU definitions, as they provide the most cost effective
data collection when personal visit interviews are needed. We have no
evidence to suggest that we'll gain a major benefit from increasing the
size of the PSUs, as travel costs will increase, thereby further
limiting the number of cases to be interviewed.

D. The Census Bureau conducted an in depth evaluation of the
availability and timeliness of administrative data as well as
comparability to SIPP definitions and concepts. After completing this
initial evaluation, as well as consultations with many major
stakeholders, the Bureau has decided to initially use administrative
records to evaluate quality of reported data and improve imputation
methodologies for missing survey data. The evaluation of the further use
of administrative data is continuing with the development of prototypes
of SIPP 2004 core with administrative data appended. These files will be
analyzed for disclosure and data quality purposes for future use in the
reengineered SIPP program. This plan has met with universal consent by
federal stakeholders, policy groups, and other data users.

> The Census Bureau is also working with the National Academy of
> Sciences to have the CNSTAT convene a panel of experts to provide an
> evaluation that addresses the accessibility of relevant administrative
> records, the operational feasibility of linking administrative records
> and survey data, the quality and usefulness of the linked data, and
> strategies for providing public access to the linked data while
> protecting the confidentiality of individual respondents as it relates
> to the reengineered SIPP.
>
> The second part of the approach, extending reengineered SIPP data via
> record matching, will rely on future technical fixes to
> confidentiality problems. One such approach is "synthetic data" that
> preserves some key features (means, covariances, etc.) in the
> individual-level data while eliminating many higher-order moments.
> This approach is very promising, but still exploratory.

[**Recommendation 13a**]{.underline}

#### Implications of OMB's New Survey Quality Standards {#implications-of-ombs-new-survey-quality-standards .unnumbered}

# American Statistical Association

"The ASA/AC commends the Census Bureau on the development of standards
for data quality and recommends that the Census Bureau institute an
audit program to identify which programs are not compliant and why.
These audits should be accompanied by access to funding, where needed,
for programs to bring them into compliance."

**[Census Bureau Response]{.underline}**

The Economic Programs Directorate already has such an audit program. The
other directorates are considering implementing their own audit
programs.

**[Recommendation 13b]{.underline}**

# American Statistical Association

# 

# "Other topics where standards or guidelines could be considered are:

-   seasonal adjustment;

-   derived statistics programs;

-   coverage;

-   benchmarking (for example of sub-annual data to annual census data);

-   data revisions; and

-   timeliness"

**[Census Bureau Response]{.underline}**

-   Seasonal adjustment -- The Census Bureau already has a guideline for
    seasonal adjustment.

-   Derived statistics -- The Census Bureau is investigating the
    different types of derived

> statistics and how standards might apply to them.

-   Coverage -- The Census Bureau has issued a standard that addresses
    coverage. These

> requirements will be carried over in the new set of standards.

-   Benchmarking -- The Census Bureau has a set of informal guidelines.
    We will consider

> formalizing these guidelines or including requirements within the
> standards.

-   Data revisions -- The Census Bureau agrees that the topic of
    revisions is complex and

> needs investigation.

-   

-   Timeliness -- The Census Bureau does not intend to issue a standard
    to address timeliness

> Timeliness usually is mandated by law (as in the delivery date for
> decennial census counts) or is written into contracts with survey
> sponsors.

**[Recommendation 14]{.underline}**

#### 2010 Census Language Program {#census-language-program .unnumbered}

# American Statistical Association

"The ASA appreciates the difficulty in selecting a limited number of
foreign languages in which to prepare 2010 questionnaires. We also
recognize that there is ultimately no single best solution to this
problem. Census staff have already carefully considered this issue and
have developed several thoughtful alternative criteria that emphasize
household versus person versus language-spoken criteria. One way to
select among these criteria is to consider which is most efficient for
identification of households that will require language assistance. The
household-based criteria, which identify linguistically isolated
households (in which no person aged 14+ speak English at least "very
well, would seem to be the criteria most likely to identify such
households. We would expect that the number of households represented by
these criteria would be optimal in that nearly 100% could be expected to
require and benefit from a language-specific questionnaire. The
proportions of households identified using the person and
language-spoken criteria that would be dependent on a language-specific
instrument would be lower, given that some of these households would in
fact include persons who spoke or read English very well. Hence,
decisions on which foreign languages to emphasize for instrument
development would seem to be best if based on the household-criteria
identified by Census staff.

Regarding the second question raised by Census Staff, the choice of
stratification criteria is ultimately a tradeoff of acceptable coverage
versus acceptable cost. Considering these alternative criteria, one
approach would be to select the option that provided the best coverage
of language-isolated households (those in which no one age 14+ speaks
English very well) while also requiring the Bureau to mail out the
fewest total number of dual-language forms. One possibility would be to
consider the final entry in Table 3, which would identify the stratum
containing 8 or more percent households which were language-isolated to
be targeted for dual language forms. A total of 9.1 million households
are represented by this stratum, which would identify 70% of these
\"most in need\" households."

**[Census Bureau Response]{.underline}**

The Census Bureau appreciates the ASA's well-considered recommendations
on these two questions. After taking into consideration the CACPA
recommendations, as well as those of the other advisory committees, we
have determined to adopt the following criterion "Number of Housing
Units where no individual in the household 15+ speaks English very
well." This is in line with your recommendation that a household-based
criterion be selected. We selected the age of 15, rather than the 14
used in the definition of Linguistic Isolation, to be in line with the
Census Bureau's general field practice that treats individuals 15 and
above as adults for interviewing purposes.

With regard to the stratification criteria, the Bureau has selected the
criteria depicted in Table 2 (One Spanish-Speaking Person 15+ in
household who speaks English "Well", "Not Well" or "Not at All") for the
2007 Census Bilingual Form Study and for Dress Rehearsal. Barring any
unanticipated results, we would use this criterion again in selecting
the 2010 housing units to receive the bilingual form. We have not yet
decided which threshold to use, e.g., whether to select all tracts in
which 15 percent of the housing units meet the criterion, or tracts in
which

20 percent meet the criterion. We plan to incorporate multiple years of
ACS data when choosing our final stratification threshold.

**[Population Association of America]{.underline}**

A. "The Census 2010 questionnaire should be available on request in
non-English languages chosen to target the largest populations of people
who are not fluent in English. Because the available census and ACS
information on English proficiency is limited in relevance --- e.g., the
question asks about ability to speak English not about ability to read
and respond to an English-language questionnaire --- and reliability,
the Census Bureau should consider tapping other sources of data, e.g.,
data on recent immigration from non-English language countries from the
Department of Homeland Security."

B. "The Census Bureau should consider having the 2010 questionnaire
available on request in more than five languages for numerous reasons.
These include: (1) the non-English speaking portion of the American
population continues to grow rapidly because of continuing high levels
of immigration; (2) it is necessary to translate only the short form
rather than the long form and translation of the additional materials
such as assistance guides into numerous languages is already planned,
and (3) it may then not be necessary to discard languages used in the
Census 2000 questionnaire."

C. "The Census Bureau should keep in mind what might happen in the areas
targeted for Spanish-English questionnaires. Many are linguistically
diverse, containing many speakers of languages other than Spanish,
together with large monolingual English populations. Monolingual English
speakers may feel boxed-in by bilingual English-Spanish questionnaires
and the publicity materials offering the possibility of obtaining
questionnaires in other non-English languages."

######### Census Bureau Response

A. The Census Bureau is planning to make the 2010 questionnaire
available upon request to the top five populations that meet the
criterion. We are unaware of any data set that provides the depth and
breadth of information about language need as in the ACS. We will,
however, solicit the views of the advisory committees this summer and
fall regarding which languages should be selected to produce language
guides, that can be mailed to respondents upon request. If the members
of this committee have any data supporting specific language needs, we
welcome it as part of that process.

B. While we appreciate the recommendation, producing and processing more
than five non-English questionnaires will not be possible within the
cost structure of the 2010 Census. We anticipate meeting the language
needs of respondents who do not speak one of these five languages with
language guides or in-person assistance.

C. We agree that we need to look at the potential linguistic diversity
of the tracts selected for distribution of the English/Spanish bilingual
form. We had hoped to examine the data from the 2007 Census Bilingual
Form Study for this information. However, upon review of the households
selected for the 2007 study that emerge in areas with high
concentrations of other languages spoken, we found that we will have an
insufficient number of cases in such areas to support any analysis or
generalization to the 2010 Census. However, we have begun to identify
diverse tracts for the 2010 Census to determine the magnitude of this
issue and will keep the issue in mind when planning and evaluating the
2010 mailing.

**[Recommendation 15]{.underline}**

**Demographic Programs Update**

**[Population Association of America]{.underline}**

# 

"The PAA/AC commends the U.S. Census Bureau for following through on our
recommendation to quickly fully staff the Population Projections Branch.

The PAA/AC recommends that a close working relationship be cultivated
between the Population Projections Branch and the Population Estimates
Branch. This will facilitate the development and production of
age-race-sex information, which is generated by a technique at the core
of population projections, the cohort-component method.

The PAA/AC makes this recommendation because age-race-sex information
generated by the cohort-component method must be combined with data
produced by the Population Estimates Branch to generate products
important to many of the Census Bureau's programs, including controls
for the American Community Survey (ACS)."

**[Census Bureau Response]{.underline}**

The Population Projections Branch and the Population Estimates Branch
have been working closely over the last several months and plan to
continue a close relationship in the future. The Population Division
believes that there is an opportunity for synergy between the two areas
and that the work of each area can inform the other. People from each
area currently participate in cross-branch teams in order to make sure
that there is a coordinated approach between the population estimates
and the population projections.

Two personnel moves will help to facilitate a closer relationship
between estimates and projections. Fred Hollmann has moved from the
Projections Branch to the Research and Methodology Branch. Dean Judson
has also moved from the Immigration Statistics Staff to the Research and
Methodology Branch.

**[Recommendation 16]{.underline}**

**Enumerating Problematic Small Multi-Unit Addresses in the 2010
Census**

**[American Statistical Association]{.underline}**

"The ASA/AC commends the Census Bureau for its attention to the
continuing problem of identification and enumeration of housing units in
small, multi-unit buildings. This problem is both widespread and diverse
in nature, ranging from basement apartments being rented in apparent
single-family dwellings to subdivision of existing units to accommodate
a growing or demographically different population. The ASA/AC recommends
a two-stage approach to the problem.

The first stage involves the identification of potential problem areas,
using wherever possible the following sources of information: the
100-percent block canvass; Regional Office knowledge from field work on
current surveys; data from Headquarters on "at-risk" areas based on an
analysis of the Master Address File (MAF) and otherwise; and local input
via the Local Update of Census Addresses (LUCA) program. The affected
areas may be too small to warrant changing the data collection
methodology for a full assignment area, or even a crew leader district.
Consequently, for the second stage the Census Bureau should consider
forming mobile teams that can be allocated to the affected areas to
provide a cost-effective solution."

**[Census Bureau Response]{.underline}**

Work to develop an approach for the identification, listing, and
enumeration of small multi-unit buildings is underway. Currently, we are
developing special guidance for our Address Canvass workers to use in
recognizing small multi-units, reconciling what is on the ground with
what is on the address list, and updating the address list in a clear
and consistent manner.

An updated address list, while important in relocating an address,
doesn\'t necessarily guarantee the delivery of a questionnaire. As a
result, we are also considering the use of an Update/Enumerate
methodology in areas with high concentrations of these problematic
multi-unit buildings. Our initial thoughts are that the designation of
areas where we would use the Update/Enumerate methodology would be
based, in large part, on local knowledge provided

via the Regional Offices. Development of our plans for handling small
multi-units is evolving.

We will consider your recommendations as we formulate our plan. Our
final plan must strike a balance between cost constraints and
operational feasibility.

**[Population Association of America]{.underline}**

# 

"The identification and enumeration of housing units in small,
multi-unit buildings has been with us for decades, but indications are
that this is a growing problem across the entire nation, extending well
beyond the traditional immigrant-receiving states. Other problems that
are not related to immigration are found in a number of communities,
ranging from places where homeowners subsidize their mortgage payments
by renting a basement apartment to a student, to areas where the
subdivision of existing units has occurred in an attempt to accommodate
a growing population. Several Panels of the National Research Council's
Committee on National Statistics dealing with census issues have
documented the existence of this problem.

The PAA/AC appreciates the difficulty of dealing with the canvassing and
enumeration issues surrounding small multi-unit buildings. The PAA/AC
commends the Census Bureau for its persistence in developing solutions
to list and enumerate these addresses and urges them to continue their
efforts to achieve a cost-effective solution to the problem.

The classification of housing units in the 100-percent block canvass
needs to be used as a means of determining whether blocks have an
abundance of problems that will interfere with the effective mail
delivery of census questionnaires. When a threshold is reached, the
local Census Regional Office ---- in cooperation with the Regional
Census Center and Census Bureau Headquarters ---- should make a decision
regarding special enumeration methods, based on the density of problems.
Namely, this involves removing blocks from the mail-out and implementing
update/enumerate methods. This means that if a number of buildings on
the blocks in a Crew Leader District are flagged in the 100-percent
Block Canvass as being problematic (based upon a set of criteria), then
these blocks should receive Update/Enumerate methods. This should
involve at least four sources of information: the 100-percent block
canvass; Regional Office knowledge from current surveys field work; data
from Headquarters on "at-risk" areas; and local input via the LUCA
program.

The PAA/AC recognizes that the problems associated with poorly labeled
housing units affects many areas of the nation, from large cities to
rural areas. The PAA/AC recommends using the 100-percent block canvass
and other locally derived input to identify target areas for the use of
update/enumerate methods."

**[Census Bureau Response]{.underline}**

Work to develop an approach for the identification, listing, and
enumeration of small multi-unit buildings is underway. Currently, we are
developing special guidance for our Address Canvass workers to use in
recognizing small multi-units, reconciling what is on the ground with
what is on the address list, and updating the address list in a clear
and consistent manner.

An updated address list, while important in relocating an address,
doesn\'t necessarily guarantee the delivery of a questionnaire. As a
result, we are also considering the use of an Update/Enumerate
methodology in areas with high concentrations of these problematic
multi-unit buildings. Our initial plans are that the designation of
areas where we would use the Update/Enumerate methodology would be
based, in large part, on local knowledge provided via the Regional
Offices. Development of our plans for handling small multi-units is
evolving. We will consider your recommendations as we formulate our
plan. Our final plan must strike a balance between cost constraints and
operational feasibility.

**[Recommendation 17]{.underline}**

**Additional recommendation on a future ACS session**

**[Population Association of America]{.underline}**

"Informed advice requires informed advisors. In this spirit, the PAA/AC
recommends that the Census Bureau devote at least one session at the
fall 2007 advisory committee meeting to a thorough exposition and
discussion of ACS person weighting procedures. These procedures are
increasingly important in Census products and analyses, and the PAA/AC
believes that detailed examination of ACS weighting would significantly
improve the quality of our committee's future advice to the Census
Bureau.

ACS person weights are, roughly speaking, adjusted to match July 1st
population estimates for (county, age, sex, race, Hispanic/non-Hispanic)
cells. Controlling in this way reduces the variance of ACS estimates,
and reduces bias caused by undercoverage and nonresponse. However, it
also adds potential bias when these highly disaggregated population
estimates are incorrect. ACS estimates will consequently exhibit the
greatest biases precisely when timely estimates are most important --
i.e., when areas experience rapid and unanticipated demographic change.

To advise the Census Bureau well, the PAA/AC would like to better
understand the bias-variance tradeoffs inherent in the current person
weighting plan. Has the Census Bureau investigated using control totals
at different levels of geographic and demographic aggregation? Is there
evidence in favor of using counties, rather than smaller or larger
geographic units? Is there evidence that disaggregating control totals
by race and Hispanic status improves local estimates? Do reported
confidence intervals for ACS estimates include estimation errors in
population control totals in any way? What is the state of research on
building a feedback system that incorporates intercensal ACS results
into later control totals? Answers to such questions are critical to
evaluating and interpreting ACS data, and the PAA/AC believes that the
Census Bureau would be well served by providing the advisory committees
with as much information as possible on these issues."

**[Census Bureau Response]{.underline}**

We certainly agree with the importance of having an informed committee
to allow for informed advice. It has been four years since we have last
given a thorough description of the ACS person weighting procedures and
certainly many new members and existing members could benefit from
adding a session on the ACS weighting in Fall 2007. However, the need to
schedule a

Joint Advisory Committee meeting on the 2010 Census Communication
strategy will interfere with our desire to get more input on the ACS. We
will, however, extend the spring 2008 Advisory Committee meeting by a
half-day in order to allow a more thorough discussion of ACS issues. The
Census Bureau welcomes the opportunity to share critical information on
weighting and estimation with the Committee. As part of that session, we
will update the PAA/AC on ongoing research investigating the tradeoff
between bias and variance by using the controls at different levels of
both geography and demographic detail. We will also provide an update on
how the ACS serves to inform the production of the population estimates
in later years.

*Census Control Number 50688*

ACO:JGreen:8/17/07

CQAS:Review:cjf/lmh/acw:8/24/07

CQAS:Review:msw:9/6/07

Edits:J. Keller:msw:9/7/07

CQAS:Final:cjf:9/10/07

cc:

R. Cymber, J. Keller, J. Green