# Presentation: 687236

## Small Area Data Quality:A Comparison of Neighborhood Estimates in the Bronx Test Site

**Joseph Salvo**
**Peter Lobo**
**Timothy Calabrese**
**Population Division**

**2000 Census and the****1999-2001 American Community Survey**

**Small Area Data Quality:****A Comparison of Neighborhood Estimates in the Bronx Test Site**

**National State Data Center Meeting**
**September 21, 2004**

## Study Objectives

- Compare content
- Focus on neighborhoods
- Evaluate “real-world” effects

## Study Area

- 1.3 million persons & 491,000 housing units
- 63 percent of housing units in structures with 20 or more units
- Dense population in the west and south
- Almost 50% Hispanic and 31% nonhispanic Black

## Slide 4

## Population Density

**Population Density**
**Bronx, NY**

## Race/Hispanic Origin

- Source: Census 2000 SF1

**Race/Hispanic Origin**
**Bronx, NY**

## Public Assistance Recipiency

- Source: NYC Human Resources Administration

**Public Assistance Recipiency**
**Bronx, NY**

## Content Comparisons

- Statistically Significant Differences
- Meaningful differences

## Analysis of Survey Content

***Total*** data items (not including universes and summary statistics) - 235
- Data items with ***statistically*** ***significant differences*** at county level - 159 (68 percent)
- Data items with ***meaningful differences*** of at least +/- 2 percent, at county level – 38 (16 percent)

## Matrix of Items

- Less than one fifth
- of all neighborhoods
- showing a statistically
- significant difference
- between Census and ACS

- One fifth or more
- of all neighborhoods
- showing a statistically
- significant difference
- between Census and ACS

- Percentage Point
- Difference at the
- County Level:

- Under 2.0 points

- 2.0 points or more

|  |  | 
 |
| --- | --- | --- |
|  |  |  |
|  |  |  |

- Total

- Total

**(B)**** Different**

- 13 items

**(A)**** Similar**

- 190 items

**(D)**** Most Different**

- 25 items

**(C)**** Different**

- 7 items

- 197

- 38

- 235

- 203

- 32

## Example of Matrix Cell APoverty:Individuals-Related Children Ages 5 to 17 Years

**Example of Matrix Cell A*****Poverty:******Individuals-*****Related Children ****Ages 5 to 17 Years**

- Sources:
- 1999-2001 American Community Survey (ACS)
- And 2000 Decennial Census

- Percent ACS:

- 2000 Census:

- 1999 – 2001
- Average ACS:

- Percent Census:

- Difference
- (%ACS - %Census):

- P Value:

- Matrix Cell:

- 117,223

- 115,820

- 29.9

- 31.0

- 1.2

- 0.016

- A

## Matrix of Items

- Less than one fifth
- of all neighborhoods
- showing a statistically
- significant difference
- between Census and ACS

- One fifth or more
- of all neighborhoods
- showing a statistically
- significant difference
- between Census and ACS

- Percentage Point
- Difference at the
- County Level:

- Under 2.0 points

- 2.0 points or more

|  |  | 
 |
| --- | --- | --- |
|  |  |  |
|  |  |  |

- Total

- Total

**(B)**** Different**

- 13 items

**(A)**** Similar**

- 190 items

**(D)**** Most Different**

- 25 items

**(C)**** Different**

- 7 items

- 197

- 38

- 235

- 203

- 32

##  Data Items of Concern

**Disability**
**Year Structure Built**
**House Heating Fuel**
**Others: Rooms, Employment Status**

## Disability: Persons 21 to 64

- ACS – 19 percent
- 	Census – 32 percent
- Allocation rates twice as high in Census than ACS
- The outcome of a planning application could be greatly affected by which survey is used

## The Disability 

- The Disability
- Questions
- (#16 and 17)
- from the
- Long-form
- questionnaire

## Example of Matrix Cell DDisability:Population 21 to 64 Years-With a Disability

**Example of Matrix Cell D*****Disability:******Population 21 to 64 Years-*****With a Disability**

- Sources:
- 1999-2001 American Community Survey (ACS)
- And 2000 Decennial Census

## Population 21 to 64 Years,

**ACS Average 1999 - 2001**

**Census Number**

**Number:**
**Population 21 to 64, with a Disability**

**Population 21 to 64 Years,**
**With a Disability**
**In the ACS and the Census**

- Sources:
- 1999-2001 American Community Survey (ACS)
- And 2000 Decennial Census

## House Heating Fuel

- Fuel Oil, Kerosene, etc.:
- 	ACS - 61 Percent
- 	Census - 40 Percent
- Utility Gas:
- 	ACS – 29 Percent
- 	Census – 43 Percent
- Electricity:
- 	ACS – 7 Percent
- 	Census – 10 Percent
- NYCHVS closer to ACS estimate
**Shows approximately 75 percent of households using oil heat**

## Example of Matrix Cell DHome Heating Fuel:Occupied Housing UnitsHeated with Fuel Oil,Kerosene, etc.

**Example of Matrix Cell D*****Home Heating Fuel:******Occupied Housing Units******Heated with Fuel Oil,******Kerosene, etc.***

- Sources:
- 1999-2001 American Community Survey (ACS)
- And 2000 Decennial Census

- Percent ACS:

- 2000 Census:

- 1999 – 2001
- Average ACS:

- Percent Census:

- Difference
- (%ACS - %Census):

- P Value:

- Matrix Cell:

- 276,296

- 182,855

- 39.5

- 60.9

- 21.4

- 0.00

- D

## Year Structure Built:1939 or Earlier

- ACS - 37 percent
-    Census - 28 percent
- High allocation rates in both surveys; ACS slightly higher than the Census
- Administrative data show that 56 percent of units were built prior to 1940

## Example of Matrix Cell DYear Structure Built:Total Housing Units-Built in 1939 or Earlier

**Example of Matrix Cell D*****Year Structure Built:******Total Housing Units-*****Built in 1939 or Earlier**

- Sources:
- 1999-2001 American Community Survey (ACS)
- And 2000 Decennial Census

- Percent ACS:

- 2000 Census:

- 1999 – 2001
- Average ACS:

- Percent Census:

- Difference
- (%ACS - %Census):

- P Value:

- Matrix Cell:

- 179,792

- 134,870

- 27.5

- 36.6

- 9.1

- 0.00

- D

## Conclusions

- Overall picture – there are only small differences between the ACS and Census for _most_ data items
- BUT... - about 16 percent of data items have significant and meaningful differences that could affect planning outcomes and data uses.

## Major Issues

- Data Quality
**Mail return rates are substantially lower for the ACS than the Census**
**Creates an over-reliance on superior nonresponse follow-up**

## Major Issues (continued)

**Some Small Hard-to-Enumerate Areas may need more Follow-up to Achieve Sufficient Sample Size**
- This carries considerable additional cost
**Cuts in Sample Size will Threaten the Quality of Estimates**
- 355 Census Tracts had to be aggregated into 88 neighborhoods because the sample size was not large enough for tract estimates