# Presentation: 506428

## IPUMS:

**IPUMS:**
**How we make it,**
**how you can get it,**
**and how you can use it**

- Trent Alexander

- Minnesota Population Center
- University of Minnesota

## Introduction to the IPUMS Project

**3. Harmonization**

**4. Additional Data Enhancements**

**5. Users and Access**

**6. Strengths and Limitations**

**7. Dissemination**

**1. What is the IPUMS?**

**2. Data entry and coding**

**Introduction to the IPUMS Project**

## Datasets in IPUMS-USA

- 1850

-   1.0

-    198,000

- 1860

-   1.0

- 354,000

- 1870

-   1.0

- 428,000

- 1880

-   1.0

-     503,000

- 1900

-   1.0

- 846,000

- 1910

-   1.4

- 1,503,000

- 1920

-   1.0

- 1,050,000

- 1930

-   0.5

- 606,000

- 1940

-   1.0

- 1,351,000

- 1950

-   1.0

- 1,922,000

- 1960

-   1.0

- 1,800,000

- 1970

-   6.0

- 12,180,000

- 1980

-   9.0

- 20,403,000

- 1990

-   6.0

- 15,000,000

- 2000

-   6.0

- 16,884,000

- 0.4-1.0

- 2001-2005

-   5,700,000

**Datasets in IPUMS-USA**

**    ****Number of **
**persons in dataset**

** **

**Census**

** **

** **

**Sample**

** **

**        **

** **

** **

**Year**

** **

** **

** **

**Density**

** **

## Datasets in IPUMS-USA

- 1850

-   10.0

-    1,980,000

- 1860

-   1.0

- 354,000

- 1870

-   1.0

- 428,000

- 1880

-   10.0

-     5,030,000

- 1900

-   6.0

- 4,230,000

- 1910

-   1.4

- 1,503,000

- 1920

-   1.0

- 1,050,000

- 1930

-   5.0

- 6,060,000

- 1940

-   1.0

- 1,351,000

- 1950

-   1.0

- 1,922,000

- 1960

-   6.0

- 10,800,000

- 1970

-   6.0

- 12,180,000

- 1980

-   9.0

- 20,403,000

- 1990

-   6.0

- 15,000,000

- 2000

-   6.0

- 16,884,000

**Datasets in IPUMS-USA**
**Planned 2007-2010**

**    ****Number of **
**persons in dataset**

** **

**Census**

** **

** **

**Sample**

** **

**        **

** **

** **

**Year**

** **

** **

** **

**Density**

** **

- 0.4-1.0

- 2001-2005

-   5,700,000

- 2006- ??

- 1.0/year

## Status of Countries in IPUMS-International

- Argentina
- Belarus
- Brazil
- Cambodia
- Chile
- China
- Costa Rica
- Ecuador
- France
- Greece
- Hungary
- Israel
- Kenya

**Status of Countries in IPUMS-International**

- Mexico
- Palestinian Territories
- Philippines
- Portugal
- Romania
- Rwanda
- South Africa
- Spain
- Uganda
- United States
- Venezuela
- Vietnam

## Status of Countries in IPUMS-International

**Latin America**

**Europe**

**Asia, Africa, Other**

- Argentina
- Belarus
- Brazil
- Cambodia
- Chile
- China
- Costa Rica
- Ecuador
- France
- Greece
- Hungary
- Israel
- Kenya

- Austria

- Armenia

- Bolivia

- Bulgaria

- Canada

- Czech Republic

- Egypt

- Dominican Republic

- Germany

- Fiji

- Indonesia

- El Salvador

- Iraq

- Guatemala

- Ireland

- Honduras

- Netherlands

- Malaysia

- Nicaragua

- Mongolia

- Panama

- Slovenia

- Pakistan

- Paraguay

- Peru

- United Kingdom

- Uruguay

- Tajikistan

- Turkmenistan

**Data Received or Agreement Signed**

**Status of Countries in IPUMS-International**

**Current funding for 44 countries by 2009**

**Next data release late Winter 2007**

- Mexico
- Palestinian Territories
- Philippines
- Portugal
- Romania
- Rwanda
- South Africa
- Spain
- Uganda
- United States
- Venezuela
- Vietnam

## Datasets in IPUMS-CPS

**Year**

**Household**

**Person**

**Year**

**Household**

**Person**

- 1962

- 31,106

- 71,741

- 1983

- 73,195

- 162,635

- 1963

- 24,649

- 55,882

- 1984

- 73,632

- 161,167

- 1964

- 23,438

- 54,543

- 1985

- 74,568

- 161,362

- 1965

- 23,600

- 54,502

- 1986

- 74,145

- 157,661

- 1966

- 48,095

- 110,055

- 1987

- 73,843

- 155,468

- 1967

- 28,924

- 68,676

- 1988

- 74,806

- 155,980

- 1968

- 46,069

- 150,913

- 1989

- 70,454

- 144,687

- 1969

- 47,028

- 151,848

- 1990

- 75,269

- 158,079

- 1970

- 44,982

- 145,023

- 1991

- 75,076

- 158,477

- 1971

- 45,952

- 146,822

- 1992

- 74,236

- 155,796

- 1972

- 44,906

- 140,432

- 1993

- 73,878

- 155,197

- 1973

- 44,467

- 136,221

- 1994

- 73,126

- 150,943

- 1974

- 44,427

- 133,282

- 1995

- 72,152

- 149,642

- 1975

- 43,714

- 130,124

- 1996

- 63,339

- 130,476

- 1976

- 46,368

- 135,351

- 1997

- 64,046

- 131,854

- 1977

- 68,291

- 160,799

- 1998

- 64,659

- 131,617

- 1978

- 67,900

- 155,706

- 1999

- 65,377

- 132,324

- 1979

- 68,375

- 154,452

- 2000

- 64,944

- 133,710

- 1980

- 80,468

- 181,488

- 2001

- 64,362

- 128,821

- 1981

- 81,451

- 181,358

- 2002

- 98,848

- 217,219

- 1982

- 73,368

- 162,703

- 2003

- 99,986

- 216,424

- 2004

- 98,979

- 213,241

- 2005

- 98,664

- 210,648

- 2006

- 98,069

- 209,542

## Data in North Atlantic Population Project (NAPP)

| Country | Census year | Population included |
| --- | --- | --- |
| Canada | 1881 | 100% |
| Great Britain | 1881 | 100% |
| Iceland | 1870 | 100% |
| Iceland | 1880 | 100% |
| Iceland | 1901 | 100% |
| Norway | 1865 | 100% |
| Norway | 1900 | 100% |
| Sweden | 1890 | 100% |
| United States | 1880 | 100% |

## NAPP Plans, 2007-2011link data across borders and time

| Canada | Great Britain | Iceland | Norway | Sweden | United States |
| --- | --- | --- | --- | --- | --- |
| 1852 | 1851 | 1703
 | 1801 | 1890 | 1850 |
| 1871 | 1881 | 1835 | 1865 | 1900 | 1860 |
| 1881 |  | 1845 | 1875 |  | 1870 |
| 1891 |  | 1870 | 1900 |  | 1880 |
| 1901 |  | 1880 |  |  | 1900 |
| 1911 |  | 1900 |  |  | 1910 |
| 1921 |  |  |  |  | 1920 |
| 1931 |  |  |  |  | 1930 |
| 1941 |  |  |  |  |  |
| 1951 |  |  |  |  |  |

## What Are Microdata?

**Individual-level data**
**	**
**   **•  every record represents a separate person
-    •  all of their individual characteristics are recorded
-    •  users must manipulate the data themselves

**Different from aggregate/summary/tabular data**
**    ****•****  **a disability table from www.factfinder.census.gov
-     •  an occupation table from a published census volume
- 	from the library

## 1930 Census Population Schedule

## Raw Census Microdata from IPUMS

**Raw Census Microdata from IPUMS**

## IPUMS Data 

**IPUMS Data **
**Structure**

**Household record**
**(shaded) followed**
**by a person record**
**for each member **
**of the household**

**For each type of**
**record, columns **
**correspond to**
**specific variables**

## The Advantages of Microdata

******   ****Combination of all of a person’s characteristics **
******   ****Characteristics of everyone with whom a person lived**
**	**
******   ****Freedom to *****make***** any table you need**
**	**
******   ****Freedom to make models examining multivariate**
**	****         ****relationships**

## Introduction to the IPUMS Project

**3. Harmonization**

**4. Additional Data Enhancements**

**5. Users and Access**

**6. Strengths and Limitations**

**7. Dissemination**

**2. Data entry and coding**

**2. Data entry and coding**

**Introduction to the IPUMS Project**

## How a case gets from the manuscript census into the IPUMS

**Vice President of the U.S., 1856-1860**
**Secretary of War, C.S.A, 1861-1865**
**Later charged with treason, fled to Cuba**
**          **

**How a case gets from the manuscript census into the IPUMS**

**An example from the 1860 census....**

## Original enumeration form from the 1860 U.S. Census

**Original enumeration form from the 1860 U.S. Census**

## Data entry screen in Minnesota (ca. 1997)

**Data entry screen in Minnesota (ca. 1997)**

## Household and person record ready for checking (ca. 1999)

**Household and person record ready for checking (ca. 1999)**

## Coding dictionary for the occupation variable (ca. 2000)

**Coding dictionary for the occupation variable (ca. 2000)**

## Checked and coded data, ready for harmonization (ca. 2001)

**Year**

**Industry**

**Page**

**Wealth**

**Age**

**Relationship**

**Checked and coded data, ready for harmonization (ca. 2001)**

**Occupation**

## Introduction to the IPUMS Project

**3. Harmonization**

**4. Additional Data Enhancements**

**5. Users and Access**

**6. Strengths and Limitations**

**7. Dissemination**

**2. Data entry and coding**

**Introduction to the IPUMS Project**

## Translation Matrix – Marital Status

**How we integrate variables across time (and countries)**

## Translation Matrix – Marital Status

**Translation Matrix – Marital Status**

**location of data in the original samples**

## Translation Matrix – Marital Status

**Translation Matrix – Marital Status**

**location of data in the 1960 U.S. Census Bureau file**

## Translation Matrix – Marital Status

**Translation Matrix – Marital Status**

**different original codes for “widowed” across the censuses**

## Translation Matrix – Marital Status

**Translation Matrix – Marital Status**

**final IPUMS coding scheme for marital status**

## Variable Description: Farm Status  (IPUMS-USA)

## Codes: Farm Status (IPUMS-USA)

## Variable Description: Literacy  (IPUMS-Intl)

## Variable Description: Union  (IPUMS-CPS)

## Introduction to the IPUMS Project

**3. Harmonization**

**4. Additional Data Enhancements**

**5. Users and Access**

**6. Strengths and Limitations**

**7. Dissemination**

**2. Data entry and coding**

**Introduction to the IPUMS Project**

## Additional Improvements to the U.S. PUMS

******    ****Additional documentation, including all**
**      ****enumeration forms and instructions**
**    ****Consistent occupation/industry classifications**
**      **
******    ****Consistent metropolitan classifications**
******    ****Missing data allocation**
******    ****Constructed family variables**

## IPUMS “Pointer” Variables

| Pernum | Relate | Age | Sex | Marst | Chborn |
| --- | --- | --- | --- | --- | --- |
| 1 | head | 46 | male | married | n/a |
| 2 | spouse | 44 | female | married | 3 |
| 3 | aunt | 77 | female | widow | 7 |
| 4 | child | 15 | female | single | 0 |
| 5 | child  | 13 | female | single | n/a |
| 6 | child | 11 | male | single | n/a |

| Pernum | Relate | Age | Sex | Marst | Chborn |
| --- | --- | --- | --- | --- | --- |
| 1 | head | 46 | male | married | n/a |
| 2 | spouse | 44 | female | married | 3 |
| 3 | aunt | 77 | female | widow | 7 |
| 4 | child | 15 | female | single | 0 |
| 5 | child  | 13 | female | single | n/a |
| 6 | child | 11 | male | single | n/a |

**Spouse’s**

**Mother’s**

**Father’s**

**IPUMS “Pointer” Variables**

| Location |
| --- |
|   |
|   |
|   |
|   |
|   |
|   |

**2**

**1**

**0**

**0**

**0**

**0**

| Location |
| --- |
|   |
|   |
|   |
|   |
|   |
|   |

| Location |
| --- |
|   |
|   |
|   |
|   |
|   |
|   |

**0**

**0**

**0**

**0**

**0**

**0**

**2**

**1**

**1**

**1**

**2**

**2**

- (Simple household)

## Introduction to the IPUMS Project

**3. Harmonization**

**4. Additional Data Enhancements**

**5. Users and Access**

**6. Strengths and Limitations**

**7. Dissemination**

**2. Data entry and coding**

**Introduction to the IPUMS Project**

## Extract requests per month, 2002-2007

**15,000 users have made 85,000 data extracts**

## IPUMS Users’ Disciplines

**Sociology  (16%)**
**Demography  (12%)**
**Other Academic  (19%)**
- Historians: only 3%!!!
**Other Non-academic  (15%)**

**IPUMS Users’ Disciplines**

## IPUMS Users’ Status

**Faculty  (23%)**
**Academic researcher  (12%)**
**Non-academic researcher  (16%)**
**Support staff  (3%)**

**IPUMS Users’ Status**

## Number of Countries Selected for Research

**2 countries  (24%)**
**3 countries  (10%)**
**4 countries  (6%)**
**5 countries  (3%)**
**6-8 countries  (17%)**

**Number of Countries Selected for Research**
**IPUMS-International**

## Other IPUMS Data Sources

**PDQ (www.pdq.com) **
**Fathom (www.keypress.com/fathom)**

## Introduction to the IPUMS Project

**3. Harmonization**

**4. Additional Data Enhancements**

**5. Users and Access**

**6. Strengths and Limitations**

**7. Dissemination**

**2. Data entry and coding**

**Introduction to the IPUMS Project**

## 4 Key Strengths of the

**Census Microdata Samples**

## Limitations of the Microdata Samples 

## Limitations of the Different IPUMS Data Series 

** ****IPUMS-USA**

**Geography**** 1940-present**

** ****IPUMS-International**

**User burden: documentation, information overload**

** ****IPUMS-CPS**

**Sample size (60 to 200K)**

## Introduction to the IPUMS Project

**3. Harmonization**

**4. Additional Data Enhancements**

**5. Users and Access**

**6. Strengths and Limitations**

**7. Dissemination**

**2. Data entry and coding**

**Introduction to the IPUMS Project**

## Lab: Using weights and making tables

**Weights: **
**    ****	****	****why you have to use them**
**	****	****which ones you should use**
**3. Exercises**

**Lab: Using weights and making tables**

## What is a weight?

**   ****It’s a variable, just like age, sex, race, etc. **
-   Every case in every sample has a weight value
**  ****The main weighting variable in IPUMS is called...**
** **Person weight (variable name is PERWT)
**   ****The person weight variable tells you how many people nationwide are represented by any given case**
**   ****If you forget to use it, your analysis could be wrong!!!**

## How do weights look in the data?

**How do weights look in the data?**

## Sample of pets in my neighborhood

**Cases in my pet sample**

- dog

- cat

- cat

- cat

- rabbit

- rabbit

- rabbit

- rabbit

- 8 cases in sample

- 50% are rabbits

## New estimates that take weights into account

**Number of pets**

**Cases in my pet sample**

**in my neighborhood**

**that each sample**

**pet represents**

**(PERWT)**

- dog

- 200

- cat

- 100

- cat

- 100

- cat

- 100

- rabbit

- 25

- rabbit

- 25

- rabbit

- 25

- rabbit

- 25

- 8 cases in _**sample**_

- 50% are rabbits

## New estimates that take weights into account

- Wrong

- Right

**Number of pets**

**Cases in my pet sample**

**in my neighborhood**

**Neighborhood pet**

**that each sample**

**population estimates**

**pet represents**

**(PERWT)**

- dog

- 200

- 200 dogs

- cat

- 100

- cat

- 100

- cat

- 100

- 300 cats

- rabbit

- 25

- rabbit

- 25

- rabbit

- 25

- rabbit

- 25

- 100 rabbits

- 8 cases in _**sample**_

-  600 pets in _**population**_

- 50% are rabbits

- 16.7% are rabbits

## Why are some cases worth more than others?

**Intentional oversampling of important small groups**
**  ****1860-1870: African-Americans**
**  ****1900-1910: Alaskans, Hawaiians, and American ****	****	****Indians**
**  ****2000-2005: rural population**
- U**nintentional over/undersampling of the entire population**
**  ****we use weights to fix this problem (for instance, by randomly alternating weights between 99 and 100)**
**  ****only done when error is very minor**
**Complex sample designs**
**  ****1940-1950: “sample-line” strategy**
**  ****1990-2000 and ACS: stratified sampling strategy**

## Which weights to use

**   ****Household weight (HHWT)**
**for analyses seeking to make statements about households (must choose 1 person to represent each household)**
**   ****Person weight (PERWT)**
**for analyses seeking to make statements about individuals**
**   ****Sample line weight (SLWT)**
**for analyses seeking to make statements about individuals,  only use when a 1940-50 “sample line” variable is used**

## Which weights to use

**   ****The question of which weight you should use depends on what YOU want to say. **
**  ****For instance.....**
**4.8 million households did not have flush toilets in 1960 **
***	******use HHWT***
**19.2 million people did not have a flush toilet in their home in 1960**
**       *****use PERWT***