CPS OVERLAP ANALYSIS TEAM TECHNICAL REPORT 3

MODE EFFECTS ANALYSIS OF

LABOR FORCE ESTIMATES

Prepared by

Jenny Thompson

for the

CPS Mode Effects Workgroup

Donna Kostanich, Chair

Robin Fisher

Jonathan Kurlander

Paul Mullin

Randy Parmer

Oscar Perez

Jennifer Rothgeb

Jenny Thompson

Mike Welch

April 14, 1994
ACKNOWLEDGEMENTS

We would like to acknowledge the assistance of Larry Altmayer in the processing of the data for

this report. We would also like to acknowledge the contributions of the members of the CPS

Questionnaire Design Subcommittee without whose guidance our work could not have been

accomplished. Members of the committee throughout the years included, John E. Bregger, Shail

Butani, Cathryn S. Dippo, Harvey R. Hamel, Anne E. Polivka, Norris C. Tucker and Alan

Tupek from the Bureau of Labor Statistics; Chester E. Bowie, Lawrence S. Cahoon, Kathleen P.

Creighton, Donna L. Kostanich, Elizabeth A. Martin, Michael McMahon, Jennifer M. Rothgeb,

Thomas S. Scopp and Ronald R. Tucker from the Bureau of the Census.
Executive Summary of

CPS OVERLAP ANALYSIS TEAM TECHNICAL REPORT 3

Mode Effects Analysis of Major Labor Force Estimates

April 14, 1994

To investigate the possible influence of changes in collection mode on major labor force

estimates, special studies were embedded in the CPS and the Parallel Survey. These studies

were designed to examine four potential sources of difference: centralized telephone

interviewing, centralized telephone interviewing combined with computer-assisted interviewing,

questionnaire (given centralized telephone interviewing), and the new questionnaire designed for

computer-assisted interviewing. Highlights of the results are:

I. Centralized telephone interviewing effects

 Some evidence that this effect exists for

 • Total unemployment rate (marginally significant)

 • Total black and black male unemployment rates

II. Centralized and computer-assisted telephone interviewing effects

 Some evidence that this effect exists for

 • Total, total female, and total male unemployment rates

 • Total white, white female, and white male unemployment rates

 • Black female unemployment rate

III. Questionnaire, given centralized telephone interviewing, effects

 Some evidence that this effect exists for

 • Total female unemployment rate (marginally significant)

 • Total black and black female unemployment rates

ii
4

IV. New Questionnaire designed for computer-assisted interviewing effects

 Some evidence that this effect exists for

 • Total, total male, and total female unemployment rates

 • Total white, white male, and white female unemployment rates

 • Total black, black male, and black female unemployment rates

 • Total female and white female CLF participation rate

These results are presented with qualifications. The first three effects were investigated using

subnational estimates. The fourth effect, the new questionnaire designed for computer-assisted

interviewing, does use national estimates, but only a quarter of the total sample. In general, the

sample sizes for these studies are small, particularly for the centralization effect. Few attempts

were made to adjust for multiple comparisons within tests. When considered jointly, the overall

confidence level decreases substantially. 
TABLE OF CONTENTS

INTRODUCTION ...............................................................................................................

1

BACKGROUND ...............................................................................................................

2

I. Sample Design ..................................................................................................

2 

II. Data Collection Design ......................................................................................

2

III. Hypotheses and Experimental Design .............................................................

4

A. Hypothesis One: No Centralized Telephone Interviewing 

Effect .................................................................................................. 6

1. Description ......................................................................................

6

2. Experimental Design ............................................................. 6

3. Limitations ......................................................................................

7

B. Hypothesis Two: No Centralized and Computer-Assisted

 Telephone Interviewing Effect .............................................................

8

1. Description ......................................................................................

8

2. Experimental Design ....................................................... 8

3. Limitations ......................................................................................

9

C. Hypothesis Three: No New Questionnaire, Given Centralized

 Telephone Interviewing Effect .............................................................

9

1. Description ......................................................................................

9

2. Experimental Design ............................................................. 9

3. Limitations ......................................................................................

10

D. Hypothesis Four: No New Questionnaire Designed for

 Computer-Assisted Interviewing Effect ................................................. 10

1. Description ......................................................................................

10

2. Experimental Design ............................................................. 10
6

3. Limitations .....................................................................................

11

IV. Estimation and Variance Estimates for Mode Effects Analysis ........... 11

V. Testing Methodology ...................................................................................... 11

A. Two Sample T-Test and Bonferroni Multiple Comparisons

 Tests .................................................................................................. 11

B. Implied Comparisons Tests ............................................................. 12

RESULTS ...........................................................................................................................

14

I. Centralized Telephone Interviewing Effect ................................................ 14

II. Centralized Telephone and Computer-Assisted Interviewing Effect ........... 15

III. New Questionnaire, Given Centralized Telephone Interviewing

 Effect ............................................................................................................... 16

IV. New Questionnaire Designed for Computer-Assisted Interviewing

 Effect .............................................................................................................. 17

SUMMARY ..........................................................................................................................

20

REFERENCES ...............................................................................................................

21
Introduction

The official monthly civilian labor force estimates from January 1994 onward are based on data

from a comprehensively redesigned Current Population Survey (CPS). The redesign

incorporates changes in the basic questionnaire and collection methodology. Bregger and Dippo

(1993) discuss the motivation for this redesign.

To gauge the effect of the CPS redesign on published estimates, a Parallel Survey (PS) was

conducted using the new questionnaire and collection procedures from July 1992 to December

1993. Annual average estimates from the PS were used to examine the effect of the CPS

redesign on major labor force estimates. Polivka (1994) presents a comparison of the labor force

estimates from the PS and CPS during 1993. Kostanich and Cahoon (1994) further consider the

possible influence of the differences in the two surveys' designs on these comparisons.

A secondary consideration was an investigation into the possible effect of selected factors

associated with the new questionnaire or collection mode on major labor force estimates. 

Special studies were embedded within the CPS and the PS during the same time period to

provide data for testing hypotheses about the effects of these methodological differences on

labor force estimates. The resultant hypothesis tests attempted to link the differences in annual

average major labor force estimates between CPS and the PS to the presence of specified

collection mode differences.

This report centers on four possible effects: two centralized telephone interviewing effects and

two new questionnaire effects. Three of the four hypothesis tests used subnational estimates of

CPS and PS data and were not necessarily representative of the national differences. The other

hypothesis test used national estimates computed from approximately one fourth of the full

sample (for both surveys).

The Background section of this report provides an overview of the two surveys' designs, a

description of the estimates and variance estimates, descriptions of hypothesis testing

methodology, and detailed descriptions of the tested hypotheses. The Results section presents

the hypothesis tests for major labor force characteristics. The Summary section provides

general conclusions.
2

Background

I. Sample Design

The CPS is a monthly survey of 60,000 eligible households. These households are

selected to represent the population of the Nation and of each State. The probability

sample of housing units is drawn using a multistage stratification procedure. The

sampled households are located in 729 selected geographic areas. The largest

metropolitan areas within each State are always included; the remaining areas of a State

are sampled with probability of selection proportionate to the population of the area1. The

sample is designed to provide a 1.7 percent monthly coefficient of variation2 on the

estimated national unemployment rate, assuming a 6 percent rate. It was also designed to

meet specific reliability criteria for the monthly level of unemployment for 11 States; the

remaining 39 have fixed levels of reliability for an annual average. At the national level,

this means that a month-to-month change of 0.2 percentage point in the estimated

unemployment rate is significant at a 90-percent confidence level. 

The sample design for the Parallel Survey (PS) was based on that of the National Crime

Victimization Survey (NCVS), which is conducted by the Bureau of the Census for the

Bureau of Justice Statistics. The PS was a monthly survey of 12,000 eligible households.

 The major purpose of the PS was to measure effects on major labor force estimates at

the national level. Consequently, these households were selected to be nationally

representative; the sample within a State was not necessarily representative of that State's

population. The probability sample of housing units was drawn using a multistage

stratification procedure. The sampled households for the PS were located in 283

geographic areas. The monthly coefficient of variation for the estimated national

unemployment rate from the PS was 3.5 percent, assuming a 6 percent rate. 

II. Data Collection Design

In an effort to balance respondent burden with improved estimates of change, households

are interviewed for 4 consecutive months, not interviewed for the next 8 consecutive

months, and then interviewed for another 4 consecutive months. Each month, a new

household panel of approximately one-eighth the total monthly sample size (60,000/8 =

7,500 households for the CPS and 12,000/8 = 1,500 for the PS) is initiated, and the panel

which received its eighth interview the previous month is dropped. Thus, each month,

 

 1 Following each decennial census, a new sample of areas is selected. The current sample

is based on the 1980 decennial census.

 2 The coefficient of variation of an estimate is defined as the standard error of the estimate

divided by the estimate.
3

eight different panels are being interviewed for the 1st, 2nd,..., and 8th time. This

rotating panel structure means that three-quarters of the sample in a given month is

retained in the sample the next month, improving the estimates of month-to-month

change. However, since the PS was initiated in 1992, and it takes 16 months to phase in

this type of rotation scheme, September 1993 was the first month in which the rotation

scheme was completely in place. In both the CPS and the PS, first and fifth month-insample households are interviewed through personal visits. For subsequent months, the

majority of interviews are conducted by telephone.

Prior to January 1994, most of the CPS data were collected with a paper survey

instrument and translated into computer readable form using FOSDIC3 technology. 

Approximately 9 percent of the data was collected by interviewers working in two

centralized facilities using computer-assisted telephone interviewing. Only households in

a subset of the CPS sample areas were eligible for centralized computer-assisted

telephone interviewing (CATI). These areas were purposely chosen based on operational

considerations and were generally large metropolitan areas. 

Centralized computer-assisted telephone interviewing had been used in the CPS since

January 1989, when a centralized facility in Hagerstown, Maryland, was opened. In order

to minimize any potential effects on published CPS estimates, the percent of sample

cases interviewed from CATI was originally kept small. Over the 5-year time period, the

percent of the CPS sample interviewed from CATI gradually increased to the 9 percent

level used in the 1993 CPS4.

From January 1991 through December 1992, the Bureau of the Census and the Bureau of

Labor Statistics jointly conducted a special study in the CPS CATI-eligible areas to

measure the effects of centralized telephone interviewing combined with computerassisted interviewing on CPS data. Findings from this study showed that inclusion of

CATI produced a 0.8 percentage point higher unemployment rate (Shoemaker, 1993). 

However, this difference could not be attributed to CATI alone. The paper-and-pencil

 

 3Film Optical Sensing Device for Input to Computers.

 4 To accommodate the increased CATI sample, a second telephone facility was opened in

Tucson, Arizona, in 1992.
6

questionnaire itself was not administered from a centralized location; rather, it was a

computerized version, with modified wording of the lead-in question to the labor force

section5. Thus, it was impossible to distinguish whether this difference was due to

centralization, computer-assisted interviewing, or a slightly modified questionnaire.

 

 5 See Rothgeb (1994) for a more detailed discussion of the lead-in question and possible

influences of computer-assisted interviewing and centralization.

All the data for the PS were collected using computer-assisted interviewing. Eighty-two

percent of the data were collected by field representatives using laptop computers, either

during personal visits to respondents' homes or by telephone from their own homes. The

remaining 18 percent of the data were collected using computer-assisted telephone

interviewing by a separate staff of interviewers working in the same two centralized

facilities used for the CPS. This CATI interviewing was conducted in a subset of the PS

sample referred to as the PS CATI-eligible areas. Select multi-interviewer sample areas

were CATI-eligible in the PS.

III. Hypotheses and Experimental Design

This analysis is a contrast study. To study the effect of a possible "treatment," a sample

was randomly split into two "independent" groups (split panels). Each panel is

statistically representative of the parent sample. The treatment is administered to

respondents in one of the two split panels. The treatment is excluded from the other

panel. The difference between the estimates from the two panels gives an estimated

difference of the "treatment effect."

The following terms are used throughout this report:

CATI Centralized Computer-Assisted Telephone Interviewing. Interviews

are conducted from a centralized telephone facility using a fully

automated version of either questionnaire (old or redesigned). A

respondent interviewed from the telephone facility rarely has the same

interviewer twice. This is centralized interviewing.

PAPI Pencil-and-Paper Interviewing (Personal Visit or Telephone).

Interviews are conducted using the old pencil-and-paper questionnaire. In

general, a respondent is interviewed by the same field representative for all

eight interviews, either by personal visit or by telephone. This is

decentralized interviewing.
5

CAPI Computer-Assisted Personal Interviewing (Personal Visit or

Telephone). Interviews are conducted with the redesigned questionnaire

using a laptop computer. As with PAPI, in general the respondent is

interviewed by the same field representative for all eight interviews, either

by personal visit or by telephone. This is decentralized interviewing.

MIS Month in Sample. This refers to the number of months that a housing

unit has been in sample. This is usually the same as the number of

interviews that a household has undergone. For example, MIS 1 refers to

the first interview. MIS 1 and MIS 5 interviews are always conducted by

personal visit.

CATI Panel Households in this type of panel are eligible for interview at one of the

centralized telephone facilities. Not all households in the panel will be

interviewed by CATI. To be interviewed by CATI, a respondent must

have a telephone and speak English or Spanish. More important, during

the personal visit interviews (usually MIS 1 and MIS 5) the household

must agree to be interviewed in subsequent months by telephone. If not,

the household's subsequent interviews will be completed by a field

representative, either by personal visit or by telephone. Generally, if the

household has not been interviewed from a centralized telephone facility

by mid-week, then the interview is transferred to a field representative for

interviewing.

Three of the four hypotheses are tested using CATI panels.

NonCATI

 Panel All households in this type of panel are ineligible for CATI interviewing. Thus,

even if a household meets all of the basic requirements for CATI, the

interview will be completed by a field representative (decentralized

interviewing only).

Three of the four hypotheses are tested using NonCATI panels.

Automation An automated questionnaire is fully computerized. This means that an

interviewer does not have to decide which question should be asked next. 

That is, after entering a response to a question, the next question will

appear on the screen automatically. Both CAPI and CATI are fully

automated.

The analysis here centers on four possible effects: two centralized telephone

interviewing (CATI) effects and two new questionnaire effects. These are not the only

mode effects that could be present. For example, one would test for an automation
6

effect, but this was impossible, because a CAPI version of the old questionnaire was

never introduced into CPS.

A description of the hypotheses follows. More detailed descriptions are provided in

Bureau of the Census (1993). Following standard procedures, hypotheses are stated in

terms of no effect. The statistical testing is performed to determine if there is sufficient

evidence present to reject these stated hypotheses.

A. Hypothesis One: No centralized telephone interviewing effect

1. Description

Tests of these hypotheses are based on data from the PS which used the

new redesigned questionnaire. The same automated questionnaire was

used by both the PS field representatives (decentralized interviewing) and

by the PS CATI interviewers (centralized interviewing). 

With decentralized interviewing, most respondents are interviewed for

eight months by the same field representative, either by personal visit or

by telephone. This allows the respondent to develop a personal rapport

with the interviewer and may also result in both field representative and

respondent conditioning. With CATI, however, the first and fifth

interviews are conducted by a field representative, and the subsequent

three interviews are conducted from a centralized telephone facility. 

Thus, personal rapport can never be developed, nor can the interviewer be

conditioned to expect certain responses. Interviewers at the centralized

facilities have the benefit of closer supervision and interaction with other

interviewers, but, in general, they have considerably less experience than

their field counterparts. See Rothgeb (1994) for more detailed discussion

of the differences in interviewer and respondent rapport seen in

decentralized and centralized interviewing.

2. Experimental Design

The sample within the PS CATI-eligible areas was randomly split into two

representative panels: Panel A (PS CATI) and Panel B (PS NonCATI). 

Households in Panel A could (not must) be sent to CATI for interviewing,

but not all cases were completed using CATI. Households in Panel B

could not be sent to CATI.

Because both panels were from a non-probability group of sample areas,

these panel estimates were not nationally representative. PS CATIeligible areas were generally very urban and had different demographic

characteristics from overall national demographics. For example, 1990
7

census counts show that approximately 12 percent of the civilian, noninstitutional population aged 16 or older is black; approximately 8 1/2

percent of the population is Hispanic. The population covered by the PS

CATI-eligible area was approximately 13 percent black and 12 percent

Hispanic.

Data obtained from the first and fifth (personal visit) interviews, were

excluded from the panel estimates for testing this hypothesis. 

Approximate monthly sample sizes of persons 16+, civilian labor force

(CLF) participants, and unemployed persons for each panel are presented

below.

PS Panel Persons 16+ CLF participants Unemployed persons

PS CATI Panel A 7,800 5,180 420

PS NonCATI Panel B 950 630 50

3. Limitations

The composition of the split panels was the major limitation of this

analysis. When the sample was randomly split into two groups, only onetenth of the sample was assigned to the Panel B (NonCATI) group. This

resulted in extremely variable Panel B estimates.

The confounding caused by having a mix of data from both CATI and

non-CATI interviews in the Panel A estimates was a second limitation. 

Restricting the estimates to data from MIS 2-4 and 6-8 sample units

reduced this effect but did not eliminate it. As mentioned above, not all of

the CATI-eligible interviews were actually completed at a CATI facility: 

Some of the households in the CATI panel did not have telephones; some

did not have the English or Spanish skills necessary for telephone

interviewing; and some could not be reached by telephone from the

centralized facility. Non-CATI cases were included in the CATI panel

estimates to avoid biasing the sample.

B. Hypothesis Two: No centralized and computer-assisted telephone interviewing

effect
8

1. Description

Tests of these hypotheses are based on data from the CPS. The PAPI

questionnaire was used by the CPS field representatives (decentralized

interviewing). A computerized version of the old questionnaire with a

slightly modified wording of the lead-in labor force question was used by

the CPS CATI interviewers (centralized and computer-assisted

interviewing). In contrast to the hypothesis described above, this

centralized telephone interviewing effect was combined with computerassisted interviewing, because the old questionnaire did not have a

computerized version outside of the CATI environment. This study was a

continuation of the study presented in Shoemaker (1993).

2. Experimental Design

The sample within the CPS CATI-eligible areas was randomly split into

two representative panels: CATI-eligible (Panel C) or nonCATI (Panel

D). The number of households in Panel C each month increased over

time. Thus, the composition of both panels changed on a monthly basis. 

The only areas included in this study were those that had sample in both

Panel C and Panel D.

As with the first hypothesis, these panel estimates were not nationally

representative, because they used data from a non-random group of sample

areas. Moreover, the geographic areas represented in this study are

different than those included in hypothesis one. The population covered by

the CPS CATI-eligible sample areas was approximately 12 percent black

and 11 percent Hispanic. 

Data obtained from the first and fifth interviews were excluded from the

panel estimates for testing this hypothesis. Approximate monthly sample

sizes of persons 16+, CLF participants, and unemployed persons for each

panel are presented below.

CPS Panel Persons 16+ CLF participants Unemployed persons

CPS CATI Panel C 14,580 9,800 730

CPS NonCATI Panel D 16,250 10,840 740

3. Limitations

The confounding caused by the mix of CATI and non-CATI interviews in

the CATI Panel C estimates was also present in these tests. Additionally,
9

the Panel C interviews which were not completed at a CATI facility were

conducted with a slightly different wording of the lead-in labor force

question.

C. Hypothesis Three: No new questionnaire, given centralized telephone

interviewing effect

1. Description

Tests of these hypotheses are based on data from both the PS and CPS

CATI-eligible panels. From October 1992 through December 1993, there

were two fully automated questionnaires used in the CATI facilities -- the

old questionnaire and the redesigned questionnaire. The fully automated

redesigned questionnaire was used by both the PS interviewers at the

centralized telephone facilities and by the PS field representatives. The

automated version of the old CPS questionnaire described in hypothesis

two was used by the CPS interviewers at the centralized telephone

facilities, whereas the PAPI questionnaire was used by the CPS field

representatives.

2. Experimental Design

The sample for this analysis is confined to an even smaller subnational

area: the intersection of the PS CATI-eligible areas and the CPS CATIeligible areas (prior to January 1994). This area is referred to as the

"Common CATI-eligible area." Again, these panel estimates were not

nationally representative. The population covered by the Common CATIeligible area was approximately 13 percent black and 13 percent Hispanic.

Estimates from the PS CATI-eligible Panel in the Common CATI-eligible

area (Panel E) were compared to estimates from the CPS CATI-eligible

Panel in the Common CATI-eligible area (Panel F).

Data obtained from the first and fifth interviews were excluded from the

panel estimates for testing this hypothesis. Approximate monthly sample

sizes of persons 16+, CLF participants, and unemployed persons for each

panel are presented below.

Common PSU Panel Persons 16+ CLF participants Unemployed persons

PS CATI Panel E 5,790 3,910 320

CPS CATI Panel F 13,130 8,540 680

3. Limitations
10

There were two primary confounding "nuisance" effects:

• Both panels included interviews that were not completed at a

CATI facility, so some decentralized interviews were unavoidably

included in the panel estimates;

• Moreover, the nonCATI PS cases were completed with CAPI, and

the nonCATI cases in CPS were completed with PAPI.

D. Hypothesis Four: No new questionnaire designed for computer-assisted

interviewing effect

1. Description

Tests of these hypotheses are based on data from the PS and the CPS in a

decentralized environment. The redesigned fully automated questionnaire

was used by the PS field representatives. The old PAPI questionnaire was

used by the CPS field representatives. The automated CPS questionnaire

exists only as a computerized instrument, and the automated design allows

complicated skip patterns and dependent interviewing. The rewording of

many of the questions (especially the lead-in) is tied to the automation. 

The old paper questionnaire had far less complicated skip patterns and did

not allow for dependent interviewing (Rothgeb, 1994).

2. Experimental Design

The sample for this analysis was a quarter of the total national sample for

both the PS and the CPS. NonCATI Panel G consisted of all MIS 1 and

MIS 5 PS cases. NonCATI Panel H consisted of all MIS 1 and MIS 5

CPS cases. Thus, these estimates were nationally representative. 

Moreover, this test was free from any confounding effect that would be

directly due to CATI, since first and fifth month interviews were never

conducted from a centralized telephone facility.

Approximate monthly sample sizes of persons 16+, CLF participants, and

unemployed persons for each panel are presented below.

Panel Persons 16+ CLF participants Unemployed persons

PS NonCATI Panel G 6,300 4,220 320

CPS NonCATI Panel H 31,500 21,110 1,580

3. Limitations
11

The reliability of the test is still not comparable to tests which use all eight

interviews' data, because it is only one-fourth the sample size. 

Additionally, the expected value of an estimate varies depending on the

number of months it has been in sample.

IV. Estimation and Variance Estimates for Mode Effects Analysis

Because the estimates for testing the two centralized telephone interviewing hypotheses

and the questionnaire (given centralized telephone interviewing) hypotheses were

restricted to sample in CATI Eligible PSUs, they were not nationally representative. 

Each panel estimate was an "unbiased" estimate. That is, the weights used to produce the

estimates were strictly a function of the probability of selection (baseweight x adjustment

factor for field subsampling x an adjustment factor for the probability of inclusion in a

split panel). Variances were computed with localized generalized variance functions

(LGVFs). For more details on variance and estimation, see Fisher et al (1993). 

Estimates based on final weights were used in testing for a new questionnaire designed

for computer-assisted interviewing effect. Final weights were derived from a poststratification to 1980 census-based population estimates. See Fisher et al (1993) for a

detailed explanation of the post-stratification used. Estimates using final weights are

considerably more precise than the unbiased estimates and have a different expected

value.

V. Testing Methodology

A. Two Sample T-Tests and Bonferroni Multiple Comparisons Tests

The tests discussed in the results section are two-sample t-tests, with a 90 percent

confidence level. These confidence levels reflect the individual test; they do not

express confidence in the joint family of tests. No attempts have been made to

estimate the power of the tests.

Bonferroni multiple comparisons tests were performed for each "significant"

individual test. For the purposes of this analysis, each hypothesis is considered to

have four "independent" statistics for race/sex breakdown: white men, white

women, black men, and black women. Unemployment rate and employment-topopulation ratio comparisons are considered independent (but not CLF

participation rate, since it is a linear combination of the two). To have joint 90

percent confidence, at least one of the p-values must be smaller than 0.025. The

multiple comparisons tests are more appropriate for drawing general conclusions

about a mode effect, even though they cannot pinpoint individual differences. 

Bureau of the Census (1994) provides further details on these tests.
12

B. Implied Comparison Tests

Two types of implied comparison tests were performed for each rejected

hypothesis test. These tests are two-sided t-tests, at the 90 percent confidence

level. Unless directly noted in the text, the implied comparison tests' results are

not significant.

To understand the implied comparison tests, define

The first set of implied comparison tests are used for the hypothesis that the mode effect in a

subpopulation is the same as seen in the parent population. For example, the test of the

implied comparison between total and total black unemployment rate is

These implied comparison tests are performed for:

Subpopulation Tested Against

Total male Total

Total female Total

Total black Total

Black male Total black

Black female Total black

Total white Total

White male Total white

White female Total white

T

NT

T

NT

T

NT

TUER = Treatment panel total unemployment rate

TUER = Non - treatment panel total unemployment rate

BUER = Treatment panel black unemployment rate

BUER = Non - treatment panel black unemployment Rate

WUER = Treatment panel white unemployment rate

WUER = Non - treatment panel white unemployment rate

Install Equation Editor and doubleclick here to view equation.
13

The second set of implied comparison tests are used for the hypothesis that the

mode effect has a differential effect on two mutually exclusive subpopulations.

For example, the test of the implied comparison between black and white

unemployment rates is

These implied comparison tests are performed for:

Subpopulation Tested Against

Total male Total female

Total black Total white

Black male White male, black female

Black female White female, black male

White male Black male, white female

White female Black female, white male

Install Equation Editor and doubleclick here to view equation.
14

Results

This section presents highlights from the four sets of hypothesis tests for major labor force

characteristics. The significance level for an individual test is 0.10.

All estimates presented are percentages. Estimated differences may not agree with the table's

estimates because of rounding.

I. Centralized telephone interviewing effect

Tests are based on estimated differences between the PS CATI Panel A and the PS

NonCATI Panel B. A significant positive difference implies that centralized telephone

interviewing results in a larger rate; a significant negative difference implies that

centralized telephone interviewing results in a smaller rate.

Results from the individual two-sample t-tests are provided in tables A-1 through A-3. 

These statistics are computed from an average of 14 months of data (October 1992

through December 1993 PS, excluding the March 1993 data because one of the

centralized telephone facilities was partially shut down during interview week because of

the blizzard of 1993). Results from the Bonferroni tests are provided in Table A-4.

None of the implied comparison tests yielded significant results. That is, there is no

evidence, based on these panel estimates, that centralized telephone interviewing

differently affects the unemployment rate, employment-to-population ratio, or CLF

participation rate by race or sex.

By labor force characteristic, these tests show:

• Unemployment rate. The estimated difference for total unemployment rate

(1.10 percentage points) was marginally significant (p-value=0.11). Neither the

estimated difference for total male unemployment rate nor the estimated

difference for total female unemployment rate were significant.

These tests did not provide any evidence of a centralization effect for white

unemployment rate (total, male, female).

A 90 percent confidence interval on the estimated difference for total black

unemployment rate (3.50 percentage points) is given by (0.01, 6.99), indicating

that centralized telephone interviewing yields a larger unemployment rate for

blacks. The same conclusion can be drawn for black male unemployment rate

(3.69 percentage points, with a p-value=0.07), although not for black female

unemployment rate (p-value=0.21). The joint comparison for unemployment rate

does not reinforce this conclusion. The smallest p-value contained in Table A-4
15

for the unemployment rate is 0.07, and therefore this joint confidence interval

does not contain at least one p-value less than 0.025, the minimum necessary for

90 percent joint confidence. 

• Employment-to-population ratio. These tests did not provide any evidence of a

centralized telephone interviewing effect for employment to population ratio for

any race or sex breakdown.

• CLF participation rate. These tests did not provide any evidence of a

centralized telephone interviewing effect for CLF participation rate for any race

or sex breakdown.

II. Centralized telephone and computer-assisted interviewing effect

Tests are based on estimated differences between the CPS CATI Panel C and the CPS

NonCATI Panel D. A significant positive difference implies that centralized telephone

and computer-assisted interviewing results in a larger rate; a significant negative

difference implies that centralized telephone and computer-assisted interviewing results

in a smaller rate.

Results from the individual two-sample t-tests are provided in tables B-1 through B-3. 

These statistics are computed from an average of fourteen months of data (October 1992

through December 1993 CPS data, excluding the March 1993 data). Results from the

Bonferroni tests are provided in table B-4.

None of the implied comparison tests yielded significant results. That is, there is no

evidence, based on these panel estimates, that centralized telephone and computerassisted interviewing differently affects the unemployment rate, employment-topopulation ratio, or CLF participation rate by race or sex.

By labor force characteristic, these tests show:

• Unemployment rate. The 90 percent confidence interval on the estimated

difference for total unemployment rate (1.00 percentage point) is given by (0.59,

1.41), indicating that centralized telephone and computer-assisted interviewing

yields a larger unemployment rate. The same conclusion can be drawn for total

female unemployment rate (0.96 percentage point, with a p-value=0.00) and for

total male unemployment rate (1.04 percentage points, with a p-value=0.00). 

These conclusions all hold at the 99 percent confidence level.

Moreover, the 90 percent confidence interval on the estimated difference for total

white unemployment rate (1.07 percentage points) is given by (0.76, 1.38),

indicating that centralized telephone interviewing and computer-assisted

interviewing yields a larger unemployment rate for whites. The same conclusion
16

can be drawn for white female unemployment rate (1.03 percentage points, with a

p-value=0.00) and for white male unemployment rate (1.11 percentage points,

with a p-value=0.00). These conclusions all hold at the 99 percent confidence

level.

Finally, a 90 percent confidence interval on the estimated difference for black

female unemployment rate (1.86 percentage points) is given by (0.35, 3.37),

indicating that centralized telephone and computer-assisted interviewing yields a

larger unemployment rate for black females. Neither the estimated difference for

total black unemployment rate nor the estimated difference for black male

unemployment rate were significant.

The above results are reinforced by the joint comparison for unemployment rate

presented in table B-4, because two of the four p-values contained in the

Bonferroni confidence interval are smaller than 0.025.

• Employment-to-population ratio. These tests did not provide any evidence of a

centralized telephone and computer-assisted interviewing effect for employment

to population ratio for any race or sex breakdown.

• CLF participation rate. These tests did not provide any evidence of a

centralized telephone and computer-assisted interviewing effect for CLF

participation rate for any race or sex breakdown.

III. New questionnaire, given centralized telephone interviewing effect

Tests are based on estimated differences between the PS CATI Panel E and the CPS

CATI Panel F. A significant positive difference implies that the new questionnaire,

given centralized telephone interviewing results in a larger rate; a significant negative

difference implies that the new questionnaire, given centralized telephone interviewing

results in a smaller rate.

Results from the individual two-sample t-tests are provided in tables C-1 through C-3. 

These statistics are computed from an average of 14 months of data (October 1992

through December 1993 Common PSU data, excluding March 1993 data). The

Bonferroni tests are provided in table C-4.

None of the implied comparison tests yielded significant results. That is, there is no

evidence, based on these panel estimates, that the new questionnaire, given centralized

telephone interviewing differently affects the unemployment rate, employment-topopulation ratio, or CLF participation rate by race or sex.

By labor force characteristic, these tests show:
17

• Unemployment rate. These tests did not provide any evidence of a

questionnaire effect for either total or total male unemployment rate, although the

test for total female unemployment rate was marginally significant (0.52

percentage point, with a p-value=0.11).

 These tests did not provide any evidence of a questionnaire effect for white

unemployment rate (total, male, female).

A 90 percent confidence interval on the estimated difference for total black

unemployment rate (1.38 percentage points) is given by (0.10, 2.66), indicating

that the new questionnaire yields a larger unemployment rate for total blacks (pvalue=0.07). The same conclusion can be drawn for black female unemployment

rate (1.96 percentage points, with a p-value=0.03), but not for black male

unemployment rate (p-value=0.31). These conclusions reinforce the

independently-drawn conclusions for these statistics using the full set of CPS and

PS data (Polivka, 1994). The joint comparison presented in table C-4 does not,

however, reinforce this conclusion of a questionnaire effect, because the smallest

p-value in the joint confidence interval is 0.03 and its unrounded value is greater

than 0.025.

• Employment-to-population ratio. These tests did not provide any evidence of a

questionnaire effect for employment to population ratio for any race or sex

breakdown.

• CLF participation rate. These tests did not provide any evidence of a

questionnaire effect for CLF participation rate for any race or sex breakdown.

IV. New questionnaire designed for computer-assisted interviewing effect

Tests are based on estimated differences between the PS NonCATI Panel G and the CPS

NonCATI Panel H. A significant positive difference implies that the new questionnaire

results in a larger rate; a significant negative difference implies that the new

questionnaire results in a smaller rate.

Results from the individual two-sample t-tests are provided in tables D-1 through D-3. 

These statistics are computed from an average of a year's (January 1993 through

December 1993) MIS 1 and MIS 5 CPS and PS data. These statistics include the March

1993 data.

Except where noted, none of the implied comparison tests yielded significant results. 

By labor force characteristic, these tests show:
18

• Unemployment rate. A 90 percent confidence interval on the estimated

difference for total unemployment rate (0.64 percentage points) is given by (0.33,

0.97), indicating that the new questionnaire designed for computer-assisted

interviewing yields a larger unemployment rate (p-value=0.00). The same

conclusion can be drawn for total male unemployment rate (0.52 percentage

points, with a p-value=0.01) and total female unemployment rate (0.79

percentage points, with a p-value=0.01). The relative increase for total female

unemployment rate between panels (1.12) is statistically

larger than the relative increase (1.07) for total male unemployment rate (pvalue=0.08).

Moreover, the 90 percent confidence interval on the estimated difference for total

white unemployment rate (0.49 percentage points) is given by (0.18, 0.80),

indicating the new questionnaire designed for computer-assisted interviewing

yields a larger unemployment rate for total whites (p-value=0.01). The same

conclusion can be drawn for white male unemployment rate (0.39 percentage

points, with a p-value of 0.04) and for white female unemployment rate (0.61

percentage points, with a p-value=0.00).

Finally, a 90 percent confidence interval on the estimated difference for total

black unemployment rate (1.63 percentage points) is given by (0.40, 2.86),

indicating that the new questionnaire designed for computer-assisted interviewing

yields a larger unemployment rate for total blacks. The same conclusions can be

drawn for black male unemployment rate (1.50 percentage points, with a pvalue=0.08) and for black female unemployment rate (1.83 percentage points,

with a p-value of 0.04).

The above results are reinforced by the joint comparison for unemployment rate

presented in table D-4 because one of the p-values contained in the Bonferroni

confidence interval is less than 0.025.

• Employment-to-population ratio. These tests did not provide any evidence of a

new questionnaire designed for computer-assisted interviewing effect for

employment to population ratio for any race or sex breakdown.

• CLF participation rate. These tests did not provide any evidence of a new

questionnaire designed for computer-assisted interviewing effect for either total

or total male CLF participation rate. However, a 90 percent confidence interval

on the estimated difference for females (1.11 percentage points) is given by (0.25,

1.97), indicating that the new questionnaire designed for computer-assisted

interviewing yields a larger CLF participation rate. The relative increase for total

female CLF participation rate between panels (1.02) is statistically larger than the

relative decrease (1.00) for total male CLF participation rate (p-value=0.02).
19

These tests did not provide any evidence of a new questionnaire designed for

computer-assisted interviewing effect for either total white or white male CLF

participation rate. However, a 90 percent confidence interval on the estimated

difference for white female (1.1 percentage points) is given by (0.24, 1.96),

indicating that the new questionnaire designed for computer-assisted interviewing

yields a larger CLF participation rate for white females. The relative increase for

white female CLF participation rate between panels (1.02) is statistically larger

than the corresponding relative decrease (1.00) for white male CLF participation

rate (p-value=0.00).

These tests did not provide any evidence of a new questionnaire designed for

computer-assisted interviewing effect for black CLF participation rate (total,

male, female).
20

Summary

This study presents strong evidence of a centralized telephone and computer-assisted

interviewing effect on unemployment rate. It also presents equally convincing evidence of a

new questionnaire designed for computer-assisted interviewing effect on unemployment rate. 

This study does not present as convincing evidence of either a centralized telephone interviewing

effect or a questionnaire effect using centralized telephone interviewing. However, we would

strongly advise against dismissing the possibility of a centralized telephone interviewing effect,

given the abundance of evidence of a centralized telephone and computer-assisted interviewing

effect presented here and in Shoemaker (1993).

For both the centralized telephone interviewing and computer-assisted interviewing effect and

the new questionnaire designed for computer-assisted interviewing effect, it is impossible to

separate the effect of computer-assisted interviewing from the respective combined effect. 

Furthermore, with the exception of the new questionnaire designed for computer-assisted

interviewing effect, the studied mode effects are confounded. Ergo, including the centralized

telephone and computer-assisted interviewing or the new questionnaire designed for computerassisted interviewing yields a larger unemployment rate. 
21

References

Bregger, John E. and Dippo, Cathryn S. (1993), Overhauling the Current Population

Survey: Why Is It Necessary to Change? The Monthly Labor Review, Bureau of Labor

Statistics, Washington, D.C.

Bureau of the Census (1993), "CPS Mode Effects--Overview of Analysis and Work Plans,"

Internal Memorandum to CPS Questionnaire Evaluation Steering Committee from CPS Mode

Effects Workgroup, June 3, 1993.

Bureau of the Census (1994), "CATI/CAPI Analysis: Individual and Joint Tests for Key Labor

Force Estimates for a 14-Month Analysis," Internal Memorandum to the Record from CPS

Mode Effects Workgroup, March 9, 1994.

Fisher, Robin; Robison, Edwin; Thompson, Jenny; and Welch, Michael (1993), Variance

Estimation in the CPS Overlap Test, Proceedings of the Section on Survey Research

Methods, American Statistical Association.

Kostanich, Donna L. and Cahoon, Lawrence S. (1994), Effect of Design Differences Between

the Parallel Survey and the New CPS, CPS Bridge Team Technical Report 3, Bureau of Labor

Statistics, Washington, D.C.

Polivka, Anne E. (1994), Comparisons of Labor Force Estimates for the Parallel Survey and the

Current CPS, Overlap Analysis Team Technical Report 1, Bureau of Labor Statistics,

Washington, D.C.

Rothgeb, Jennifer M. (1994), Revision to the CPS Questionnaire Effect on Data Quality, CPS

Overlap Analysis Team Technical Report 2, Bureau of the Census, Washington, D.C.

Shoemaker, Harland H. (1993), Results from the Current Population Survey CATI Phase-In

Project, Proceedings of the Section on Survey Research Methods, American Statistical

Association.
Table A-1

EFFECT OF CENTRALIZED TELEPHONE INTERVIEWING

 Unemployment Rate

(14 Month Average, 10/92 - 12/93 excluding 3/93)

PS CATI

Panel A

PS NonCATI

Panel B

Difference

A-B P-value

Total

 Men

 Women

 7.88

 7.94

 7.81

 6.79

 6.78

 6.80

1.10

1.17

1.01

0.11

0.21

0.14

White

 Men

 Women

 6.70

 6.91

 6.46

 6.29

 6.48

 6.07

0.41

0.43

0.39

 .40

 .59

 .52

Black

 Men

 Women

15.16

15.60

14.82

11.67

11.92

11.45

3.50

3.69

3.37

0.10*

0.07*

0.21

PS CATI Panel A = PS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with laptop

computers (CAPI).

PS NonCATI Panel B = PS sample that cannot be sent to CATI. All sample

interviewed with laptop computers (CAPI).

MIS 1 and MIS 5 not included in Panel A or Panel B.

*Differences significant at the 10% level.
Table A-2

EFFECT OF CENTRALIZED TELEPHONE INTERVIEWING

 Employment to Population Ratio

(14 Month Average, 10/92 - 12/93 excluding 3/93)

PS CATI

Panel A

PS NonCATI 

Panel B

Difference

A-B P-value

Total

 Men

 Women

61.89

69.57

55.17

62.10

70.33

54.95

-0.22

-0.77

 0.22

0.94

0.83

0.93

White

 Men

 Women

63.38

71.38

56.13

63.30

71.64

55.86

 0.08

-0.26

 0.27

0.98

0.95

0.93

Black

 Men

 Women

52.84

56.21

50.47

50.91

55.08

47.76

 1.92

 1.13

 2.71

0.88

0.93

0.84

PS CATI Panel A = PS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with laptop

computers (CAPI).

PS NonCATI Panel B = PS sample that cannot be sent to CATI. All sample

interviewed with laptop computers (CAPI).

MIS 1 and MIS 5 not included in Panel A or Panel B.
Table A-3

EFFECT OF CENTRALIZED TELEPHONE INTERVIEWING

 Civilian Labor Force Participation Rate

(14 Month Average, 10/92 - 12/93 excluding 3/93)

PS CATI

Panel A

PS NonCATI 

Panel B

Difference

A-B P-value

Total

 Men

 Women

67.18

75.57

59.84

66.62

75.45

58.96

0.56

0.12

0.89

0.85

0.97

0.74

White

 Men

 Women

67.93

76.68

60.01

67.55

76.61

59.47

0.38

0.07

0.54

0.92

0.99

0.87

Black

 Men

 Women

62.28

66.60

59.25

57.64

62.53

53.94

4.64

4.07

5.32

0.74

0.79

0.72

PS CATI Panel A = PS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with laptop

computers (CAPI).

PS NonCATI Panel B = PS sample that cannot be sent to CATI. All sample

interviewed with laptop computers (CAPI).

MIS 1 and MIS 5 not included in Panel A or Panel B.
Table A-4

Bonferroni Multiple Comparisons Tests for Centralized Telephone Interviewing Effect

(alpha=0.10)

Unemployment Rate Test:

Individual P-Values

White Male 0.59

White Female 0.52

Black Male 0.07

Black Female 0.21

Employment to Population Ratio Test:

Individual P-Values

White Male 0.95

White Female 0.93

Black Male 0.93

Black Female 0.84

(No significant results)
Table B-1

EFFECT OF CENTRALIZED TELEPHONE AND COMPUTER-ASSISTED

INTERVIEWING

 Unemployment Rate

(14 Month Average, 10/92 - 12/93 excluding 3/93)

CPS CATI

Panel C

CPS NonCATI

Panel D

Difference

C-D P-value

Total

 Men

 Women

 7.55

 7.78

 7.28

 6.54

 6.81

 6.24

1.00

0.96

1.04

0.00*

0.00*

0.00*

White

 Men

 Women

 6.64

 6.91

 6.32

 5.57

 5.80

 5.30

1.07

1.11

1.03

0.00*

0.00*

0.00*

Black

 Men

 Women

13.42

14.10

12.84

12.30

13.90

10.98

1.12

0.20

1.86

0.18

0.76

0.04*

CPS CATI Panel C = CPS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with paper and

pencil (PAPI).

CPS NonCATI Panel D = CPS sample that cannot be sent to CATI. All sample

interviewed with paper and pencil (PAPI).

MIS 1 and MIS 5 not included in Panel C or Panel D.

*Differences significant at the 10% level.
Table B-2

EFFECT OF CENTRALIZED TELEPHONE AND COMPUTER-ASSISTED

INTERVIEWING

 Employment to Population Ratio

(14 Month Average, 10/92 - 12/93 excluding 3/93)

CPS CATI

Panel C

CPS NonCATI

Panel D

Difference

C-D P-value

Total

 Men

 Women

62.78

71.42

55.16

61.94

70.48

54.51

0.84

0.95

0.65

0.45

0.58

0.57

White

 Men

 Women

64.09

73.12

55.94

63.32

72.22

55.32

0.77

0.91

0.62

0.58

0.65

0.65

Black

 Men

 Women

55.54

60.22

52.12

53.47

58.30

50.17

2.07

1.92

1.95

0.61

0.70

0.67

CPS CATI Panel C = CPS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with paper and

pencil (PAPI).

CPS NonCATI Panel D = CPS sample that cannot be sent to CATI. All sample

interviewed with paper and pencil (PAPI).

MIS 1 and MIS 5 not included in Panel C or Panel D.
Table B-3

EFFECT OF CENTRALIZED TELEPHONE AND COMPUTER-ASSISTED

INTERVIEWING

 Civilian Labor Force Participation Rate

(14 Month Average, 10/92 - 12/93 excluding 3/93)

CPS CATI

Panel C

CPS NonCATI

Panel D

Difference

C-D P-value

Total

 Men

 Women

67.91

77.45

59.49

66.28

75.63

58.13

1.63

1.82

1.36

0.16

0.31

0.27

White

 Men

 Women

68.65

78.55

59.72

67.05

76.66

58.42

1.59

1.89

1.30

0.27

0.37

0.36

Black

 Men

 Women

64.16

70.11

59.80

60.97

67.71

56.36

3.19

2.40

3.44

0.46

0.67

0.48

CPS CATI Panel C = CPS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with paper and

pencil (PAPI).

CPS NonCATI Panel D = CPS sample that cannot be sent to CATI. All sample

interviewed with paper and pencil (PAPI).

MIS 1 and MIS 5 not included in Panel C or Panel D.
Table B-4

Bonferroni Multiple Comparisons Tests for Centralized Telephone and Computer-Assisted

Interviewing Effect

alpha=0.10

Unemployment Rate Test:

Individual P-Values

White Male 0.00

White Female 0.00

Black Male 0.76 

Black Female 0.04

This test is significant at 10% joint confidence level

Employment to Population Ratio Test:

Individual P-Values

White Male 0.65

White Female 0.65

Black Male 0.70

Black Female 0.67

This test is not significant.
Table C-1

EFFECT OF QUESTIONNAIRE

Unemployment Rate

(14 Month Average, 10/92 - 12/93 excluding 3/93)

PS CATI

Panel E

CPS CATI

Panel F

Difference

E-F P-value

Total

 Men

 Women

8.12

8.07

8.16

7.87

8.06

7.65

 0.25

 0.01

 0.52

0.44

0.97

0.11

White

 Men

 Women

6.91

7.03

6.76

6.92

7.12

6.68

-0.02

-0.09

 0.08

0.95

0.80

0.80

Black

 Men

 Women

15.03

15.16

14.92

13.65

14.46

12.96

 1.38

 0.70

 1.96

0.07*

0.31

0.03*

PS CATI Panel E = PS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with laptop

computers (CAPI).

CPS CATI Panel F = CPS sample that can be sent to CATI but includes nonCATI

sample. NonCATI sample interviewed with paper and

pencil (PAPI).

MIS 1 and MIS 5 not included in Panel E or Panel F.

*Differences significant at the 10% level.
Table C-2

EFFECT OF QUESTIONNAIRE

Employment to Population Ratio

(14 Month Average, 10/92 - 12/93 excluding 3/93)

PS CATI

Panel E

CPS CATI

Panel F

Difference

E-F P-value

Total

 Men

 Women

61.80

69.59

54.95

61.54

70.40

53.80

 0.26

-0.81

 1.15

0.84

0.65

0.35

White

 Men

 Women

63.36

71.48

55.94

62.72

72.10

54.36

 0.64

-0.63

 1.57

0.72

0.78

0.30

Black

 Men

 Women

53.16

56.83

50.57

55.01

59.59

51.68

-1.85

-2.76

-1.11

0.72

0.63

0.84

PS CATI Panel E = PS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with laptop

computers (CAPI).

CPS CATI Panel F = CPS sample that can be sent to CATI but includes nonCATI

sample. NonCATI sample interviewed with paper and

pencil (PAPI).

MIS 1 and MIS 5 not included in Panel E or Panel F.
Table C-3

EFFECT OF QUESTIONNAIRE

Civilian Labor Force Participation Rate

(14 Month Average, 10/92 - 12/93 excluding 3/93)

PS CATI

Panel E

CPS CATI

Panel F

Difference

E-F P-value

Total

 Men

 Women

67.26

75.70

59.83

66.79

76.57

58.26

 0.47

-0.87

 1.58

0.73

0.64

0.23

White

 Men

 Women

68.06

76.88

59.99

67.39

77.63

58.26

 0.67

-0.75

 1.73

0.71

0.75

0.28

Black

 Men

 Women

62.56

66.98

59.45

63.71

69.67

59.38

-1.15

-2.68

 0.07

0.84

0.68

0.99

PS CATI Panel E = PS sample that can be sent to CATI, but includes nonCATI

sample. NonCATI sample interviewed with laptop

computers (CAPI).

CPS CATI Panel F = CPS sample that can be sent to CATI but includes nonCATI

sample. NonCATI sample interviewed with paper and

pencil (PAPI).

MIS 1 and MIS 5 not included in Panel E or Panel F.
Table C-4

Bonferroni Multiple Comparisons Tests for Questionnaire Effect

(alpha=0.10)

Unemployment Rate Test:

Individual P-Values

White Male 0.80

White Female 0.80

Black Male 0.31

Black Female 0.03

Employment to Population Ratio Test:

Individual P-Values

White Male 0.78

White Female 0.30

Black Male 0.63

Black Female 0.84

(No significant results).
Table D-1

EFFECT OF NEW QUESTIONNAIRE DESIGNED FOR COMPUTER-ASSISTED

INTERVIEWING

Unemployment Rate

(12 Month Average, 1/93 - 12/93)

PS NonCATI

Panel G

CPS NonCATI

Panel H

Difference

G-H P-value

Total

 Men

 Women

7.48

7.57

7.36

6.83

7.06

6.57

0.64

0.52

0.79

0.00*

0.01*

0.01*

White

 Men

 Women

6.43

6.59

6.23

5.94

6.20

5.62

0.49

0.39

0.61

0.01*

0.04*

0.00*

Black

 Men

 Women

15.06

15.52

14.67

13.43

14.02

12.84

1.63

1.50

1.83

0.03*

0.08*

0.04*

PS NonCATI Panel G = All MIS 1 and MIS 5 PS sample. This sample is never sent

to CATI. Sample is interviewed with laptop computers

(CAPI).

CPS NonCATI Panel H = All MIS 1 and MIS 5 CPS sample. This sample is never

sent to CATI. Sample is interviewed with paper and pencil

(PAPI).

*Differences significant at the 10% level.
Table D-2

EFFECT OF NEW QUESTIONNAIRE DESIGNED FOR COMPUTER-ASSISTED

INTERVIEWING

Employment to Population Ratio

(12 Month Average, 1/93 - 12/93)

PS NonCATI

Panel G

CPS NonCATI

Panel H

Difference

G-H P-value

Total

 Men

 Women

61.98

69.59

55.00

61.98

70.22

54.43

-0.01

-0.63

 0.56

1.00

0.14

0.28

White

 Men

 Women

63.11

71.06

55.74

63.08

71.68

55.06

 0.03

-0.62

 0.69

0.94

0.14

0.19

Black

 Men

 Women

53.92

58.20

50.40

54.62

59.20

50.86

-0.70

-1.00

-0.46

0.56

0.36

0.66

PS NonCATI Panel G = All MIS 1 and MIS 5 PS sample. This sample is never sent

to CATI. Sample is interviewed with laptop computers

(CAPI).

CPS NonCATI Panel H = All MIS 1 and MIS 5 CPS sample. This sample is never

sent to CATI. Sample is interviewed with paper and pencil

(PAPI).
Table D-3

EFFECT OF NEW QUESTIONNAIRE DESIGNED FOR COMPUTER-ASSISTED

INTERVIEWING

Civilian Labor Force Participation Rate

(12 Month Average, 1/93 - 12/93)

PS NonCATI

Panel G

CPS NonCATI

Panel H

Difference

G-H P-value

Total

 Men

 Women

66.99

75.29

59.37

66.53

75.55

58.26

 0.45

-0.26

 1.11

0.24

0.51

 0.03*

White

 Men

 Women

67.46

76.07

59.44

67.06

76.42

58.34

 0.40

-0.35

 1.11

0.29

0.37

 0.04*

Black

 Men

 Women

63.46

68.85

59.02

63.09

68.85

58.35

0.37

0.01

0.65

0.78

1.00

0.55

PS NonCATI Panel G = All MIS 1 and MIS 5 PS sample. This sample is never sent

to CATI. Sample is interviewed with laptop computers

(CAPI).

CPS NonCATI Panel H = All MIS 1 and MIS 5 CPS sample. This sample is never

sent to CATI. Sample is interviewed with paper and pencil

(PAPI).

*Differences significant at the 10% level.
Table D-4

Bonferroni Multiple Comparisons Tests for Questionnaire Designed for Computer-Assisted

Interviewing Effect

(alpha=.10)

Unemployment Rate Test:

Individual P-Values

White Male 0.04

White Female 0.00

Black Male 0.08

Black Female 0.04

This test is significant at the 10% joint confidence level.

Employment to Population Ratio Test:

Individual P-Values

White Male 0.14

White Female 0.19

Black Male 0.36

Black Female 0.66

(No significant results)