URBAN INSTITUTE

Justice Policy Center

Christine Lindquist

Kristine Fahrney

RTI International

Presented at the

NADCP 13th Annual Training 

Conference, 

Washington, DC

June 14, 2007

Data Collection for the MADCE Longitudinal 

Offender Survey

The views expressed are those of the authors and should not be 

attributed to The Urban Institute, its trustees, or its funders.
URBAN INSTITUTE

Justice Policy Center

Overview of Data Collection

 Eligible sample members identified on a 

rolling basis from participating sites

– Almost all sites required a “consent for 

release of information” procedure

• Loss at this stage impacted response rates and 

introduces selection bias

– Sample identification required extensive 

involvement from UI, RTI, and CCI “court 

liaisons” 
URBAN INSTITUTE

Justice Policy Center

Overview of Data Collection (cont’d)

 Eligible sample members assigned to RTI 

field interviewers and approached for 

participation

– Mailed lead letters and brochures prior to inperson contact

 Baseline interview conducted up to 6 weeks 

post program enrollment

 All baseline participants approached for 

follow-up interviews at 6- and 18-months 

post-baseline
URBAN INSTITUTE

Justice Policy Center

Overview of Data Collection (cont’d)

 Interviews done via computer assisted 

personal interviewing (CAPI)

– All interviews were conducted by trained RTI 

interviewers

• The same FI follows the respondent for all waves

 Interviews conducted in respondents’ homes, 

jails, prisons, and treatment facilities

– Facility access typically negotiated by UI, RTI, and 

CCI court liaisons
URBAN INSTITUTE

Justice Policy Center

Overview of Data Collection (cont’d)

 Interviews lasts 1.5 to 2 hours

 Interview content: 

– Demographics, family, supervision status, 

drug court participation, treatment 

motivation, violation, sanctions, rewards, 

substance use, criminal behavior, mental 

health, treatment and other support 

services, attitudes and perceptions

– Many behaviors “calendared”
URBAN INSTITUTE

Justice Policy Center

Overview of Data Collection (cont’d)

 During the interview, consent is also obtained 

for retrieving non-public administrative data

 Oral fluid drug test conducted at final 

interview (for non-incarcerated respondents)

 Developed a PAPI version of instrument for 

maximum security facilities and NY DOC
URBAN INSTITUTE

Justice Policy Center

Overview of Data Collection (cont’d)

 Interview compensation structure:

– Baseline: $35

– 6 month f/u: $40

– 18 month f/u: $50

• Oral fluids swab: $15

– Bonus for completing all 3 waves: $25

– Bonus for calling ahead to set up interview: 

$5
URBAN INSTITUTE

Justice Policy Center

Overview of Data Collection (cont’d)

 Mid-wave contact between interviews

 For each interview wave, a random 10% 

of completed interviews are verified by 

telephone recontact

– Designed to detect interview falsification or 

deviations from study protocols

– No problems have been identified through 

verification 
URBAN INSTITUTE

Justice Policy Center

Data Collection Schedule

Interview 

Wave

2005 2006 2007

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Baseline 

6 Month f/u

18 Month f/u
URBAN INSTITUTE

Justice Policy Center

Data Collection Status

Sample Sizes by Interview Wave

1791

1542

646

0

500

1000

1500

2000

Baseline 6 Month 18 Month

Response Rates by Interview Wave

72%

85% 88%

0%

20%

40%

60%

80%

100%

Baseline 6 Month 18 Month
URBAN INSTITUTE

Justice Policy Center

Lessons Learned from Data 

Collection

 The Obvious: need for a well-designed study 

& collaborative relationship between courts 

and evaluation team

 The not-so-obvious:

– Staffing the “Right” Field Team

– Training the Field Team

– Developing Appropriate Study Protocols

– Effectively Recruiting Respondents

– Successfully Retaining Respondents
URBAN INSTITUTE

Justice Policy Center

Staffing the “Right” Field Team

 Importance of hiring professional 

interviewers that are independent of the 

criminal justice system

– FI credibility essential for subject 

recruitment

– FI availability and flexibility essential for 

high response and retention rates

– FI supervision and performance monitoring 

essential for data quality
URBAN INSTITUTE

Justice Policy Center

 Importance of FI training on key topics

– Human subjects protection

– Dealing with distressed respondents, 

intoxicated respondents, etc.

– Interviewing in correctional facilities (FI 

safety, protecting R’s rights, sexual 

harassment, etc.)

– Recruitment strategies

– Tracing/locating techniques

Training the Field Team
URBAN INSTITUTE

Justice Policy Center

Developing Appropriate Study 

Protocols

 Protocols for interviewing in jails/prisons

– Ensure that access is appropriately negotiated 

and documented

– Ensure that details about laptops, incentives, and 

privacy of interview room are agreed upon

– Reveal as little as possible about the nature of the 

study to facility staff

– Do not reveal R’s decision to do the current 

interview (or behavior during the interview)
URBAN INSTITUTE

Justice Policy Center

Developing Appropriate Study 

Protocols (cont’d)

 Protocols/strategies for interviewing in 

treatment facilities

– Understand that staff cannot confirm that a 

respondent is in the facility and use hypotheticals

•“If [name] is in your facility, could you please pass along 

this information?”

– Reveal as little as possible about the nature of the 

study to facility staff

– Note that messages are frequently not passed 

along and showing up in person is essential

– Try to develop a variety of contacts at the facility 

and use this to one’s advantage
URBAN INSTITUTE

Justice Policy Center

– For treatment facilities affiliated with a drug court, 

try to involve drug court staff in negotiating FI 

access

• Best to do this early on in the project

– Take advantage of holidays or temporary leaves, 

in which the respondent might return home for a 

few days (allowing for a home interview)

– On the sneaky side:

• If the FI does get access to the floor, they should jot 

down the floor phone number so that next time they can 

call the # directly!

Developing Appropriate Study 

Protocols (cont’d)
URBAN INSTITUTE

Justice Policy Center

Developing Appropriate Study 

Protocols (cont’d)

 Protocols for handing distressed respondents

– Separate protocols for interviews in correctional 

facilities and community

– Threat to self vs. threat to others

 Protocols for witnessing child abuse/neglect

– Requires familiarity with state mandatory reporting 

laws
URBAN INSTITUTE

Justice Policy Center

Effectively Recruiting Participants

 From FI debriefing:

– emphasize that you are not part of the 

criminal justice system

– show respondents that you care about 

them as individuals

– be flexible

– be personable and charming

– Keep commitments
URBAN INSTITUTE

Justice Policy Center

Successfully Retaining Participants

 Challenges to locating this population

– Very transient population

– Unstable relationships

– Some traditional tracing resources are not 

effective

• Credit bureau data bases

• Talking to current/former neighbors

• Talking to family members

 Our mid-wave contact and incentive for the 

respondent to call us are effective
URBAN INSTITUTE

Justice Policy Center

Successfully Retaining Participants 

(cont’d)

 Essential to plan ahead for the next interview

– Modify traditional question: “Who are 3 people 

who will always know where you are?”

• FIs obtain information on people who will know where 

they are if they are using and people who will know 

where they are if they are clean

• For respondents who are fathers, FIs try to always obtain 

contact information for child’s mother

• FIs ask respondents to tell the contact people that they 

will be contacting them and that it is okay to disclose 

his/her whereabouts to them
URBAN INSTITUTE

Justice Policy Center

Successfully Retaining Participants 

(cont’d)

• From one wave to the next, it may be 

necessary for FIs to (tactfully) press the 

respondent for new contact people

– Find out where respondent spends time 

when he/she is using and when he/she is 

clean

– Find out about any other names that the 

respondent uses
URBAN INSTITUTE

Justice Policy Center

Successfully Retaining Participants 

(cont’d)

 Try to develop rapport with family and 

friends of respondent

– Families are very important gatekeepers

– Important to demonstrate persistence and 

knowledge about the respondent

– Important for FI to convey that she/he is 

not a part of criminal justice system

– Emphasize monetary incentive

– Charm factor important
URBAN INSTITUTE

Justice Policy Center

Successfully Retaining Participants 

(cont’d)

 Use public records

– Search the court dockets for other court 

appearances

– Conduct daily inmate searches in prisons and jails

– Obtain arrest records (may contain useful 

information about location of crime, codefendants, etc.)

– For R’s out on bail, find out who bailed them out

– Birth certificates are public records that might 

contain locating information (for new parents)
URBAN INSTITUTE

Justice Policy Center

Successfully Retaining Participants 

(cont’d)

 For locating drug court participants, FIs 

attend regularly scheduled drug court 

meetings, drug court graduations, and other 

drug court events

 Some tracing strategies too risky to employ 

with criminal justice populations:

– Contacting probation officers or case managers 

about unlocatable respondents may alert them 

that the individual is not where he/she is supposed 

to be, which could trigger a violation 
URBAN INSTITUTE

Justice Policy Center

Summary

 Key lessons that may be useful for other studies

– Staff the field team professionally (monitor performance, 

strive for FI continuity) 

– Train effectively

– Develop appropriate study protocols for the numerous 

situations that the field team will need to be prepared for

• Consideration of human subjects protection very important 

when working with this population

– Use effective recruitment techniques

– Employ a variety of effective retention techniques

• Remember that much of effective locating is dependent on the 

information obtained by the FI at the previous interview(s), so 

planning ahead for the next interview essential

• FI rapport with family, friends, and criminal justice/treatment 

providers is also key