The author(s) shown below used Federal funding provided by the U.S. 

Department of Justice to prepare the following resource: 

Document Title: Ten-Year Follow-Up on the RCT Study of 

Mentoring in the Learning Environment 

(SMILE): Effects of the Communities in 

Schools Mentoring Program on Crime and 

Educational Persistence

Author(s): Michael J. Karcher, Ed.D., Ph.D.

Document Number: 254619

Date Received: April 2020

Award Number: 2013-JU-FX-0008

This resource has not been published by the U.S. Department of 

Justice. This resource is being made publically available through the 

Office of Justice Programs’ National Criminal Justice Reference 

Service. 

Opinions or points of view expressed are those of the author(s) and 

do not necessarily reflect the official position or policies of the U.S. 

Department of Justice.
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 1

Final Technical Report 

Ten-Year Follow-Up on the RCT Study of Mentoring in the Learning Environment (SMILE): Effects 

of the Communities in Schools Mentoring Program on Crime and Educational Persistence 

The U.S. Department of Justice, Office of Justice Programs 

Office of Juvenile Justice and Delinquency Prevention 

OJJDP Fiscal Year 2013 Research on Best Practices for Mentoring 

Michael J. Karcher, Ed.D., Ph.D., Principal Investigator 

Disclaimer: This project was supported by Grant No. (2013-JU-FX-0008) awarded by the Office of 

Juvenile Justice and Delinquency Prevention, Office of Justice Programs, U.S. Department of 

Justice. The opinions, findings, and conclusions or recommendations expressed in this publication 

are those of the author and do not necessarily reflect those of the Department of Justice. 

Acknowledgements: This report benefitted from substantial contributions provided by Joe Avera 

and Yulia Zholu as Research Assistants, and the guidance of Barbara Tatem Kelley, Social Science 

Analyst, Innovation and Research Division at the Office of Juvenile Justice and Delinquency 

Prevention throughout the project. Funding for the original impact study came from the William T. 

Grant Foundation (2004-2008, Effective interactions with Hispanic youth in school-based 

mentoring). That study’s success depended heavily on the assistance of staff at the Communities in 

Schools of San Antonio agency over a three-year period. 

Correspondence concerning this report should be addressed to Michael J. Karcher, Professor, 

Department of Educational Psychology, 501 W. Cesar Chavez Blvd., DB 4.337, University of Texas, 

San Antonio, Texas, 78207 michael.karcher@utsa.edu 210-458-2032 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 2

TABLE OF CONTENTS 

ABSTRACT p. 5 

EXECUTIVE SUMMARY p. 6 

BACKGROUND AND BRIEF REVIEW OF THE LITERATURE p. 16 

RESEARCH QUESTIONS AND OBJECTIVES p. 31 

STUDY METHODS AND ANALYTICAL TECHNIQUES p. 32 

FINDINGS p. 43 

Goal 1: Long-term Outcomes of School-based Mentoring p. 45 

Goal 2: Identify relationships between interactions, experiences, and adult criminality p. 60 

DISCUSSION p. 75 

IMPLICATIONS FOR POLICY, PRACTICE, AND FUTURE RESEARCH p. 79 

REFERENCES p. 83 

APPENDICES p. 90 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 3

Tables 

Table 1: Coding Mentoring Interactions for Corresponding TEAM Framework Categories 

Table 2.1: Age at Enrollment in August 2003 to 2004 

Table 2.2: Sample Breakdown by Sex, Intervention Group and Grade Level 

Table 2.3: Size of Samples in the Final Dataset at Each School Level 

Table 2.4: Size of Samples in the Final Dataset at Each School Level for Boys and Girls 

Table 2.5: Number of Boys and Girls Assigned to Treatment and Control Conditions 

Table 3.1: Mean Rate of Arrest for Any Crime (Misdemeanor and Felony) Committed by Individuals 

by Sex, School Level, and Intervention Group 

Table 3.2: Main Effects of Treatment Condition Assignment and Sex on Probability of Arrest 

Table 3.3: Number of Misdemeanor Arrests by Age 21 for Youth in Treatment vs. Control Groups 

Table 3.4: Number of Misdemeanor Arrests by Age 21 for Males in Across Treatment Groups 

Table 3.5: Number of Misdemeanor Arrests by Age 21 for Females in Across Treatment Groups 

Table 3.6: Effects of Treatment and Sex on Probability of Arrest Adjusted for Age and Grades 

Table 3.7. Number of Individuals Arrested (Events) Vs. Censored 

Table 3.8. Means and Confidence Intervals for Survival Time by Group and Sex 

Table 3.9. Survival Analysis Estimates and CI for Independent Variables in the Equation 

Table 3.10: Enrolled by Age 22 for Treatment Groups by Sex 

Table 3.11: Number Enrolled by Age 22 for Treatment Group by Sex and School Level 

Table 3.12: Logistic Regression Predicting Post-Secondary Enrollment (with Covariates) 

Table 3.13: Bootstrap Logistic Regression Predicting Post-Secondary Enrollment (with Covariates) 

Table 3.14: Zero-order and Partial Correlations Between Long-term Outcomes, Mentoring 

Interactions and Relationship Quality 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 4

Table 3.15: Partial Correlations Between Long-term Outcomes, Mentoring Interactions and 

Relationship Quality by Sex 

Table 3.16: Logistic Regression Coefficients for Main, Moderator, and Conditional Effects 

Figures 

Figure 1. The Theoretically Evolving Activities in Mentoring (TEAM) Framework 

Figure 2. CONSORT 2010 Flow Diagram for SMILE study 

Figure 3.1. Probability of Misdemeanor Arrest by Sex in Each Treatment Condition 

Figure 3.2. Cumulative Survival Rate by Month Starting at Age 17 

Figure 3.3. Cumulative Hazard Function by Month Starting at Age 17 

Figure 3.4. Moderation of Likelihood of Misdemeanor Arrest by Conversation Focus and Reported 

Mattering to One’s Mentor 

Figure 3.5. Mattering Moderates Effects of Problem Focus When Focus on Relationship is Low 

Figure 5. Annual Income Difference at Varying Levels of Post-Secondary Education 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 5

ABSTRACT 

This “Mentoring Best Practices” Long-Term Follow-up project assessed outcomes for adults who as 

children were part of the Study of Mentoring in the Learning Environment (SMILE; Karcher, 2008), 

which took place between 2003-2007 at Communities In Schools (CIS) of San Antonio. The original 

randomized study tested short-term benefits for students from participation in the CIS school-based 

mentoring program. Initially, 516 students who enrolled in CIS-San Antonio at one of 20 

elementary, middle, and high schools were randomly assigned to receive either (1) standard CIS 

services or (2) standard CIS services plus assignment of a mentor. The present study looked at 

outcomes in adulthood ten years later, and found evidence that those who had enrolled in the CIS 

school-based mentoring program were half as likely to have been arrested for a misdemeanor and 

were 10% more likely to have pursued some form of post-secondary education by age 21 than who 

received only standard CIS services. Educational benefits of mentoring were strongest for females. 

Analyses linking actual mentoring interactions to these long-term outcomes further revealed the 

largest benefits were achieved when relationship-building was frequent, and that frequent problemfocused conversations in matches with youth who did not feel they were valued or mattered to their 

mentor predicted increased likelihood of arrest later. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 6

EXECUTIVE SUMMARY 

The purpose of this longitudinal study was to estimate the long-term effects of school-based 

mentoring on study participants’ subsequent criminal activity and educational pursuits during early 

adulthood. The study adds to a body of research that has consistently found short-term benefits of 

youth mentoring in preventing criminal activity. In a meta-analysis by Tolan et al. (2014) examining 

the effects of youth-mentoring program participation on delinquency, criminality, and drug use (in 

studies published between 1970 and 2005), the authors found that the largest effects of youth 

mentoring were reductions in delinquency and aggression. They also found that the provision of 

emotional (i.e., relational) support was a key moderator of program effects on these reductions in 

delinquency and aggression. They noted, however, a problematic lack of information on the types of 

interactions that contribute to effective mentoring (p. 2). Another meta-analytic review by DuBois et 

al. (2011) also concluded that better understanding mentoring interactions is critical to more fully 

understanding the effects of mentoring-program participation on a variety of youth outcomes. The 

second part of the present study, therefore, attempted to link both the types of mentoring activities 

reported by mentors and the quality of the relationship reported by their mentees with long-term 

programmatic effects of program participation on adult criminality and educational persistence. 

The present study extends an investigation called the Study of Mentoring in the Learning 

Environment (SMILE; Karcher, 2008), which took place between 2003-2007, by gathering and 

examining publicly available data on participants’ criminal records and educational attainment ten 

years later. In the original study, 516 students were enlisted for participation in the Communities In 

Schools (CIS-San Antonio) program. Follow-up data collected from 466 of the 516 students was 

used in this study. As one of several services provided by CIS, this study examined the effects of 

random assignment in receiving a school-based mentor. The students were randomly assigned to 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 7

one of two experimental conditions: (1) receipt of standard CIS services or (2) receipt of standard 

CIS services plus assignment of a mentor. This design provides an experimental test of the additive 

effects of being assigned a mentor for students already receiving other support services.1 

SMILE was the first of three large-scale studies of school-based mentoring to report the 

short-term effectiveness of school-based mentoring (Wheeler, DuBois, & Keller, 2010). It was 

unique, however, in three ways. First, it was the only one of three studies to include a sufficient 

number of high-school-age mentees to effectively measure the short-term impact of school-based 

mentoring on adolescents (high school students). Another unique sample characteristic is that it 

included mostly Latino/a children and youth, thereby offering insights into the effects of this 

approach specifically for this ethnic group. Second, an effective collaboration between program staff 

and researchers allowed the collection of relatively complete information about what happened 

during each mentoring meeting (mentor’s activity reports). Third, the study did not employ a waitlist 

comparison group. Thus, neither resentful demoralization among youth or parents who were denied 

a mentor posed a viable threat to the validity of the study inferences, nor was the present long-term 

follow-up study compromised by delivery of the mentoring experience to the control group at a later 

date (which complicates the estimation of long-term outcomes in experimental studies using a waitlist comparison group). 

In the original publication of findings from the SMILE study (Karcher, 2008), the short-term 

effects of being assigned a school-based mentor were estimated nine months after program 

enrollment (at the end of the academic school year) by comparing course GPA, attendance scores, 

1 The original title of this project as originally awarded was “Ten and 40 Years After Mentoring: 

Longitudinal Analyses of Relationship and Developmental Processes as Moderators of Outcomes in 

Two Experimental Studies.” Substantial changes in the goals of the project necessitated a more 

narrowly focused title for this report. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 8

and youths’ responses to attitudinal surveys. This report compared differences across the two 

experimental groups on outcomes at post-test, controlling for pre-test scores, using a between-group, 

intent-to-treat design. The outcomes were generally positive, favoring those assigned to the 

mentoring group, with main effect differences benefitting youth in the mentoring condition, who 

reported more connectedness to peers, greater self-esteem (global and present-oriented), and more 

social support from friends than youth in the control condition. Subsequent analyses considering 

school level and sex as moderators, however, revealed that the program did not affect all students 

similarly. 

The short-term effects of participating in the school-based mentoring program varied by sex 

and school level with elementary-age boys in the mentoring group benefitting the most, and high 

school boys in the mentoring group with no positive and one iatrogenic effect from program 

participation. Specifically, elementary boys in the mentoring condition reported higher social skills 

(empathy and cooperation), hopefulness, and connectedness both to school and to culturally different 

peers. Whereas, in terms of iatrogenic effects, mentored high school boys evidenced general (albeit 

not statistically significant) declines in most outcomes and a statistically significant decline in 

connectedness to teachers relative to high school boys in the control group at the end of the first 

year. These iatrogenic effects were one reason for the importance of conducting the present, longterm follow-up study. 

Subsequent within-group analyses of mentoring interactions suggested time spent in 

discussions of academics, behaviors, and attendance was negatively related to multiple outcomes and 

occurred most frequently in matches with high school boys (Hansen & Karcher, 2009). High school 

boys also reported feeling the least valued by their mentors (Avera et al., 2014), and these youthreported feelings were negatively correlated with many pre-post outcome change scores for the 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 9

mentees. This led to the hypothesis that the iatrogenic effects of mentoring program participation 

among high school boys (Karcher, 2008) might be due to the boys’ reaction to the greater frequency 

of problem or goal-focused conversations (e.g., discussions about grades, misbehavior, attendance) 

in their mentoring match meetings. The present study attempted to test the hypothesis that types of 

interactions might explain long-term outcomes as well. 

There were two goals of the present study. The first was to test the hypothesis that there 

would be long-term benefits of mentoring program participation on rates of crime and the likelihood 

of post-secondary educational enrollment. Because the study also might reveal persistent iatrogenic 

effects for the older boys, two-tailed significance tests were required to address this alternative 

hypothesis. The second goal was to examine the relationship between two types of interactions 

(relationship-building vs. problem- or goal-focused) and these two long-term outcomes (see Karcher 

& Nakkula, 2010), and determine if any link between activities and outcomes could be a function of 

the way specific interactions influenced how valued mentees felt by their mentors. 

Method 

The sample for the study included 466 predominantly Latino/a students from the 20 schools 

(6 elementary, 7 middle, and 7 high schools) who participated in the Communities in Schools (CISSan Antonio) program. Recruitment began with the distribution of 675 consent forms (475 wave I, 

August 2003; 200 wave II, August 2004), and receipt of 525 signed parental consent forms returned 

to authorize student enrollment in the study. After omitting youth with histories of abuse and those 

whom staff removed from the starting study sample, the remaining 466 (89% of 525) youth were 

randomly assigned either to the mentoring condition (and were retained in this group regardless of 

whether they ever did meet with a mentor, n = 234) or to the control condition (n = 232), in which 

students received only the standard CIS services. All students received, on average, 29 hours of 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 10

standard CIS services. Those in the CIS services plus mentoring received, on average, 8 hours of 

mentoring (SD = 5.3 hours, with range of actual time with mentors between 0 and 30 hours). 

Dissimilar proportions of female and male youth, between mentored (143 females/91 males) and 

control groups (females 168/64 males) and across grade levels (see Table 2.2), necessitated 

considering sex either as a covariate, main effect, or moderating factor in most analyses. 

To account for age differences across treatment groups, and to account for the varying length 

of time participants had to accrue a criminal arrest record and pursue post-secondary education in 

adulthood, outcomes for all youth were restricted to the range of activities up to age 21. With 

outcome distributions for the number of criminal arrests and years enrolled in post-secondary 

education being heavily skewed, estimates of change relied on non-parametric tests, making the use 

of logistic regression and survival analysis to test for the presence or absence of the outcome in 

question (not the number of crimes or the degree of persistence in post-secondary education) most 

appropriate. Between-group differences in the probability of arrest or post-secondary educational 

pursuit were estimated using non-parametric Cox regression and survival analysis, then evaluated 

using Kaplan-Meier hazard functions, Wald, and Chi-squared statistics, and associated significance 

tests. 

Findings 

Tests of the effects of being assigned to participate in the CIS mentoring program using 

logistic regression, including both main effects and moderator tests, yielded consistent, positive 

effects of mentoring on decreasing the likelihood of being arrested for a misdemeanor. The same 

three-way interactions (viz. treatment moderated by sex and grade level), including the same 

covariates as in the original study (Karcher, 2008), were replicated and yielded positive effects on 

misdemeanor arrests. Less consistent evidence of change was found for serious, property, and 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 11

violent crimes, likely because of their infrequency. 

In terms of the likelihood of being arrested for a misdemeanor before age 21, the simple 

between-group odds ratio suggests youth in the mentoring group were 55% less likely to have 

committed a misdemeanor by age 21. Including sex as a main effect predictor in logistic regression 

analysis revealed youth in the mentoring group were 61% less likely to have committed a 

misdemeanor by age 21. Girls who had participated in the mentoring program were half as likely to 

have been arrested for a misdemeanor (2%) as girls not in the program (4.3%). The percentage of 

boys in the mentoring program arrested for a misdemeanor by age 21 (4.4%) was 67% lower than for 

the comparison group (12.5%). Both main effects of mentoring were statistically significant. After 

factoring in school level and starting grades, the adjusted odds ratio of .33 suggests those assigned to 

the mentoring condition were 67% less likely to have been arrested for a misdemeanor by age 21. 

Logistic regression analyses (using SPSS 24) revealed a statistically significant difference in 

the probability of post-secondary educational enrollment between groups, with rates of pursuing 

some form of post-secondary education (certificate programs, trade school, college) higher for the 

treatment (33%) than control group (30%). Therefore, mentees were 10% more likely to pursue postsecondary education, and the odds of enrolling in some post-secondary pursuit were 14% greater for 

mentees; the odds of enrolling in some form of post-secondary education or training program were 

27% greater for mentees when the odds ratio from logistic regression accounted for differences in 

starting levels of problem behaviors, grades, and sex. Within-sex comparisons, however, indicate the 

only statistically significant difference was the higher rate of post-secondary enrollment for girls 

assigned to the mentoring condition (38%) compared to girls in the control (30%) condition. Thus 

the effects of the mentoring program on post-secondary educational pursuit were found only for 

girls, indicating girls were more likely to pursue post-secondary education by age 21 if they were 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 12

assigned to the mentoring program. For boys, the difference in the percent of youth in the mentoring 

(25%) and control group (31%) who had enrolled in post-secondary education favored the control 

group, but the difference was not statistically significant. The statistical significance of these 

differences—significant for girls, not statistically significant difference for boys—was the same both 

when estimating odds ratios without and then with pre-intervention grades in math and reading and 

starting levels of problem behavior as covariates, and with and without bootstrapping. 

Having confirmed the long-term benefits of school-based mentoring, subsequent analyses 

examined what types of experiences and mentoring interactions contributed most to relationship 

quality as well as these long-term outcomes. Bivariate and partial correlations examining the 

relationship between rates of matches engaging in specific mentoring activities, youth-reported 

relationship experiences (e.g., closeness experienced in the match), and the two long-term outcomes 

of crime and post-secondary pursuit yielded several statistically significant associations. Although 

correlations between both mentoring activities and mentee experiences and these outcomes are not 

evidence of causality, the pattern of association supports several hypotheses from the original study, 

the TEAM framework (Karcher & Nakkula, 2010), and the mentoring literature by revealing that 

specific mentoring interactions were useful predictors of long-term outcomes. 

Partial correlations between likelihood of arrest for a misdemeanor in adulthood, mentoring 

interactions during the study, and self-reported experiences of the quality of the mentoring 

relationship while in the program, controlling for initial rates of misbehavior and grades, revealed 

that youth whose matches engaged in more relationship-building discussions were less likely to be 

arrested for a misdemeanor in adulthood. In comparison, time spent in discussions of grades, 

attendance, and behavior, by comparison, was not directly related to likelihood of arrest in adulthood 

but was negatively related to multiple relationship quality measures and predicted shorter match 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 13

duration. Youth who reported greater connectedness to the program had longer matches, and 

mentees reporting they felt they mattered more to their mentors were all less likely to be arrested for 

misdemeanors by age 21. 

In terms of explaining post-secondary educational pursuit, the direction of the association 

between match activities and long-term outcomes was similar. Both zero-order and partial 

correlations between relationship-building discussions and post-secondary education were positive, 

and time spent on homework was negatively associated with post-secondary pursuit of some kind. 

The negative association with homework was stronger for males than females, but the positive 

association between time talking to build the relationship and later post-secondary pursuit held only 

for females. Also, only for females was the total number of mentoring hours associated with later 

education. 

To test the hypotheses posed in the TEAM framework (Karcher & Nakkula, 2010) about the 

interactive nature of mentoring activities and relationship quality, moderator analyses were 

conducted to test the way one type of mentoring activity may have moderated the influence of the 

other. To test the hypothesis that mentoring conversation topics affect the way the quality of the 

relationship predicts outcomes, conditional analyses were undertaken that considered the frequency 

of specific relationship-building and problem-focused activities and examined how each moderated 

the way that relationship quality contributed to long-term risk for arrest. These analyses followed the 

approach to conditional process analyses described by Hayes (2013) and used the SPSS macro 

PROCESS developed by Hayes. 

For both preventing criminality and fostering post-secondary educational persistence, 

PROCESS analyses revealed that the largest benefit of school-based mentoring in deterring later 

arrest manifested when problem/achievement-focused interactions occurred in relationships in which 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 14

the youth felt she or he mattered to the mentor and where time had been spent getting to know the 

mentee. Specifically, in the absence of time spent getting to know the mentee, among youth feeling 

least valued by their mentors, frequent engagement in problem/achievement-focused interactions 

predicted a greater probability of later criminal arrest for a misdemeanor than for similar matches 

that spent less time discussing school problems. Subsequent analyses are required to confirm the 

nature and directionality of these relationships, of course, but one interpretation that is consistent 

with the hypothesized causal model is that relationship-building interactions set the stage for 

problem/achievement-focused interactions to be effective through the way they make youth feel 

important to their mentors. When little time is spent getting to know mentees, and when mentees feel 

they matter little to their mentors, a heavy focus on mentees’ problems may be counterproductive. 

Discussion 

The goals of this study were (a) to examine whether the harmful outcomes as well as positive 

programmatic outcomes of school-based mentoring found ten years earlier (Karcher, 2008) persisted 

over time; and (b) to explore relationships between two types of mentoring interactions and later 

rates of crime and pursuit of post-secondary education. Findings from this randomized study reveal 

positive long-term effects of school-based youth mentoring in decreasing rates of arrest for 

misdemeanor crimes and increasing educational enrollment in adulthood. Ten years after 

participating in the Communities in Schools of San Antonio mentoring program, intent-to-treat 

analyses comparing 234 youth randomly assigned to receive a mentor (regardless of whether they 

ever did) to 232 youth assigned to the control condition revealed that mentored youth were 55% less 

likely to have committed a misdemeanor by age 21 and as much as 67% less likely to have 

committed a misdemeanor once the role of age, pre-match characteristics, and mentee sex were 

considered. Mentored males were 67% less likely than control males, and mentored females were 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 15

51% less likely than control females to commit a misdemeanor. In terms of post-secondary 

enrollment, the results varied by sex. Overall, the odds of youth in the mentoring treatment condition 

having enrolled in some form of post-secondary education within five years of graduating were just 

14% greater than for the control group, but females in the mentoring condition were 27% more 

likely to have pursued post-secondary education than females in the control group. 

In addition, it was evident that both match strength (measured by indicators of feeling one 

matters to the mentor and feeling valued by one’s mentor) and specific interactions defining the 

nature of the relationship (measured by the type and frequency of specific interactions) were 

important predictors of long-term outcomes. The findings suggest that engaging in the relationshipbuilding conversations that are the hallmark of youth mentoring may contribute to the way 

mentoring influences arrest rates in adulthood, and that conversations that focus on the mentees’ 

academic performance and in-school behavior may be counterproductive if insufficient time has 

been spent getting to know the mentee, time which perhaps leads the mentees to feel that they 

uniquely matter to the mentor. The findings from these conditional process analyses, which 

accounted for interactive effects of activities and relationship quality on long-term outcomes, 

support the short-term findings from the original study ten years prior (Avera, Karcher, & Zholu, 

2014), wherein bivariate correlations revealed negative associations between problem/achievementfocused interactions and both short-term outcomes and relationship experiences; suggesteding a 

problem focus may undermine relationship quality in school-based matches. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 16

BACKGROUND AND BRIEF REVIEW OF THE LITERATURE 

The positive effects of youth mentoring on short-term (i.e., post-test) outcomes are now well 

documented through multiple meta-analytic investigations (DuBois et al., 2002, 2011; Tolan et al. 

2014). The effect size of youth mentoring is consistently, generally small; however, these metaanalyses also reveal that moderators of main effects can double a program’s impact, which 

underscores the need to better understand moderators of program impacts. Better understanding how 

mentor and mentee interactions (activities and discussions), separately or in combination, moderate 

program impact to yield the strongest outcomes is seen by many as the next critical step in the field 

of youth mentoring research (DuBois & Karcher, 2014). In fact, identifying what mentoring 

activities are best for youth of different ages has been recognized in the field as a topic needing to be 

better understood for many years (Naom, Malti, & Karcher, 2014), yet the field has made only 

modest movement toward better understanding the effect of specific mentoring interactions on 

mentoring outcomes, with little work specifically exploring the way school setting or developmental 

status may moderate the effect of specific interactions on outcomes for mentees (Karcher & Hansen, 

2014). 

Youth mentoring has been shown to be effective in fostering short-term changes that may 

facilitate successful adult development. It has been found effective across multiple program contexts 

and for youth of varying degrees of risk for adult criminality. The meta-analyses conducted by 

DuBois and colleagues (2002; 2011) have revealed technically small effects of youth mentoring 

(ranging from d= .15 for children in schools to .28 for adolescents in the workplace). Early work of 

DuBois et al. (2002) also revealed the cumulative effect of best practices. Their analyses illustrated 

how effect sizes could be doubled, from small (Cohen’s d = .20) to nearly medium-sized effects (d = 

.40), when a half dozen or more known best practices are employed (compared to the average use of 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 17

two or three by most programs). Two such practices, for example, are utilizing mentors with 

backgrounds in the helping professions and using structured activities in mentor-mentee match 

meetings (DuBois et al., 2002). Many view the first of these as a proxy indicator that the mentor will 

understand the value of a compassionate approach to mentoring that establishes an atmosphere of 

trust before any difficult topics or tasks are undertaken. The second, using structured activities, is 

frequently seen as a means to convey purposefulness to the participants in this otherwise potentially 

awkward context for program-based friendships (i.e., mentoring). Some have argued this is 

particularly critical when mentoring adolescents (Noam, Malti & Karcher, 2014). 

These two practices reveal the importance of effectively guiding mentors’ use of both 

relational and goal-directed activities, which makes better understanding these practices and their 

effect on mentoring relationships an important policy or pragmatic research focus. Consider Tolan 

and colleagues’ (2012) meta-analysis of youth mentoring program outcomes on delinquency, 

criminality, and drug use in studies that were reported between 1970 and 2005. Two of the main 

conclusions of their analyses were, first, that youth mentoring can reduce delinquency and 

aggression, and second, that, when using mentoring for the prevention of delinquency and 

aggression, the provision of emotional (i.e., relational) support was a key moderator of program 

impacts. Tolan et al. lamented a problematic lack of information on these interactions in the studies 

they included in their analyses, stating “the collected set of studies are less informative than expected 

with quite limited detail in studies about what comprised mentoring activity” (p. 2). The lack of 

clarity may be especially problematic when mentoring teens in schools because of the increased 

awkwardness older students may experience when they imagine how their peers view their meeting 

with a stranger in a formal mentoring program (Noam, Malti & Karcher, 2014). 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 18

Indeed, despite the consistent evidence of positive effects of mentoring programs, there is 

evidence that older adolescents may not benefit as much from traditional school-based youth 

mentoring as younger children. Consider the Study of Mentoring in the Learning Environment 

(SMILE; Karcher, 2008), which is the only large-scale study of school-based mentoring to date with 

a sizeable number of high-school-age students. In that study, Karcher reported markedly different 

outcomes for older and younger youth. Pre-adolescent Latino boys benefitted the most and 

demonstrated statistically significant improvements in conventional outcomes such as connectedness 

to school, social skills, and hopefulness. Older boys, however, reported declines in connectedness to 

school (d = -.23) relative to controls and statistically significant declines in connectedness to 

teachers after receiving a school-based mentor. One reason may be the type of mentoring 

interactions employed. In that study, mentors were four times more likely to focus on mentees’ 

behavior and attendance problems when matched with high-school-age mentees than with 

elementary mentees. 

Similarly, research on the Big Brothers Big Sister (BBBS) community-based mentoring 

program by Grossman and Rhodes (2002) found that older adolescents were more likely to terminate 

their matches early and thereby experience fewer benefits. It is worth noting that, in both the BBBS 

program and the school-based Communities in Schools mentoring program evaluated in the SMILE 

study (Karcher, 2008), staff trained mentors to focus on relationship-building as their primary 

intervention approach regardless of the age of youth. 

These findings along with common sense agree that how mentoring works may differ for 

older and younger youth. Much youth-mentoring research with children suggests that one of the 

ways mentoring affects long-term outcomes is through improving the relationships in youths’ 

familial, academic, and social networks. For example, several studies have found that gains from 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 19

school-based mentoring for preadolescents, specifically upon academic achievement and attitudes, 

appear to be mediated by improvements in mentees’ relationships with parents (Karcher, Davis, & 

Powell, 2002; Rhodes, Grossman, & Resch, 2000) and with teachers (Cavell, Karcher, & Elledge, 

2010). In all these studies, however, the majority of the mentees were under the age of thirteen; thus, 

little can be extrapolated regarding effects of school-based mentoring for teens. 

With older youth, their peers’ perceptions become more salient factors in self-esteem and 

may be more important in influencing program outcomes in positive and negative ways. Because 

school-based mentors are more likely to interact with their mentees alongside their peers during 

lunch than with younger mentees (Herrera & Karcher, 2014), how their peers understand the 

meaning of a mentees’ having been assigned a mentor may enhance or undermine program success. 

It is possible that teens may recoil against the stigma associated with having a mentor by investing 

more heavily in peer relationships as a consequence of their peers’ observing the presence of a 

mentor. Evidence of this negative peer influence may appear, for example, as an “improvement” in 

peer relationships as an immediate mentoring program outcome. Evidence supporting the 

interpretation of this increase in peer connectedness as an iatrogenic effect would be when such 

increases are not accompanied by corresponding improved parental or teacher relationships. This is 

what was observed in the SMILE study (Karcher, 2008) for high-school-age boys and girls. In 

particular, high-school-age Latinas seemed to have benefited from being assigned mentors, but they 

reported only higher levels of self-esteem, peer support, and connectedness to peers; however, in the 

absence of any evidence of more conventional improvement in attendance, grades or connectedness 

to school, parents, or teachers, these outcomes appear suspect. It has already been noted that the 

high school boys reported declines across the board compared to their peers, and most strongly in 

decreased teacher connectedness, after being assigned mentors. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 20

Negative Effects of Youth Mentoring Programs (Iatrogenic Processes) 

Until recently, most notably in the work by Cavell (Cavell & Elledge, 2014) whose work 

strategically enlists peers alongside mentors to positively influence mentees’ outcomes, most 

research discussing the role of peers in the mentoring process, particularly for adolescents, has been 

about the ways in which peers can negatively influence the effects of program participation for 

youth. Analyses of the way program interactions may influence long-term outcomes from the 

Cambridge-Somerville Youth Study in the 1950s, which included mentoring as one intervention 

component of a larger set of intervention services, suggested that frequent association of 

participating youth with other deviant youth put their less deviant peers on a trajectory toward 

criminality (McCord, 1978). For this reason, further work to better differentiate helpful from harmful 

mentoring interactions and to better understand their contribution to long-term outcomes like 

criminality in adulthood is very much needed, specifically on programs that exclusively provide 

mentoring. 

Better understanding what intervention characteristics moderate the treatment-effect 

relationship in youth mentoring for different ages is critical given the divergent outcome trajectories 

outlined above. Barlow (2010) commented that over the past 40 years, continued refinement of 

psychotherapeutic interventions has taken place, but concomitant study of the reasons for negative 

outcomes of other interventions has not occurred. The work by Dishion, McCord and Poulin (1999), 

which re-analyzed the data from the Cambridge-Somerville study to better understand the reasons 

behind McCord’s (1978) reports of negative intervention effects, provided one of the most important 

contributions to the youth development field, with over 1400 citations in the past ten years. Their 

analyses on the amount of time participants spent with one another outside the supervision of staff 

found that peer interaction could foster deviancy training that cemented some program youth on a 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 21

path toward crime. The relationship to school-based mentoring is that, to date, limited research has 

examined the way in which the delivery of mentoring to youth in settings alongside their peers, in 

unstructured settings, like when students meet with their mentors in the cafeteria alongside other 

students, may affect program outcomes. 

Mentoring Interactions 

The study of what happens in youth mentoring interactions is critical to understanding not 

only positive as well as negative effects of youth-mentoring programs but also may help inform our 

understanding of the manner in which mentoring relationships achieve these outcomes. This was the 

premise emphasized by Tolan et al. (2014) in their Campbell report on the effects of youth 

mentoring on delinquency and aggressiveness. They argue we need to know more about both what 

happens in mentoring relationships—specifically in mentoring programs that do prevent crime and 

delinquency—and also about the ways in which specific interactions are leveraged to achieve these 

outcomes most efficiently. 

In the mentoring field, two main theories have been used to explain how a mentors’ specific 

interaction focus influences the quality of the relationships that emerge in their mentoring 

relationships with youth. These two approaches, the instrumental and developmental styles, are 

described below to introduce a framework that uses lessons learned from research on both styles to 

propose several hypotheses about the causal manner through which and interpersonal conditions 

under which specific mentoring interactions are most influential. 

Styles of Mentoring--Instrumental and Developmental 

The two predominant theoretical perspectives in the field about what constitutes effective 

mentoring styles are born from research conducted with mentees of different ages. This is one reason 

a developmental lens seems important to bring to any study of school-based mentoring that involves 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 22

both children and adolescents. The first view comes from research on internships in the workplace, 

which suggested mentoring with teens using an “instrumental” style that focuses first on skills 

development is most useful (Darling, 2005; Hamilton & Hamilton, 2002, 2005). Indeed, there is 

some evidence that structured, workplace mentoring yields larger impacts than community or 

school-based approaches that tend to focus first on relationships (DuBois, Holloway, Valentine, & 

Cooper, 2002). Yet, more often cited in the field of youth mentoring is research on programs in 

which younger children are mentored in the community, such as through the well-known Big 

Brothers Big Sisters program. In one study of this program, Morrow and Styles (1995) dubbed the 

“developmental” interaction style of relationship-building, having fun, and getting to know each 

other first as most viable. Indeed, this approach to mentoring, which assumes the relationship must 

be formed first, before other skills and attitudinal changes can be realized, is advocated by many 

(Rhodes, 2002). 

These two constructs—the “developmental” and “instrumental” mentoring styles—have had 

considerable staying power in mentoring research but have not been advanced or investigated further 

than what was originally proposed until recently (Karcher & Nakkula, 2010). Originally proposed by 

Morrow and Styles (1995) and Steve and Mary Agnes Hamilton (1992) to characterize match 

interactions in two different studies, these two styles remain the most empirically supported 

approaches. 

Morrow and Styles (1995) conducted a study of matches in the Big Brothers Big Sisters 

program. Their study, in contrast to that by Hamilton and Hamilton (1992) described above, focused 

mostly on preadolescents. The results revealed a portrait of successful mentoring relationships which 

they described as “developmental.” These relationships were given the label “developmental” 

because the adult partner in the match focused on providing the youth with a comfort zone in which 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 23

to address a broad range of developmental tasks—such as building emotional well-being, developing 

social skills, or gaining straightforward exposure to a range of recreational and cultural activities. 

“Developmental volunteers responded flexibly, adjusting any preconceived notions as to the reality, 

circumstances, and needs of their younger partner. Furthermore, these volunteers intentionally 

incorporated youth into decision-making about the relationship, allowing them to help choose 

activities and have a voice in determining whether and when the adult would provide advice and 

guidance” (Morrow & Styles, 1995, p. 19). That is, the mentors were initially relational and 

collaborative in dealing with their mentees, avoiding interactions that might stigmatize, and only 

later addressed problems or skills training. 

By contrast, another approach called the “instrumental style” emerged from research with 

older adolescents that revealed the unique developmental needs of teenage mentees and relationship 

patterns in their mentoring relationships with adults. The primary authors of the literature on this 

approach, Steve and Mary Agnes Hamilton (2002), focused on the process of workplace mentoring 

and apprenticeship. Their research into reasons why some matches tended to meet less frequently 

than they should revealed that “understanding [the mentors’] purpose was a critical predictor of the 

regularity of meetings.” Those who saw their initial or primary purpose as developing a relationship 

with their mentees were least likely to meet regularly, whereas “the mentors who seemed best able to 

overcome the frustrations of their task were those who combined the aims of developing competence 

and developing character” (1992, p. 548). It is for this reason they suggest that mentoring for highschool-age youth is more appealing to youth and more effective when “it occurs in the context of 

joint goal-directed (instrumental) activity” and when “the relationship develops around shared goals 

and actions more than purely social interaction” (2005, p. 352-353). They suggest, furthermore, that 

“instrumental mentoring” is more effective for high school students than younger youth. Recall that, 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 24

by contrast, in the work by Morrow and Styles (1995) and most studies of the Big Brothers Big 

Sisters program (e.g., Herrera et al., 2007), the mentees are children and younger adolescents. This 

highlights the importance of mentees’ specific developmental needs. 

Although these two styles reflect what the mentoring literature seems to agree are the two 

most effective mentoring approaches, there have been no formal tests of the relative benefits of each 

in the context of school-based mentoring. Yet there may not be a need to test each as competing, 

given that these two styles share several properties that allow their integration into one model. In 

both styles, for example, it is clear that, across the course of a developing relationship, both 

relationship-building and goal, problem, or task-focused interactions will take place. One match will 

start relational and become goal-directed later (instrumental) and another will start goal-directed and 

become relational (developmental). What is needed is a contextual view of when one or the other 

focus may work better for a specific age group; indeed, the two views agree that the more effective 

mentoring relationships, regardless of the mentee’s age, incorporate both goal-directed and 

relationship-building interactions. What is needed is testing the interactive or moderating effects of 

one interaction focus on the other as both views suggest it is the presence of both that yields larger 

program effects, especially when a collaborative interpersonal relationship, one in which both 

individuals feel valued and important, has become established. 

The TEAM Framework: A Theoretical Integration of Existing Research 

Karcher and Nakkula (2010) proposed a three-pronged typology that integrates both 

approaches—goal-oriented (“instrumental”) and relationship-focused (“developmental”) activities— 

by viewing them as complementary and central to the interpersonal process of collaboration. The 

Theoretically Evolving Activities in Mentoring (TEAM) Framework (Karcher & Nakkula, 2010; 

Figure 1) was developed to empirically distinguish between those mentoring activities that are 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 25

primarily focused on building a relationship versus those that are directed at achieving a specific 

goal, skill, or outcome, but also to illustrate the critical importance of interpersonal collaboration (of 

both feeling they are receiving and giving) in the relationship. 

The second dimension of this framework is the purpose of the interaction—whose values, 

goals, or needs are met in a given interaction. The view of purpose draws on Jessor and Jessor's 

(1971) problem behavior theory research, which revealed that delinquent youth were far more 

committed to unconventional goals and relationships (such as with peers), than to conventional 

relationships (e.g., with parents, teachers, employers) and goals (e.g., job and educational success). 

This second dimension—a conventional vs. a playful purpose of any given interaction—indicates 

whose goals or purposes are served in that mentoring interaction. Its nature or definition, however, is 

subjective and developmental. It is developmental in that young children prioritize and value 

playing, and playful mentoring interactions may be seen as meeting the mentees’ needs. However, 

for teenagers, who straddle the worlds of adults and children, the decision about whether a given 

teen wants to think about his future or have fun in the moment, may be determined subjectively, and 

often can reflect her or his expectations based on prior experiences with adults and her or his own 

determination about the value of a mentor for helping to achieve some goal in the future, versus the 

likelihood that the youth would want to become friends with an unfamiliar adult. This issue of 

purpose, and mentors’ attunement to the youths’ needs, is what may make or break a mentoring 

relationship with a teen (see Pryce, 2012). 

Finally, the TEAM Framework identifies the degree of youth-centeredness of mentor-mentee 

interactions or collaboration. In both the research espousing the benefits of a relationship focus 

(Morrow & Styles, 1993) as well as research on the more goal-focused, apprenticeship-style 

mentoring (Hamilton & Hamilton, 1992; 2005; 2013), researchers reported that a youth-centered 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 26

relationship, in which the youth was given equal authority to author or shape the match's interaction 

focus, was essential. The youth-centeredness may be particularly important when mentoring teens 

because without the youth’s communicating interests to the mentor, it often can’t be known whether 

a given interaction is viewed as purposeful by the youth. The collaborative or co-authored nature of 

the relationship seems a particularly important determinant of the effectiveness of a given mentoring 

interaction with a teen. 

Figure 1 illustrates the three dimensions of the TEAM framework. In this figure, we see that 

the center column reflects the presence of co-authorship and collaboration—the degree of shared 

decision-making in the match activity selection. We see adult-youth relationship styles that lack 

collaboration and are more unilaterally directed on the right (youth purpose) and left (adult purpose) 

sides of the center column. A unilateral, youth-driven relationship (right column) yields an overly 

playful interaction, regardless of whether it is relationally focused or goal-directed. A unilateral, 

mentor-driven approach (left column) that does not reflect the mentees’ desired purpose puts the 

emphasis on the youth’s future, overcoming problems, and building skills to a degree that negates 

the mentoring relationship. The figure illustrates the focus of interactions reflecting a playful focus 

at the top and goal-directed interactions at the bottom. 

The TEAM framework postulates that mentoring occurs when there are both playful and 

goal-directed interactions at some point in the match. It states that adult-youth relationships that are 

only goal-focused in nature are not mentoring (bottom row), nor is meaningful mentoring happening 

in directionless, solely playful adult-youth relationships. These reflect other standard adult-youth 

roles. This is why the TEAM framework hypothesizes that programs (and matches within programs) 

wherein adults engage youth, over time, through both play and goal-directed activities (center 

squares #5 and #8) that are of mutual interest are the most impactful. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 27

Figure 1. The Theoretically Evolving Activities in Mentoring (TEAM) Framework 

Purpose: Talk Unilateral Authorship: Collaborative Authorship: Unilateral Authorship: “Me” Purpose: Play 

“Me” as Mentor’s focus “We” as Shared focus as Mentee’s focus 

Serves Focus: Minimally goal- Serves 

conventional 

(adult) purpose 

(Adult-centric Talking) directed/structured and/or 

highly relational (Youth-centric Playing) playful 

(youth) purpose 

Adult-led 

spontaneous 

(non-relational) 

1. Preacher/Bore as in 

mentor-driven, but goal is 

vague. Mentor talks about 

whatever seems important at 

2. Peer, classmate or 

acquaintance (Keller & Pryce) 

as in doing whatever both can 

agree on in the moment, this is a 

3. Joker Mentor as in 

unstructured and overly 

playful (e.g., mentee has 

fun, play is spontaneous), 

Youth-led 

spontaneous 

(non-relational) 

the time, mentee is 

disengaged (usually a nonrelational approach) 

non-relational and unstructured 

relationship “about nothing.” 

but mentor feels 

insignificant (non-relational 

approach) 

Adult-oriented 

preventive and 

developmental 

activities or 

discussions 

(relational focus) 

4. Counselor takes a youthdevelopment focus on 

prevention (e.g., indirectly 

addresses conventional 

concerns such as school, 

work); the focus is the 

5. Developmental Mentor (from 

Morrow & Styles) as initially 

relational interaction focus yet 

very collaborative (includes talk 

about interests, relationships, 

experiences; play, casual 

6. Playmate as playful, 

supportive, relational 

interactions focused on 

youth’s interests (e.g., may 

learn skills indirectly); 

focus is the mentee’s selfYouth-oriented 

preventive and 

developmental 

activities or 

discussions 

(relational focus) 

mentee (self-in-the-future) 

and on their relationship as 

the primary means to achieve 

growth 

activities). “We” authorship 

supports the incorporation of 

more goal-oriented interactions 

later on 

in-the-present as enhanced 

through the relationship 

Conventional 

Skill 

Development 

Purpose relevant 

to adult/societal 

goals, interests, or 

7. Tutor (e.g., Keller & 

Pryce). Focus on goaldirected interactions that are 

conventional. Focused on 

developing skills for adult 

world, such as reading or 

8. Instrumental Mentor (from 

Hamilton & Hamilton) as 

collaborative, goal-oriented 

focus on character and 

competence; shared purpose in 

the goal they choose or agree to 

9. Teammate as being 

goal-directed and playful 

(e.g., older and wiser peer) 

to help teammate (mentee) 

develop the skills needed to 

play well 

Playful Skill 

Development 

Purpose relevant 

to the youths’ 

goals, or 

emphasizes 

what mentee 

needs to prepare 

for future (goaloriented focus) 

writing) or goal-directed and 

future-oriented (coaching of 

job skills). Often didactic 

focus on; goal-directed at first, 

the interactions become 

increasingly relational over time 

or may focus in the 

mentee’s present concerns 

(e.g., peers, personal 

relationships) 

outcomes in the 

present (Primarily 

goal-oriented) 

Remedial/ 

Interventionoriented: Serves 

10. Prescriptive/Colonel 

Mentor as heavy-handed 

(often insensitive), 

bombastic, directed at 

11. Master with apprenticeship 

Highly instructive (directive), 

minimally relational 

but has some youth buy-in 

12. Coach as active, fun, 

but very directive and 

minimally relational. Focus 

on youth’s goals, such as 

Remedial/ 

Interventionoriented: Serves 

adults’ goals 

(goal-oriented) problems and adult-identified 

goals 

through shared purpose improved skills 

youths’ goals 

(goal-oriented) 

Serves Focus: Highly structured and Serves 

conventional 

(adult) purpose 

(Adult-centric Learning) goal-directed (and/or 

minimally relational) (Youth-centric Doing) playful 

(youth) purpose 

Purpose: Learn Unilateral Authorship: 

“Me” as Mentor’s focus 

Collaborative Authorship: 

“We” as Shared focus 

Unilateral Authorship: 

“Me” as Mentee’s focus 

Purpose: Do 

Adapted from Karcher and Nakkula (2010). 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 28

In the use of the TEAM framework, it can be assumed that where there lacks a collaborative, 

reciprocal relationship in which both mentor and mentee feel they matter to the other and feel 

valued, no specific focus is likely to achieve long-term positive outcomes in general youth 

development. Perhaps tutoring relationships (Fig. 1 cell 7) that are highly directive (i.e., lack 

collaboration) can still be successful in teaching skills or information, which may yield long-term 

effects as well as provide the starting point for a relationship. But directive, skill-focused tutoring is 

not a mentoring relationship. It may become one, the framework suggests, through gradual inclusion 

of reciprocal exchanges of personal information and playful interactions. 

Table 1: Coding Mentoring Interactions for Corresponding TEAM Framework Categories 

TEAM Cell 

in Figure 1 

Activity Log in SMILE study (Karcher, 2008) 

3 Casual conversation (Discussed sports, weekend activities, holiday plans, fun things 

to do in the community, neighborhood, etc.) 

1 Conversation on social issues (Discussed current events in the news, poverty, 

neighborhood events, religion, cultural issues, etc.) 

2, 4 Conversation about relationships: • Family • Teachers • Friends •

Romantic Friend 

2, 3, 4 Listening & learning (Discussed mentee’s hobbies & interests, feelings, or mentee 

talked most of the time while mentor listened.) 

9, 12 Sports, athletic activity, or outdoor game (activity) (Played basketball, soccer, 

catch, volleyball, tennis...) 

6, 11 Creative activities (activity) (Engaged in drawing, arts and crafts, reading and 

writing for fun, photography, crafts, art projects, etc.) 

4, 6 Indoor games (activity) (Board games, playing cards, chess, Uno, checkers, 

computer games, puzzles, etc.) 

1, 7 Academics (Discussed grades, school, testing, etc.) 

1, 7, 10 Tutoring/Homework (activity) (Helped with homework, did tutoring, helped with 

reading, library, academic computer work.) 

1, 4 Behavior (Discussed youth’s behaviors that were related to problems with peers, 

teachers, adults, specifically misbehavior.) 

1, 10 Attendance, graduating and “stay-in-school” discussion 

1, 11 Future (Discussed college, careers, jobs, goals, dreams, etc.) 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 29

Cells 5 and 8 in Figure 1 reflect a balance of goal-directed and relationship-focused 

interactions over time within the context of a relationship in which both partners feel their input 

matters and feel valued by the other. Around cells 5 and 8 are other youth-adult roles which vary in 

terms of focus and collaboration. In Table 1, the top seven rows reflect the types of activities that 

serve a relationship-building function, and that may provide a foundation for the developmental style 

(cell 5). The bottom five rows reflect the type of skill-focused, task-focused, or problem-remediation 

focused activities that may serve as the starting place for building an instrumental style relationship. 

To shift from the more conventional adult-youth roles to an instrumental style (cell 8) requires the 

inclusion of both relationship building activities (top 7 rows) as well as shared guidance or 

collaboration regarding the focus of their discussions or activities. Thus, cells 5 and 8 reflect some 

degree of cross-over or incorporation of interactions from the top and the bottom, and joint decisionmaking as the way that blend of functional and relational focus and purpose take place. It is out of 

this second element, namely collaboration in decision-making, that feelings of mutual respect and 

appreciation begin to form that typically lead to longer and stronger matches over time. 

For an adult-youth mentoring relationship to manifest, the experience of feeling one matters 

and that one is valued by the other is central. This is what is often captured, reflected in, or meant by 

the term relationship quality. Mattering, and specifically the feeling that a person (the mentee and 

the mentor) believes the other person cares about what they want to do together, conveys that one 

feels respected by his or her partner. It is a necessary characteristic of the relationships shown in 

cells 5 and 8. When both feel they matter, it can be assumed that they feel partnered. They feel they 

are collaborating in steering their ship together. 

The relevance to this study is that, by recording the specific focus of activities and 

discussions that took place in the match (e.g., through collection of weekly activity logs by mentors, 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 30

see Table 1), and knowing the degree to which mentees feel they matter and are valued by their 

mentors (assessed through self-reports), analyses that include combinations of interaction focus and 

interpersonal experience can be tested to determine whether patterns reflective of cell 5 and 8 are 

indeed the best predictors of long-term outcomes. 

This study draws on the archival data from Karcher’s 2008 study and extends it through the 

collection and analysis of additional publicly available follow-up data on adult criminality and 

higher educational pursuits for both studies in adulthood. Karcher’s 2008 Study of Mentoring in the 

Learning Environment is unique in three ways. First, it employed a randomized, experimental, 

alternative-treatment control group. Second, researchers collected weekly activity logs (see Table 1) 

that allow tests of the relationships among specific mentoring interactions and immediate, 

intermediate, and long-term outcomes. Third, it assessed relationship quality in the match through 

mid-year youth reports. With these archival data, in combination with adult outcomes collected ten 

years later, tests of mentoring styles can be examined for the prediction of long-term outcomes for 

the first time. 

This long-term SMILE study also offers a unique opportunity to determine whether initially 

iatrogenic processes (viz. declines on several outcomes as a function of program participation) 

persist in a sizeable number of adolescents, or instead serve as a steeling or hardening effect that 

made them more resilient in the long run, as Karcher (2008) initially proposed. Knowing this could 

inform best practices in recruitment of mentees (such as whether or not to recruit mentees in high 

school at all). This study builds on the primary study analyses by conducting new analyses of how 

activities and relationship-quality assessments contribute to long-term outcomes of school-based 

mentoring. The specific research questions and combinations of variables needed to test these 

TEAM framework hypotheses are described in the following chapter. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 31

RESEARCH QUESTIONS AND OBJECTIVES 

The goal of the first part of this research study was to conduct a longitudinal follow-up on an 

experimental study of school-based youth mentoring: The Study of Mentoring in the Learning 

Environment (SMILE). The SMILE study analyzed data from 466 youth from twenty schools (6 

elementary, 7 middle and 7 high schools) who were identified as at risk for educational 

underachievement by teachers, parents and staff members of the Communities in Schools (of San 

Antonio) agency. The first part of this study examined rates of pursuing post-secondary education 

and engaging in criminal activity. 

The second part of the study examined what happens inside of the matches. Little is known 

about how specific mentoring interactions contribute to program outcomes. The goals of this second 

part of the study were to (a) test theory-driven hypotheses about the most appropriate mentoring 

interactions for children versus adolescents and (b) examine whether initial reports of both 

iatrogenic/harmful and positive programmatic outcomes in the original study (Karcher, 2008) have 

persisted 10 years later. 

Research Questions 

RQ1: Does youth mentoring affect increases in post-secondary educational pursuit by adults? 

RQ2: Does youth mentoring affect reductions in later criminal behavior in adults? 

RQ3: Are there styles of mentoring (i.e., different activities and conversation focuses over 

time) that explain variability in proximal and distal outcomes in youth mentoring? 

RQ4: Are there types of youth, types of mentoring interactions, or interactions between types 

of youth and interactions that predict long-term outcomes from youth mentoring? 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 32

STUDY METHODS AND ANALYTICAL TECHNIQUES 

Present Research 

The goals of this study are to examine the degree to which iatrogenic as well as positive 

programmatic outcomes persist ten years after the original study, and to test hypotheses regarding 

the school-based mentoring interactions that best explain program outcomes for children and 

adolescents. 

Goal 1: Long-term Outcomes of school-based mentoring. The first step toward measuring 

long-term effects was to track the 466 youth who originally began in the school-based mentoring 

study (in 2003 and 2004). In order to conduct intent-to-treat impact analyses, efforts were made to 

identify police arrest and conviction reports in public records as well as post-secondary educational 

pursuit for all youth from the original study and then estimate differences in the proportional 

representation of study participants with these outcomes across the intervention groups to which they 

were initially assigned (regardless of their adherence, compliance, or receipt of intended condition). 

After main effect (treatment vs. control) comparisons were conducted, tests of the three-way 

interaction of treatment condition, sex, and school level (as in Karcher, 2008) were run. 

Goal 2: Identify relationship between youth mentoring interactions and relationship 

and adult criminality. The TEAM Framework (Karcher & Nakkula, 2010) was used to code the 

focus (relational vs. goal-directed) of mentoring interactions in logs completed by mentors after 

every visit (see Table 1). Using logs, mentors specified the time spent in each of several different 

interactions, discussion focus, and activity. Individual match interactions from mentors’ activity log 

reports were clustered into two general types (playful and problem-focused), and were then analyzed 

using bivariate associations and partial correlations with mentees’ reports of relationship quality and 

the long-term program crime and education outcomes. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 33

In order to test whether mentoring activities explain how program participation contributed to 

long-term outcomes, conditional analyses were conducted. Conditional analyses tested whether the 

effect of combinations of playful and problem-focused mentoring interactions on long-term 

outcomes was moderated by mentees’ reported experience of feeling valued and that they mattered 

to their mentors. Because control youth had no reports of mentoring activities, they were not 

included in these analyses. In these conditional analyses of data from youth in the treatment group 

(mentees) whose mentors completed mentoring interaction logs, mentor-reported activities and 

match quality served as the moderating variables explaining variability in long-term outcomes 

associated with these mentoring processes. 

Design 

Intent-to-treat analyses were conducted that compared 232 youth randomly assigned to 

receive a mentor (regardless of whether they ever did) to 234 youth assigned to the control condition 

for two types of dichotomous outcomes, pursuit of post-secondary education and criminal arrests. 

Both logistic regression and survival analyses were conducted to answer questions for Goal 1 above, 

and in order to determine the degree of corroboration of findings from the two approaches. 

Regression and random-effects structural equation models were employed to answer questions 

related to Goal 2 above. 

Sample 

Original study. The Study of Mentoring in the Learning Environment (SMILE, Karcher, 

2008) ran from 2003 to 2007. It originally included 516 at-risk youth (see Figure 2 for CONSORT 

illustration on sample attrition). However, the final analyses included 466 youth from 20 schools 

(six elementary, seven middle schools, and seven high schools) who were identified as at risk for 

educational underachievement by teachers, parents or staff members of the agency Communities in 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 34

Schools (of San Antonio) to offer these youth a host of support services through participation in their 

program. Mentoring is one of those services. The youth were between the ages of 10 and 18, in 

grades five through twelve (see Table 2.1). The majority of the study participants were Latinas (2:1 

ratio of females to males, see Table 2.2). All were designated by either the school, parent or CIS 

staff member as academically at risk. Their average family income was less than $20,000 a year. 

Table 2.1: Age at Enrollment in August 2003 or 2004 

Age Frequency Percent Valid Percent Cumulative 

10 

11 

12 

13 

14 

15 

16 

17 

18 

Total 

55 

83 

84 

52 

36 

64 

48 

35 

9 

466 

11.8 

17.8 

18.0 

11.2 

7.7 

13.7 

10.3 

7.5 

1.9 

100 

11.8 

17.8 

18.0 

11.2 

7.7 

13.7 

10.3 

7.5 

1.9 

100 

11.8 

29.6 

47.6 

58.8 

66.5 

80.3 

90.6 

98.1 

100.0 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 35

N Percent 

Treatment Group Non-Mentee 232 49.8% 

Mentee 234 50.2% 

Total 466 

Sex Female 311 66.7% 

Male 155 33.3% 

Total 466 

Grade Group Elementary 114 24.5% 

Middle School 146 31.3% 

High School 206 44.2% 

Total 466 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 36

Figure 2. CONSORT 2010 Flow Diagram for SMILE Study 

Assessed for eligibility (n= 525) 

Enrollment 

Excluded (n=9) 

� Not meeting inclusion criteria due 

to history of abuse revealed (n=9) 

Allocation 

Randomized (n=516) 

Allocated to Mentoring Condition (n=267) 

� Received Standard Services Plus Mentoring (n=240) 

� Did not receive allocated intervention (n=27): 

Refused to meet with mentor (n=3) 

Never met assigned mentor (n=9) 

Insufficient mentors to assign (n=15) 

Allocated to Control Group (n=249) 

� Received Standard Services Only (n=245) 

� Did not receive allocated intervention (n=4) 

Crossover: Received mentoring (n=4) 

Follow-Up 

Lost to follow-up (n=17) Lost to follow-up (n=31) 

Discontinued intervention Discontinued intervention: 

Moved/Withdrew from the school (n=16) Moved/Withdrew from the school (n=22) 

Graduated early (n=1) Withdrew from CIS services (n=3) 

Transferred to alternative school (n=2) 

Graduated early (n=2) 

Unknown (n = 2) 

Analysis 

Analyzed as Non-Mentoring Control Group Analyzed as Mentoring Treatment Group (n=234) 

(n=232) � Excluded from analysis (Insufficient reliable 

� Excluded from analysis (n=0) identifying information to track person) (n=2) 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 37

Random assignment was stratified within school level in order to ensure comparable numbers 

of treatment and control group participants at the elementary, middle, and high school levels. Table 

2.3 illustrates that in the final sample of 466 participants, the number of participants in each 

randomly assigned group were statistically equivalent in terms of the number within each school 

level. Although there were more participants at the high school level than the other two school 

levels, this is a reflection of the structure around grade levels, as there were 50 participants in each 

grade (with the exception of 5th grade, where there were 114 students). Students in grades 4 and 

below were excluded from the study because it was determined they would not be able to complete 

the self-reported outcome surveys reliably. 

The 5th grade was oversampled to allow for a sufficiently large sample at the elementary 

level to allow comparisons across school levels, as reported in the original study (Karcher, 2008). 

Oversampling at the elementary level, as well as of boys at the high school level, into the treatment 

group also was conducted in order to allow adequate representation of elementary students and of 

boys in the sample of mentees to allow for statistical tests of the relationships between mentoring 

practices and outcomes. This oversampling of boys was possible, and also somewhat necessary, 

because of the greater number of adults who wanted to mentor at the elementary level. In fact, most 

of the 22 youth who were assigned to the mentoring condition but who never met with a mentor 

were at the high school level. The result of this approach to stratified random assignment was a 

sample that was statistically balanced across treatment conditions as a whole and across school grade 

levels. Therefore, simple main effect and between-group tests of treatment effects were possible. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 38

Random Assignment 

Intent to Treat Groups 

Non-Mentee Mentee Total 

Grade Group Elementary 57 57 114 

Middle School 74 72 146 

High School 101 105 206

Total 232 234 466 

The sampling strategy effectively ensured that the proportion of treatment and control group 

participants within each grade level was similar. Similarly, for boys and girls, the proportional 

assignment to each treatment condition, when examined separately for each sex, was balanced across 

the three school levels, as shown in Table 2.4 below. Chi-square tests for boys (X2

=.84) and Girls 

(X2=.89) were non-significant, suggesting school level and treatment assignment were not 

confounded. However, between the sexes there were marked differences that resulted from the 

oversampling of boys into the treatment condition and the disproportionate number of girls enrolled 

in CIS. 

There were, however, large differences in the allocation of participants to treatment 

conditions between boys and girls. There were half as many boys interested in participating in the 

CIS program as girls to begin. In order to achieve satisfactory statistical power for planned analyses 

of mentoring program practices on outcomes, boys were randomly assigned to the treatment group at 

a 3:2 ratio to reach a total of 100 male mentees (This number was subsequently reduced to 91 after 

attrition and refusal [all three treatment refusers were male]). 

Final Technical Report: 2013-JU-FX-0008 
Final Technical Report: 2013-JU-FX-0008 39

Table 2.4: Size of Samples in the Final Dataset at Each School Level for Boys and Girls 

Sex 

Intent to Treat Group 

Non-Mentee Mentee Total 

Female School 

Level 

Elementary 41 29 70 

Middle School 48 41 89 

High School 79 73 152 

Total 168 143 311 

Male Grade 

Group 

Elementary 16 28 44 

Middle School 26 31 57 

High School 22 32 54 

Total 91 155 

 

 

 

 

 

 

 

 

 

 

64 

Table 2.5: Number of Boys and Girls Assigned to Treatment and Control Conditions 

Random Assignment 

Non-Mentee Mentee 

(Control) (Treatment) Total 

Sex Female 168 143 311 

Male 64 91 155 

Total 232 234 466 

This resulted in imbalanced proportions of boys and girls across treatment and control 

conditions (see Table 2.5). As a result, all analyses need to include sex as a main effect (and possibly 

as a moderator as well). 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 40

Measures Collected for the Present Study 

Pursuit of post-secondary education. To determine whether study participants had engaged 

in any form of post-secondary education, a list of participant names and dates of birth was submitted 

to the Research Center of the National Student Clearinghouse (NSC; https://nscresearchcenter.org/). 

The NSC gathers data for all types of post-secondary institutions (i.e., two-year and four-year 

institutions, public and private institutions, and nonprofit and for-profit institutions). Analyses 

focused on whether enrollment in any form of post-secondary education had occurred, not whether 

or not individuals had graduated, because there were only 24 graduates out of 466 participants, or 

five percent of the sample, and this was across several different types of educational programs, 

making the term “graduation” vague or unreliable (e.g., awarded certificate, earned a degree) as well 

as low number for analyses. 

Adult arrests. The adult arrest history of each participant in the original sample was recoded 

to indicate whether or not each study participant had been arrested for a misdemeanor during the first 

four years for which study participants had an adult criminal record. In Texas, criminal records are 

collected in a permanent record starting at age 17, so each person’s record was screened for evidence 

of an arrest for a misdemeanor before age 21. The study focused on misdemeanor crimes, because 

rates of property, violent, and other serious crimes were too infrequent and resulted in too many data 

analysis limitations to yield reliable findings. See Appendix for list of crimes in each category. 

Data Management and Confidentiality 

Several steps were taken to ensure confidentiality of participants’ information throughout the 

duration of the study. Personal identifiers in documents from the original studies (surveys by youth, 

parents, and teachers, as well as grades and attendance records) were previously de-identified by 

removing personally identifiable information from the paper files once in the electronic dataset. Data 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 41

in the electronic dataset were linked to personal information only by a number in the analysis dataset 

that was linked in a separate document (source form) with just personally identifying information 

and the common ID. This source form is kept locked in a separate file cabinet by a different 

researcher in another department. First, the researcher created two datasets that were retained in 

separate, secure, and confidential locations. Only a participant’s Common ID number was included 

in the two files. The researcher had access to the de-identified data for the purpose of conducting 

ongoing analyses. Control of the original file with name and basic demographic characteristics was 

released by the researcher and secured in a location to which the researcher had no direct access 

without securing IRB approval. The actual surveys, after all identifying information (usually only 

names and addresses) was marked out, were deposited into UTSA’s long-term storage facilities. 

At the start of the current study, the researcher was given access to an Excel file including 

only the name and birthdate of individuals in the second dataset and which included no ID code 

linked to the Principal Investigator’s data file. Following the conclusion of the original data 

collection for the original study, the above efforts were made to ensure the confidentiality of 

participants but also to retain information necessary for the follow-up study in the following way. To 

initiate the present study, the Principal Investigator requested from the other researcher holding the 

source form a copy of the names and birthdates on the source form (once stripped of the Common 

ID). Using only the names and birthdates, two graduate students working for the Principal 

Investigator conducted searches for crime and education data using online databases. Once data was 

retrieved from online sites, the students created a second random ID code linked to each participant’s 

name, deleted the collected outcome data from a copy of the file, and delivered this copy back to the 

independent researcher holding the source form. Using the names in the returned file, that 

researcher merged the Common ID with the names, deleted the names, and returned to the Principal 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 42

Investigator a file with the Common ID and secondary ID associated. Through this process, 

names/DOBs of participants from the original source forms were not linked by (nor known to) the 

original researcher at any point, securing that the dataset gathered for the present study (publicly 

available data on crimes and educational activity) remained de-identified and confidential. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 43

FINDINGS 

This section reports three related but distinct sets of findings, each building on the previous 

set of findings. First reported are the intent to treat analyses of the effects of participation in the CIS 

mentoring program on the probability of criminal activity and the pursuit of post-secondary 

education through age 21. Following the identification and description of these long-term findings, 

simple correlational analyses are reported that link mentoring experiences with these long-term 

outcomes. Finally, more complex analysis findings of theory-driven tests of the interactions between 

what happened in the mentoring relationships, how participants felt about their relationships, and 

variation among mentees in these long-term outcomes are described. 

The effect of participation in the school-based youth-mentoring program on criminal activity 

in early adulthood was the first outcome examined as it is probably the most important outcome for 

the Department of Justice. In an effort to directly test whether effects observed in the original study 

were sustained over time, the three-way test of the interaction of treatment group by sex and grade 

level was attempted as well as the main-effect between-group intent to treat analyses without these 

moderators. The moderator tests, as well as general linear models, require cell count representation 

that was not obtained in the data; however, as evident by the rare frequency of crimes in general, but 

especially for violent and property crimes. For that reason, these analyses were deemed unreliable, 

and the study focused primarily on the presence of long-term effects rather than the maintenance of 

differences across subgroups over time. Misdemeanors were committed at a rate sufficient to have 

group counts of at least 5 when accounting for both sex and treatment, which allowed logistic 

regression to be used for group comparisons. In order to not overgeneralize findings to other crimes, 

such as arrests for felony violent and property crimes, which were reported so infrequently, those 

these outcomes were excluded from the final analysis. As a result the study focused solely on the 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 44

likelihood of being arrested for a misdemeanor by age 21. While these tests of the main effect of 

program participation on the likelihood of misdemeanor arrests do not replicate the three-way 

interaction of the original study, they are superior in that they provide a stronger test of program 

impact by providing a true experimental test without distraction by what could be considered quasiexperimental subgroup comparisons across grade levels. 

Subsequently, in the second set of analysis of long-term effects of mentoring program 

participation on educational pursuit in early adulthood, the same approach was taken to estimate 

long-term effects of school-based mentoring program participation on the likelihood of pursuit of 

any form of post-secondary education. Just as main effect tests for differences in the probability of 

arrest for a misdemeanor between treatment and control groups were given the most attention—i.e., 

because this type of crime had the greatest frequency and could be reliably tested for boys and girls 

independently (testing main effects of treatment after accounting for main effect of sex on crime 

rates), which was necessary given the oversampled boys in the treatment group and 3:2 ratio of girls 

to boys in the sample overall—the most reliable analyses of long-term effects on post-secondary 

education were on the likelihood of engaging in post-secondary education of any kind within the 

first five years beyond high school. 

Goal 1: Long-term Outcomes of School-based Mentoring 

First long-term outcome: Arrests for crime by age 21. The first analysis of long-term 

effects of mentoring program participation on crime in early adulthood was an attempt to replicate 

the three-way interaction reported in the original study. (Presenting the three-way interaction before 

the main effect analyses is unconventional, but serves the purpose of first responding to the funded 

objective of determining whether the effects first reported by Karcher (2008) persisted into 

adulthood.) The three-way interaction analyses reported by Karcher revealed differential effects of 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 45

mentoring on short-term outcomes, especially between elementary and high-school-age boys, 

wherein the high school boys worsened while the elementary boys improved on outcomes as a 

consequence of program participation. Using general linear modeling in SPSS 24, the first analysis 

used Poisson regression to run this same model to estimate between-group differences in number of 

arrests for any crime. The Poisson regression transformed the highly skewed count data to allow 

tests of mean differences in the rates of having been arrested for any type of crime. 

Although the model test statistics indicate there was a statistically significant three-way 

interaction of treatment differences across sex and grade levels (Wald Chi-Square = 7.08, p < .01), 

there was also evidence from model fit indices that this model may not have been completely 

reliable. The problem appeared to be a consequence of there being no criminal arrests for females in 

the mentoring condition who were elementary age girls at the age of mentoring (Table 3.1), which 

resulted in singularity in the Hessian Matrix, rendering the model statistics and estimates unreliable. 

The descriptive statistics, however, do provide an answer to the question of whether longterm iatrogenic effects had persisted and were manifest in records of adult criminal activity. 

Observations from the rates of crime across groups reveal that, unlike in the original study, 

differences between older and younger boys in terms of overall arrest rates were not consistent with 

the initial report (Karcher, 2008). Results of these follow-up analyses indicate that there is no 

evidence of long-term iatrogenic effects of intervention for those mentored as older boys and, quite 

possibly, suggest greater benefits for boys mentored in high school than boys in elementary school, 

when each is compared to their same grade-level peers in the control group. However, the women 

who were mentored during high school showed no long-term benefit of participation in the 

mentoring program on crime activity in adulthood. It should be noted, this is the subgroup of study 

participants in the treatment condition least likely to have actually been matched with a mentor 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 46

because none were available for them. This is not the type of non-compliance typically observed in 

treatment studies as those who refuse treatment; rather, it is the high school girls who were most 

likely to never have been given the opportunity for treatment (i.e., to meet with a mentor) and they 

are, therefore, least likely to reflect evidence of treatment on the treated. 

Table 3.1: Mean Rate of Arrest for Any Crime (Misdemeanor and Felony) Committed by Individuals 

by Sex, School Level, and Intervention Group 

Treatment 

Group Sex Grade Group Mean 

Std. 

Error 

95% Wald CI 

Lower Upper 

Non-Mentee Female Elementary .10 .049 .04 .26 

Middle School .17 .059 .08 .33 

High School .04 .022 .01 .12 

Male Elementary .25 .125 .09 .67 

Middle School .58 .149 .35 .96 

High School .68 .176 .41 1.13 

Mentee Female Elementary .00 .000 .00 .00 

Middle School .10 .049 .04 .26 

High School .11 .039 .05 .22 

Male Elementary .18 .080 .07 .43 

Middle School .61 .141 .39 .96 

High School .13 .063 .05 .33 

To better estimate the direct effects of mentoring, age was included as a control to address 

school-level variability, and the interaction of sex and treatment was examined. Given the 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 47

unreliability of the prior model, the model was re-run including age as a continuous covariate, rather 

than categorical school level, to test the interaction of sex and treatment. In this Poisson general 

linear model, a main effect of mentoring on mean number of arrests for any crime was found that did 

not vary as a function of sex (the interaction term was not significant). This model, however, 

included arrests for all crimes, which is problematic because of the infrequency of the more serious 

crimes; this would make an unconvincing general statement about mentoring’s effect on criminal 

activity which is unwise based on these analyses. Therefore, subsequent analyses examined only 

misdemeanor crimes, for which there had been the largest number of arrests. 

The time frame for the outcome variable also was narrowed to the likelihood of an arrest for 

a misdemeanor crime that occurred only within the period between ages 17 and 21 for all 

participants. This was done as a way to address the differences in outcomes that may result from 

those in later grades having more time as adults in which to be arrested. By looking at rates of arrest 

for all youth by age 21, which was the age of the youngest program participants when the crime 

record data was accessed in 2016, the amount of time each participant had to accrue an arrest record 

was held constant. 

Finally, subsequent analyses looked at the likelihood of any arrest, rather than total number 

of arrests by age 21. Narrowing the time frame and focusing only on the experience of being arrested 

(versus the number of arrests) was both a more logical and statistically sound approach to analysis. 

The response variable of whether or not the individual had been arrested for a misdemeanor is a 

naturally occurring and meaningful outcome, and also one that was deemed more appropriate for this 

study given the low base rate and need to account for anticipated sex differences. Using the total 

number of arrests required severe data transformations to correct positively skewed count data, but 

also thrust significant bias into the model by the heavy influence of a few individuals with multiple 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 48

arrests. These low arrest counts require minimal subgroup categories in order to avoid 

overdispersion. Therefore, viewing misdemeanor arrests as a dichotomous categorical experience 

was the most theoretically and statistically sound approach. 

Table 3.2 reveals both actual arrests and the odds of arrest for each group. Odds of arrest can 

be interpreted as the number of individuals who were arrested divided by the number of individuals 

who were not arrested in each group. (This is not the probability, which would be the number of 

those arrested divided by the total number of people in that group.) Results indicate that among those 

in the control group, the odds of being arrested for a misdemeanor by age 21 (Odds of Arrest = .069) 

were more than twice the odds of arrest among those in the treatment group (Odds of Arrest =.031; 

see Table 3.2). Mantel-Haenszel OR = .446, CI: .178—1.12; Eta = .082). The resulting odds ratio is 

.45 (or .031/.069), with odds of arrest 55% lower for mentees than control youth. Nevertheless, this 

number is confounded by the disproportionate numbers of boys in the treatment condition and rates 

of arrest (see Table 3.1). 

Table 3.2. Number of Misdemeanor Arrests by Age 21 for Youth in Treatment vs. Control Groups 

Intent to Treat Group 

Non-Mentee Mentee Total 

Arrested for Misdemeanor by Age 21 No 217 227 444 

Yes 15 7 22 

Total 232 234 466 

Odds of Arrest (Probability of Arrest) .069 (.065) .031 (.030) 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 49

The treatment group differences in odds of arrest for a misdemeanor by age 21 were in the 

same direction, favoring mentees, for both boys and girls (see Tables 3.4 and 3.5). The differences 

in odds of arrest were somewhat larger for among boys; for whom the odds of being arrested for a 

misdemeanor for those in the mentoring group was 67% less than for those in the control group 

(Mantel-Haenszel OR = .322, CI: .09-1.12; see Table 3.4). 

Table 3.4: Number of Misdemeanor Arrest by Age 21 for Males in Across Treatment Groups 

Intent to Treat Group 

Control Treated Total 

Arrested for Misdemeanor by Age 21 No 56 87 143 

8 4 12 

Total 64 91 155 

Odds of Arrest (Probability of Arrest) .14 (.125) .046 (.044 ) 

Yes 

Figure 3.1. Probability of Arrest for a Misdemeanor by Sex in Each Treatment Condition 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 50

The probability of misdemeanor arrest for boys in the mentoring program was one-third the 

likelihood for boys in the control group and the likelihood of arrest for girls in the mentoring 

program was half that of control group girls (See Figure 3.1). Among girls, the odds of being 

arrested for a misdemeanor for those in the mentoring group were 50% less than for those in the 

control group (Mantel-Haenszel OR = .493, CI: .125-2.08)(Eta=.058)(See Table 3.5). 

Table 3.5: Number of Misdemeanor Arrests by Age 21 for Females in Across Groups 

Intent to Treat Group Total 

Non-Mentee Mentee 

Arrested for Misdemeanor by Age 21 No 161 140 301 

Yes 7 3 10 

Total 168 143 311 

Odds of Arrest (Probability of Arrest) .043 (.042) .021 (.021) 

To account for the over-representation of boys in the mentoring group, a logistic regression 

model including treatment group membership and sex as predictors was estimated. Including no 

other covariates in estimating probability of being arrested for a misdemeanor by age 21, both the 

magnitude of treatment condition and sex were statistically significant. The results reveal reduced 

odds of arrest for a misdemeanor associated with being assigned to the mentoring program, after 

accounting for the over-representation of boys in the treatment group. By including sex in the model 

(Table 3.3), we see the adjusted odds ratio is .387. In other words, controlling for differences in 

proportion of boys and girls in each group, the odds of arrest for a misdemeanor by 21 were reduced 

by approximately 61% by being assigned to the mentoring program. This logistic regression model 

provides the most direct test of the benefits of mentoring in terms of reductions in the probability of 

committing a crime. 

Final Technical Report: 2013-JU-FX-0008 
Final Technical Report: 2013-JU-FX-0008 51

Table 3.3: Main Effects of Treatment Condition Assignment and Sex on Probability of Misdemeanor Arrest 

B (S.E.) Wald Sig. 

Odds Ratio 

Exp(B) 

95% C.I. for Exp(B) 

Lower Upper 

Treatment -.95 (.48) 3.99 .046 .387 .15 .98 

Sex 1.05 (.45) 5.56 .018 2.87 1.19 6.88 

Constant -2.96 (.42) 49.99 .000 .052 

Reference Categories: Treatment=Mentees, Sex=Males. Wald (df = 1) 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In terms of meeting required statistical assumptions, it was clear that the main effect logistic 

regression model above, including only sex and treatment as main effects with no school level 

variable, provides a more trustworthy estimate of program impact than the planned three-way 

interactions (paralleling the Karcher, 2008 study). Including sex and treatment group status allowed 

the more straightforward use of logistic regression analysis of the logit link (Fox, 2008), because 

excluding a factor for school level eliminated risks posed by zero-cell counts for absent arrests 

(O’Connell & Amico, 2010, see Table 3.1). Additionally, without additional continuous covariate 

predictors, threats of multicolinearity and linearity were minimized. Further evidence of this model’s 

meeting key assumptions was revealed by leverage and residual statistics. All DFBeta values were 

less than .20, Cook’s distances less than .22, and no leverage values were above .15. Only four 

normalized residuals for the sample of 466 exceeded 1.98, or less than 1%. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 52

Table 3.6: Effects of Treatment and Sex on Probability of Misdemeanor Arrest Adjusted for Age and Grades 

95% C.I. for EXP(B) 

B (S.E.) Wald Sig. Exp(B) Lower Upper 

Treatment -1.12 (.49) 5.02 .025 .33 .12 .87 

Sex 1.03 (.46) 5.00 .025 2.80 1.14 6.912 

Starting Math -.06 (.020) 10.09 .001 .94 .90 .98 

School Level -.99 (.51) 3.84 .050 .37 .14 1.00 

Constant 2.17 (1.58) 1.89 .170 8.73 

Reference Categories: Treatment=Mentees, Sex=Males. School Level=High School. Wald (df = 1). 

Nevertheless, to provide one more test of program participation effects that includes 

additional individual explanatory variables, another model that included school level and the only 

other covariate (starting math grades) found to be associated with the outcome is presented in Table 

3.6. This model confirms the main effects of mentoring on the likelihood of misdemeanor arrests and 

indicates the effects are slightly stronger (i.e., mentees 67% less likely to be arrested) when 

variability due to starting levels of academic skills are considered and school level of participants are 

accounted for. Risk of arrest was also lower for those in high school during the study and was 

negatively associated with math grades prior to randomization (See Table 3.6). 

Survival Analysis Cumulative Hazard Rate 

The proportion of study participants who had not yet been arrested for a misdemeanor by age 

21 is illustrated in Figure 3.2, which presents a Cumulative Survival rate for each month following 

the year the youth turned 17 and could begin to have a public criminal record. Between-group 

differences in arrest rates for misdemeanor crimes between age 17 and 21 were estimated using nonparametric Cox (logistic) regression in SPSS 24 and are presented below in terms of survival 

analysis (Kaplan-Meier) hazard functions (see Figure 3.3, Table 3.7, 3.8., and 3.9). 

Final Technical Report: 2013-JU-FX-0008 
 

 

Final Technical Report: 2013-JU-FX-0008 53

Figure 3.2. Cumulative Survival Rate by Month Starting at Age 17 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 54

Table 3.7. Number of Individuals Arrested (Events) Vs. Censored 

Sex 

Intent to Treat 

Group Total N 

N of 

Events 

Censored 

N Percent 

Female Non-Mentee 168 7 161 95.8% 

Mentee 143 3 140 97.9% 

Overall 311 10 301 96.8% 

Male Non-Mentee 64 8 56 87.5% 

Mentee 91 4 87 95.6% 

Overall 155 12 143 92.3% 

Overall Overall 466 22 444 95.3% 

Table 3.8. Means and Confidence Intervals for Survival Time by Group and Sex 

Sex Intent to Treat Group Estimate Std. Error 

95% Confidence Interval 

Lower Bound Upper Bound 

Female Non-Mentee 170.780 2.312 166.249 175.310 

Mentee 168.161 1.632 164.961 171.361 

Overall 172.277 1.477 169.382 175.171 

Male Non-Mentee 150.031 5.956 138.357 161.706 

Mentee 169.308 3.288 162.863 175.753 

Overall 164.239 3.271 157.827 170.650 

Overall Overall 169.910 1.482 167.006 172.814 

a. Estimation is limited to the largest survival time if it is censored. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 55

The length of time before average onset of an arrest was significantly greater for the youth in 

the mentoring condition, with odds of survival 1.5 times greater for the treatment group (OR = 2.5). 

Table 3.9. Survival Analysis Estimates and CI for Independent Variables in the Equation 

B SE Wald df Sig. Exp(B) 

95.0% CI for Exp(B) 

Lower Upper 

Intent to Treat Group .925 .461 4.017 1 .045 2.521 1.021 6.227 

Sex 1.027 .432 5.660 1 .017 2.792 1.198 6.505 

The cumulative hazard (Figure 3.3) corroborates earlier findings of group differences in probabilities 

for risk of arrest and reveals the risk to be increasing over time for those in the control condition. 

Figure 3.3. 

Conditional Cumulative Hazard Function by Month Starting at Age 17 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 56

Second long-term outcome: Predicting post-secondary education. Tests of the effects of 

school-based mentoring on the likelihood of pursuing post-secondary education by age 22 (within 

five years after high school) were first conducted using logistic regression without any covariates. 

The omnibus test for the first model, which replicated the three-way interaction of the original study 

(Karcher, 2008), was not statistically significant and revealed the effects of mentoring did not vary 

across grade levels between the sexes. 

A second logistic regression model without school level as a variable was statistically 

significant, revealing a main effect for mentoring and sex. Table 3.10 shows a small difference 

between groups, with 33% of mentees compared to 30% of control group participants having 

enrolled in some form of post-secondary education (77/157 divided by70/162 = OR=.1.14). In other 

words, the odds of post-secondary pursuit were 14% greater for mentees, or that mentees were 10% 

more likely to pursue post-secondary education; however, there was a significant difference in the 

probability of enrollment as a function of the participants’ sex, with girls in the treatment/mentoring 

group showing a greater likelihood of pursuing post-secondary education (OR = 1.27). Including an 

interaction term for treatment group by sex, as well as several covariates, revealed that clearly girls 

benefitted the most from program participation in terms of their likelihood of long-term educational 

pursuit after high school (See Table 3.12). The confidence interval for treatment main effects 

generated from this same model with bootstrapping further suggests the benefits were restricted to 

girls (Table 3.13). 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 57

Table 3.10: Enrolled by Age 22 for Treatment Groups by Sex 

Sex 

Intent to Treat Group 

Non-Mentee Mentee Total 

Female Enrolled by 

Age 22 

.00 118 89 207 

1.00 50 54 104 

Total 168 (30%) 143 (38%) 311 

Male Enrolled by 

Age 22 

.00 44 68 112 

1.00 20 (31%) 23 (25%) 43 

Total 64 91 155 

Total Enrolled by 

Age 22 

.00 162 157 319 

1.00 70 77 147 

Total 232 (30%) 234 (33%) 466 

Gender differences in treatment effects were found in logistic regression estimates of 

differences in the probability of youth pursuing some form of post-secondary education (including 

certificate programs, trade school, college). These results were true both without and when 

including pre-intervention grades in math and reading as covariates as well as parent-rated preassignment problem behaviors. The interaction of sex and treatment group was statistically 

significant in both models. Post-secondary educational enrollment was higher for girls in the 

mentoring (38%) vs. control (30%) condition. The largest difference was for the 70 girls in 

elementary school, wherein 59% of girls in the mentoring condition had enrolled but only 37% of 

the control group girls had (see Table 3.11). For boys, the difference in the percent of mentored 

Final Technical Report: 2013-JU-FX-0008 
Final Technical Report: 2013-JU-FX-0008 58

Final Technical Report: 2013-JU-FX-0008 

(25%) and control group (31%) youth enrolled in post-secondary education by age 22 favored the 

control group but was not statistically significant (see Table 3.11). 

Table 3.11: Number Enrolled by Age 22 for Treatment Group By Sex and School Level 

Sex Grade Group 

Number Per Group 

Total 

Control 

(Percent) 

Treatment 

(Percent) 

Female Elementary Enrolled by 

Age 22 

No 26 12 38 

Yes 15 (37%) 17 (59%) 32 

Total 41 29 70 

Middle 

School 

Enrolled by 

Age 22 

No 33 26 59 

Yes 15 (31%) 15 (37%) 30 

Total 48 41 89 

High 

School 

Enrolled by 

Age 22 

No 59 51 110 

Yes 20 (25%) 22 (30%) 42 

Total 79 73 152 

Male Elementary Enrolled by 

Age 22 

No 11 22 33 

Yes 5 (31%) 6 (21%) 11 

Total 16 28 44 

Middle 

School 

Enrolled by 

Age 22 

No 18 23 41 

Yes 8 (31%) 8 (26%) 16 

Total 26 31 57 

High 

School 

Enrolled by 

Age 22 

No 15 23 38 

Yes 7 (32%) 9 (28%) 16 

Total 32 54 

 

 

 

 

 

 

 

 

 

 

 

 

22 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 59

Table 3.12: Logistic Regression Predicting Post-Secondary Enrollment (with Covariates) 

95% C.I. for EXP(B) 

B (S.E.) Wald Sig. Exp(B) Lower Upper 

Sex .550 (.342) 3.094 .079 1.73 .939 3.199 

Math Grades (Pre) .034 (.012) 8.056 .005 1.04 1.011 1.060 

Reading Grades (Pre) .040 (.014) 8.004 .005 1.04 1.012 1.070 

Connors Global Index -.859 (.280) 9.394 .002 .42 .245 .734 

Treatment Group .508 (.257) 3.902 .048 1.66 1.004 2.752 

Treatment by Sex -.912 (.464) 3.862 .049 .40 .162 .998 

Constant -6.186 (1.20) 26.465 .000 .00 

Reference Categories: Treatment=Mentees, Sex=Males. Wald (df = 1). 

Table 3.13: Bootstrap Logistic Regression Predicting Post-Secondary Enrollment (with Covariates) 

B Bias S.E. Sig. 

95% C.I. 

Lower Upper 

Sex .550 .003 .328 .081 -.099 1.176 

Math Grades (Pre) .034 .000 .013 .012 .009 .062 

Reading Grades (Pre) .040 .001 .016 .010 .011 .072 

Connors Global Index 

(Parent Report Pre) 

-.859 -.024 .289 .002 -1.505 -.319 

Treatment Group .404 .001 .384 .279 -.305 1.179 

Treatment by Sex -.912 -.011 .471 .047 -1.902 -.035 

Constant -6.228 -.114 1.185 .001 -8.721 -4.068 

Reference Categories: Treatment=Mentees, Sex=Males. Wald (df = 1). 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 60

Goal 2: Identify Relationships Between Mentoring Interactions, Relationship Styles, and the 

Likelihood of Arrest in Adulthood 

The second goal of this project was to better understand whether and perhaps how mentoring 

interactions and relationship styles contribute to long-term outcomes of school-based mentoring. 

Used to guide the analyses in this section, the TEAM Framework (Karcher & Nakkula, 2010) was 

both derived from findings of two pioneering studies of community-based mentoring, and further 

developed through an application of those findings to the early analysis of activity data derived from 

the present study summarized below (Avera et al., 2014; Karcher, Benne et al., 2006). 

The analyses in this section first consider bi-variate correlations among types of mentorreported activities, youth-reported assessments of relationship quality, and long-term outcomes 

(misdemeanor arrest and post-secondary enrollment by age 21). Bivariate correlations, both zeroorder and partial (controlling for factors that might influence the selection of specific mentoring 

activities—namely, pre-match grades, attendance, problem behaviors, age and sex) were used to test 

whether relationship-building discussions and fun activities (hallmark “mentoring” interactions) 

were more strongly associated with long-term outcomes than the academically and problem-oriented 

discussions and homework that more commonly occur in school-based mentoring settings. 

Then an attempt was made to test the TEAM framework hypotheses that (a) the combination 

of relationship-building discussions and goal-directed activities best predicts long-term outcomes 

and (b) their interactional quality varies as a function of the youth’s report of relationship quality. 

These analyses focused on the degree to which the youth felt he or she mattered to the mentor as the 

indicator of relationship quality. One set of analyses and graphical output from PROCESS analyses 

run in SPSS 24 tested the moderation of the degree to which the relationship between mattering and 

the likelihood of arrest by 21 varied as a function of the extent of relational or problem discussions. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 61

The mentoring activities indices reported first in correlations reflect aggregates of the total 

data collected into four theoretically derived interaction categories—the same as presented in Figure 

1 to illustrate the TEAM Framework dimensions. Earlier factor analyses of the individual match 

interactions described in Table 1 (and listed, as reported by mentors in the activity log, in Appendix 

A of this report) use data from the original study (Karcher, Benne et al. 2006). These factor analyses 

revealed the presence of two general purposes of mentoring interaction (serving adult/future or 

youth/present purpose), as well as two types of interaction focus, either a relationship focus or a goal 

or problem focus. (These are the horizontal and vertical dimensions, respectively, in Table 1.) 

However, factor analyses also differentiated discussions from activities, resulting in four 

categories—Playing, Talking, Learning, and Doing—that are central to the TEAM framework 

(Karcher & Nakkula, 2010) and which were used to identify the aggregate categories of interaction 

used in correlations presented in this chapter. Each category was further identified as either 

developmental or instrumental to indicate the type of interaction that typically happens first in each 

of these two relationship styles. In the developmental style, social discussions and playful activities 

are focused on first and constitute a relationship-building focus initially: Developmental Activities 

such as “Play” (games, sports) and Developmental Discussions such as “Talk” (casual conversation, 

discussions of specific relationships). The instrumental-style relationships start with a skilldevelopment focus or a problem-remediation purpose: Instrumental Activities are represented by 

doing homework or tutoring (“Doing”) and Instrumental Discussions focus on the conventional, 

adult purpose of solving problems, school misbehavior, and poor attendance and grades 

(“Learning”). The mentoring interaction data used in these analyses are from responses on activity 

logs completed by mentors after every visit (see Table 1) recording the time spent in each of several 

different kinds of interaction (typically ranging from 30 to 75 minutes a meeting). (See Appendix A 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 62

for Activity Log indicators and names, and Appendix B for Playing, Talking, Learning, and Doing 

Variable Coding based on combinations of Activity Log tallies). The total time for activities and 

discussions in each category for the two relationship style types was divided by the total hours the 

youth was mentored to create an average “percent time” spent on each. These were used for 

estimating associations between activity frequency and several measures of both relationship quality 

and long-term program outcomes in correlational analyses (both zero-order and partial correlations). 

Bivariate and partial correlations examined the relationships among specific mentoring 

activities engaged in, the youth-reported relationship quality, and several program experiences 

assessed ten years prior, and their association with the two long-term outcomes. These correlations 

yielded several significant and theoretically consistent associations (See Table 3.14). The pattern of 

correlations associating mentoring activities and mentee experiences with long-term outcomes most 

closely replicates prior findings by Morrow and Styles (1995) revealing the utility of the 

developmental style, but also support for Hamilton and Hamilton’s (2005) instrumental style. It is 

consistent with Wampold et al.’s (1997) meta-analysis regarding the efficacy of different 

psychotherapy approaches, and his use of a line from Alice in Wonderland, “both have one and both 

shall have prizes.” 

Consistent with the developmental style advocated by Morrow and Styles (1995), which at 

first prioritizes conversations through which relationships are built, we see that zero-order 

correlations between time spent in developmental discussions were most predictive of avoiding 

arrest and pursuing post-secondary education ten years later for all mentees. Table 3.14 reveals 

statistically significant negative correlations with likelihood of arrest for misdemeanors (r =-.19) and 

increases in likelihood of post-secondary educational pursuit (r =.17). 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 63

Table 3.14: Zero-order and Partial Correlations Between Long-term Outcomes, Mentoring Interactions and Relationship Quality 

1 2 3 4 5 6 7 8 9 10 11 12 13 

1. Arrested 1 .03 -.05 -.08• .06 -.16* .04 .09 -.07 -.08 -.15* -.09 -.16* 

2. Enrolled .00 1 -.02 .07 .03 .19** -.09 -.18** -.08 .04 -.05 -.05 -.03 

3. Total CIS hrs. -.07 .01 1 .25** -.03 -.07 .11 -.01 .08 .05 .13• .08 .10 

4. Mentoring hrs. -.08 .07 .25** 1 -.25** .05 .20** -.05 .25** .22 .27** .30** .21** 

5. Learning .05 -.10 -.09 -.23** 1 -.31** -.48** -.10 -.13• -.12• -.10 -.01 -.17* 

6. Talking -.19** .17* -.07 .02 -.12 1 -.44** -.41** .02 .08 .10 .05 -.02 

7. Playing .07 .02 .13 .19** -.62** -.52** 1 -.22** .07 .07 .04 -.01 .15* 

8. Homework .08 -.14* .02 -.04 -.17* -.40** -.08 1 .03 -.05 -.08 -.05 .00 

9. Valued -.08 .00 .11 .24** -.24** -.02 .17* .07 1 .72** .69** .68** .72** 

10. Motivated -.08 .08 .07 .22** -.12* .04 .12 -.02 .72** 1 .70** .56** .62** 

11. Matter -.16* .03 .16* .26** -.22** .07 .14 -.03 .70** .70** 1 .72** .71** 

12. C_Mentor -.09 .01 .10 .29** -.11 .02 .08 -.02 .70** .58** .730** 1 .58** 

13. C_Program -.16* .02 .13 .22** -.26** -.06 .23** .05 .71** .62** .72** .58** 1 

Notes. • p < .10, * p < .05, ** p < .01, Full labels: 1. Arrested for Misdemeanor by Age 21; 2. Enrolled in Post-Secondary Ed. by Age 21; 3. Total CIS 

services (excl. mentoring); 4. Total hours of Mentoring; 5. Percent of Time in Instrumental Discussion; 6. Percent Time in Developmental Discussion; 7. 

Percent Time in Developmental Activities; 8. Percent Time Doing Tutoring or Homework; 9. Mentee feels valued; 10. Feels motivated by mentor; 11. 

Matter to mentor; 12. Connectedness to Mentor ; 13. Connectedness to Program; Below the diagonal are zero-order correlations, and above the diagonal 

are partial correlations controlling for age, sex, and initial problem behaviors, attendance, and grades. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 64

Table 3.15: Partial Correlations Between Long-term Outcomes, Mentoring Interactions and Relationship Quality by Sex 

1 2 3 4 5 6 7 8 9 10 11 12 13 

1. Arrested 1 .00 -.05 -.06 .07 -.16• -.07 .22** .08 .02 -.02 .04 -.04 

2. Enrolled .07 1 -.03 .13* .02 .25** -.15• -.18* .02 .14 -.01 -.01 .05 

3. Total CIS hrs. -.09 .01 1 .25** -.03 -.10 .16• -.02 .05 .02 .15 .05 .06 

4. Mentoring hrs. -.12 -.08 -.12* 1 -.27** .02 .30** -.09 .27** .23** .34** .31** .20* 

5. Learning .06 .07 -.10 -.22• 1 -.39** -.40** -.10 -.12 -.16• -.15 -.02 -.17•

6. Talking -.16 -.03 -.09 -.02 -.22• 1 -.39** -.47** .01 .07 .12 .10 .01 

7. Playing .11 .09 .11 .16 -.57** -.51** 1 -.21** -.01 .06 .03 -.09 .07 

8. Homework -.06 -.23* .06 .05 -.07 -.23* -.30* 1 .123 .001 -.04 -.004 .07 

9. Valued -.24* -.27* .15 .22* -.12 -.02 .18 -.13 1 .71** .65** .66** .67** 

10. Motivated -.18 -.13 .14 .25* -.05 .03 .10 -.15 .71** 1 .68** .53** .61** 

11. Matter -.28* -.12 .08 .13 -.01 .03 .06 -.14 .74** .73** 1 .74** .72** 

12. C_Mentor -.24* -.13 .15 .31** .02 -.11 .12 -.09 .69** .60** .70** 1 .55** 

13. C_Program -.29* -.18 .21• .24* -.15 -.15 .28* -.09 .80** .61** .67** .58** 1 

Notes. • p < .10, * p < .05, ** p < .01, Full labels: 1. Arrested for Misdemeanor by Age 21; 2. Enrolled in Post-Secondary Ed. by Age 21; 3. Total CIS 

services (excl. mentoring); 4. Total hours of Mentoring; 5. Percent of Time in Instrumental Discussion; 6. Percent Time in Developmental Discussion; 7. 

Percent Time in Developmental Activities; 8. Percent Time Doing Tutoring or Homework; 9. Mentee feels valued; 10. Feels motivated by mentor; 11. 

Matter to mentor; 12. Connectedness to Mentor ; 13. Connectedness to Program; Below the diagonal are correlations for boys, above the diagonal are for 

girls. Both are controlling for age, sex, and initial problem behaviors, attendance, and grades. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 65

Partial correlations with these two long-term outcomes, controlling for initial rates of 

misbehavior, grades in reading and math, absences, age and sex, remain statistically significant 

(rpartial =-.16, rpartial =.19, Table 3.14 above diagonal). 

Gender differences in the association between frequency of developmental conversations and 

long-term outcomes were also observed. Looking at just the women (Table 3.15), the partial 

correlations with later arrests (r =-.16,) and post-secondary pursuits (rpartial = .25) were the same as 

for the full sample (Table 3.14), suggesting that women whose matches engaged in more 

relationship-building discussions were less likely to be arrested for a misdemeanor in adulthood and 

were more likely to pursue post-secondary education. For men, the negative associations (rpartial = -

.16) between developmental conversations and later arrest were similar in size and direction to the 

full sample, but did not reach statistical significance; however, there was no evidence of a direct 

association between developmental conversations and post-secondary pursuits rpartial = -.03). 

Time spent doing homework was not associated with good long-term outcomes. For all youth 

(both sexes combined, Table 3.14), time spent doing homework was negatively associated with later 

pursuit of post-secondary education (r = -.14). Even controlling for differences in mentees’ initial 

grades, attendance, school behavior, sex, and age, the negative relationship between time spent on 

homework and post-secondary pursuits remained statistically significant (rpartial =-.18). For women 

separately (Table 3.15), partial correlations between time doing homework and both later arrest 

(rpartial =.22) and post-secondary enrollment (rpartial =-.18) ten years later were statistically significant, 

revealing iatrogenic effects of doing homework with girls. For the men, doing homework was also 

negatively associated with later post-secondary pursuits (rpartial =-.23), though not with arrests. 

Problem-focused conversations (Learning or “instrumental discussions”) were not directly 

associated with either long-term outcome in zero-order or partial correlations for the mentees when 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 66

women and men were both included in analyses (Table 3.14). For men, despite the fact that a zeroorder association with later pursuit of post-secondary education was found (r =-.14), it was not 

statistically significant, and the relationship was entirely absent in partial correlations (r = .02), 

suggesting such conversations may have been related to other problems (e.g., problem behaviors) 

that explained the link to long-term outcomes in the zero-order correlation. There were no direct 

associations for men. However, this does not mean they had no influence on the match. 

Problem-focused “learning” discussions (instrumental discussions) were negatively 

correlated with total hours of mentoring (r =-.23; rpartial = -.25), both for the whole sample as well as 

for women (r = -.24, rpartial = -.27) and men (r = -.22, rpartial = -.22) separately (Zero-order correlations 

are in Appendix F and Partial correlations for men and women in Table 3.15). Thus, for all youth, 

regardless of (i.e., controlling for) the perceived need to discuss problematic grades, attendance, or 

school behavior, such discussions predicted shorter relationships, suggesting problem-focused 

conversations may have contributed to poorer quality relationships. The statistically significant 

negative associations between the “Learning” interactions and most process measures of mentees’ 

views of the program and of relationship quality support this assertion. 

Time spent playing games (“developmental activities”) was positively related to total hours 

of mentoring (r =.19)—so predictive of a longer relationship in general—regardless of the child’s 

age, grades, attendance, and problem behaviors (rpartial =.20), suggesting it was relationshipenhancing, even if no direct association with long-term outcomes was observed. Multiple positive 

associations between playing and mentees’ feeling valued in the match (r = .17, rpartial = .07) and 

being more connected to the CIS mentoring program (r = .23, rpartial = .15) support this interpretation. 

Mentees’ reports of feeling they mattered to their mentors (r =-.16, rpartial =-.15) and the 

strength of their connection to the CIS mentoring program (r =-.16, rpartial =-.16) were negatively 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 67

associated with adult arrest by age 21, but no such associations were found with pursuit of postsecondary education. However, these associations were moderated by sex (see Table 3.15). For men, 

all of the relationship process measures were negatively related (in partial correlations) to being 

arrested by age 21, and three measures were significantly associated with longer matches; for 

women girls, none of the four relationship quality measures nor the program connection measure 

were associated with either long-term outcome, even though all were positively related to total hours 

of mentoring. 

As a whole, these correlations support the value of the developmental model, which places a 

priority on friendship-building initially. Developmental discussions, focused on learning about the 

mentee, were predictive of positive adult outcomes. Conversely, time spent doing homework was 

not. Regardless of the need (as evidenced by the corroborating zero-order and partial correlations), 

the youth whose mentors acted like tutors were less likely to be drawn to higher education. 

Similarly, although not related directly to long-term outcomes, having fun (“developmental 

activities”) predicted longer match length, while problem-focused conversations predicted shorter 

matches, again, controlling for whether or not the child’s behavior, grades, or attendance were 

suggestive that play or problem conversations were needed by the youth. 

What these correlations could not do, however, is reveal the presence of any interaction 

between different activities and relationship processes on long-term outcomes. The TEAM 

Framework, building on the research that informed the “developmental” and “instrumental” styles, 

proposes that both goal-directed (problem-focused) and relationship-focused interactions are present 

in the most efficacious mentoring relationships, and that their interplay contributes to outcomes 

through changes in relationship quality. This hypothesis, however, cannot be tested with bivariate 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 68

correlations, but instead requires tests of how activities may shape or moderate the way relationship 

quality contributes to program outcomes. 

Testing the Interplay of Mentoring Interactions and Mentee Experiences on Outcomes 

To better understand the way some mentoring activities may moderate the influence of 

relationship experiences on long-term outcomes, and to test the hypothesis that they are interactive 

in nature, analyses in this section examined moderators of the relationship between mentoring 

relationship quality and long-term outcomes. Analyses here consider the frequency of relationshipbuilding and academically or problem-focused activities, and estimate how the frequency of these 

interactions may have moderated the way relationship quality affected long-term outcomes. 

Regression analyses using the SPSS macro PROCESS (Hayes, 2013) allowed regression 

analyses to be conducted that provide precise tests of significance of interactions across multiple 

moderators and reveal the strength and significance of the moderating effect at specific cut points of 

the moderator that reflect subgroups of respondents. The PROCESS macro facilitates the estimation 

of the contribution of a second moderator (for example, also measured and tested at three cut points) 

at specific points of the first moderator, and provides tests for significance wherein the difference is 

not zero. Cut points were the mean and one standard deviation above and below the mean. 

This is similar to a three-way interaction but is calculated in a manner that allows tests of the 

degree to which a second moderator reliably (Z) moderates the effect that the first moderator (W) 

has on the X-Y relationship at specific cut-points of W, even when the moderating effect of the 

second moderator is not evident at all levels of the first moderator. It reveals the way a moderator 

explains the interaction between two other variables for subgroups of participants, whereas typical 

significance tests of interaction variables test that the moderating interaction works the same way 

across all levels of the XY relationship. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 69

Table 3.16: Logistic Regression Coefficients for Main, Moderator, and Conditional Effects 

Variables: redob: Behavior or “Discussed youth’s behaviors that were related to problems with peers, teachers, adults, specifically 

misbehavior”; redoh: Getting to know the mentee or “Discussed mentee’s hobbies & interests, feelings, or mentee talked most of the 

time while mentor listened”. Cgi_P1: Connors Behavior Rating Scale, Parent Report; ssrsa1: Youth-reported social skills rating scale, 

assertiveness; sexfis0: Sex with female coded 0. Int_1 and Int_2 are activity by mattering moderator interaction terms. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 70

The model was run twice to rule out alternative variables that might account for this effect. 

First it was run with only the four variables, X, Y, W, and Z and no covariates; then it was run again 

including a number of covariates to rule out the likelihood that patterns of what happened in the 

match were simply proxies for other exogenous predictors or causal variables, namely the following 

characteristics: initial rates of problem behavior, social skills, sex, and age. Results were 

corroborated across runs; results with covariates are provided in Table 3.16. 

Results in Table 3.16 are from a test of whether multiple mentoring activities moderate the 

effect of program participation experiences on long-term outcomes through their separate and 

combined influence on mentees’ feeling they mattered to their mentors. For explaining how 

mentoring activities contributed to the negative association between feeling the mentee mattered and 

the likelihood of misdemeanor arrest in adulthood, a logistic regression model was constructed with 

mattering predicting likelihood of arrest by 21, controlling for youth’s self-reported social skills, 

parent-reported problem behaviors, age, and sex, and including two moderator variables, the 

frequency of time spent discussing school problems (redob) and time spent listening and learning 

about the mentee (redoh). Of the two moderating variables, the first (redob) was the frequency at 

which the match “Discussed youth’s behaviors that were related to problems with peers, teachers, 

adults, specifically misbehavior,” and it was a statistically significant predictor of later arrest (Y) and 

a moderator variable of the XY relationship. The second mentoring interaction (redoh) was not. This 

variable (the mentors’ report of frequency of time they “Discussed mentee’s hobbies & interests, 

feelings, or mentee talked most of the time while mentor listened”), however, was influential in its 

absence. 

Although only problem-focused discussion was found to be an unconditional moderator of 

the way mattering predicted later arrests (X2 = 6.60, p < .01), suggesting that it was at low levels of 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 71

mattering that problem-focused conversations most strongly predicted increased likelihood of later 

arrest, this effect was further conditioned by the absence of time spent getting to know the mentee. 

Conditional effects, listed in Table 3.16, reveal that it was when mentors spent little time 

getting to know the mentee (i.e., redoh = .00 = -1 SD) that the negative effect of a focus on problem 

behavior most strongly affected likelihood of later arrest. When little time was spent getting to know 

the mentee and the mentee felt like he or she did not matter to the mentor, high rates of problemfocused conversations (redob = .75 = + 1 SD), and controlling for differences in social skills, 

problem behaviors, sex, and age, predicted a higher likelihood of arrest (Beta = -5.30, p = .14). 

These multiple moderator (PROCESS Model 2) analyses, specifically the significant mattering x 

problem-focused conversations interaction term (Int1 in Table 3.16), suggest that mattering most 

strongly predicted later arrest when problem-behavior conversations were frequent, and this effect 

was further conditioned by time spent getting to know the mentee (variable redoh). With little time 

spent getting to know the mentee, frequent conversations about problems at school predicted a 

greatly increased risk for later arrest among kids who felt they did not matter much to their mentors. 

Final Technical Report: 2013-JU-FX-0008 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 73

probability of arrest was found in matches where mentors spent the least time getting to know the 

mentee (top panel), and frequently discussed school problems, as depicted by three sets of lines 

reflecting low, medium and high frequency of the first moderator (W, i.e., redob). 

Figure 3.4 shows that the X-Y relationship—viz. the degree to which mentees feeling they 

mattered to their mentors predicted the probability of being arrested for a misdemeanor by age 21— 

differed across three horizontal panels reflecting three rates of time the mentor spent getting to know 

the mentee. Lines represent three rates (frequency) of time spent focusing on school problems. The 

dash-dotted green-tipped line (“average” amount of time spent discussing problems), serves to 

indicate the difference in the probability of arrest that can be expected across the range of reported 

mattering at each of three cut points for low (- 1 SD), average and high rates (+1 SD) of time getting 

to know the mentee (top, middle, bottom panel). At an average frequency of problem focus, the 

probability line (at intercept) varies from 4% probability of arrest for matches with “little time” time 

spent getting to know the mentee and with low reported mattering (in the bottom panel of Figure 3.4) 

to 1% arrest probability for those in matches spending a lot of time learning about mentees (in 

bottom panel in Figure 3.4). This moderating effect is present, however, only when youth reported 

feeling they mattered very little to their mentors (-1 SD on X or 2.82), which is the left side of the X 

axis on each panel (i.e., those youth who said they felt they mattered little to their mentors). The 

probability of arrest was three times greater in highly problem-focused matches that spent little time 

(14% probability) compared to a lot of time getting to know the mentee (1% probability), among 

mentees who felt unimportant (i.e., low on X). When mentees felt they mattered to their mentors 

(high on X), the probability of a future arrest did not vary depending on either the amount of time 

spent getting to know the mentee. 

Final Technical Report: 2013-JU-FX-0008 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 75

Specifically, the likelihood of being arrested was over times higher (dashed, top line at the 

intercept at 14% probability of arrest) when their matches focused heavily on the youths’ problems 

than for youth in matches focusing an average amount on the mentees’ problems (dashed-dotted line 

ending in circles, meeting the intercept below 4% probability of arrest) that also were low in 

mattering and low in getting to know the mentee. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 76

IV. DISCUSSION

There was good reason to expect that if any outcomes from the original study had persisted, 

it would be the iatrogenic or harmful effects of school-based mentoring found initially or that these 

negative effects would have grown larger, but the opposite was true. What appeared to be solid 

evidence of harmful effects for the high school boys ten years prior seems to have been transformed 

into processes propelling positive youth development and successful adult outcomes. Also surprising 

was that the effects initially found, which varied across students of different ages and sex, also 

evened out over the ten-year period since the original study concluded. In this current study, no 

evidence of persistent iatrogenic effects was found; in contrast, even stronger positive effects of 

youth mentoring emerged that were surprisingly consistent across age and sex of study participants. 

Compared to the group of youth who had not been assigned to receive a mentor (even though some 

inadvertently did), the youth who were randomly assigned to meet with a school-based mentor 

(including those 24 in the treatment group who never did meet with a mentor) were less likely to be 

arrested for a misdemeanor and more likely to have pursued some form of post-secondary education 

by the age of 21. 

Female Mentees’ Higher Likelihood of Initiating Post-Secondary Education by Age 21 

Mentored girls benefitted more than boys in terms of long-term educational outcomes. Girls 

in the mentoring program were 27% more likely to have pursued some form of post-secondary 

education than girls in the control condition. There was a trend in the other direction for mentored 

boys, such that they tended to be less likely to pursue post-secondary education than those boys not 

assigned to receive a mentor. But this difference was not statistically significant and thus may reflect 

a chance finding. 

All Mentees’ Lower Probability of being Arrested for a Misdemeanor Crime by Age 21 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 77

The mentored youth were 55% less likely to have committed a misdemeanor by age 21. 

These effects were statistically larger than the typical program-impact estimates for youth mentoring 

on short-term outcomes like grades, attendance, and self-reported attitudes towards school or adults. 

Indeed, survival analyses suggest that the magnitude of the program’s impact increased with each 

passing year from age 17 to 21. After adjusting for proportional differences in boys and girls across 

treatment conditions, we see odds of being arrested was reduced by 61% from being assigned to the 

mentoring program, and fully 67% after accounting for initial, between-group differences in grades 

and problem behaviors as well as sex differences. 

The Relationship Between Mentoring Experiences and Long-Term Outcomes 

For both preventing criminality and fostering post-secondary educational persistence, 

PROCESS analyses revealed that the largest benefit of school-based mentoring was achieved when 

both relationship-building and problem/achievement-focused interactions occurred in relationships 

in which the youth felt valued or that she or he mattered to the mentor. For example, in the absence 

of relationship-building interactions, for children feeling least valued by their mentors, frequent 

engagement in problem/achievement-focused interactions predicted a probability of later criminal 

arrest for a misdemeanor four times higher than for similar matches that spent the least time in 

problem/achievement-focused interactions. Subsequent analyses are required to confirm the 

directionality of these relationships, but one interpretation that is consistent with the hypothesized 

effect is that relationship-building interactions set the stage for problem/achievement-focused 

interactions to be effective by the way they make the youth feel valued by their mentors. 

The Benefits of Doing It All with Someone You Think Likes You 

Conditional process analysis (Hayes, 2013), which estimated the interactive effects of the 

two types of theory-specific activities described Chapter 1 and shown in Table 1.1 (cells 5 and 8), 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 78

which are separately viewed as the two most effective mentoring styles in the existing mentoring 

literature (as highlighted in the TEAM framework, Karcher & Nakkula, 2010)), found both styles to 

be meaningful moderators of long-term outcomes. Confirming the TEAM framework hypothesis 

specifically, the way in which these two contrasting types of activities (relationship-building and 

problem-focused discussions) interact is mediated by the degree to which the mentee feels he or she 

is valued by the mentor or matters to the mentor. Findings revealed the probability of arrest that can 

be expected across the range of reported perceptions of mattering at each of three frequencies of time 

spent getting to know the mentee varied from 1% to 14%, depending on how much mentees reported 

feeling they mattered very little to their mentors and the frequency of specific activities that occurred 

in the match. The probability of arrest was 3 times greater than average in matches where mentors 

spent little time learning about their mentees and when those mentees felt unimportant. When 

mentees felt they mattered to their mentors, the probability of a future arrest did not vary depending 

on the amount of time spent learning about mentee. 

The varying degrees to which the way the experience of mattering predicts the probability of 

arrest differently at varying levels of engagement in relationship-building also differed based on the 

amount of time matches spent in discussions of mentees’ problems. When little time was spent 

getting to know each other, and mentees felt they mattered little to their mentors, the likelihood of 

being arrested was three times higher for youth in matches focusing heavily on the youths’ problems 

compared to matches focusing very little on the mentees’ problems. 

Subsequent analyses are required to confirm the directionality of these relationships, but one 

interpretation that is consistent with the hypothesized effect is that relationship-building interactions 

set the stage for problem/achievement-focused interactions to be more or less effective by the way 

they make the youth feel they matter to their mentors. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 79

Some Limitations and Unanswered Questions 

Although the original study did not observe the type of gains in grades or attendance that 

would predict the increases in post-secondary educational pursuits observed here, there are multiple 

reasons that this finding is not incompatible with this outcome. The most direct explanation for why 

the SMILE study originally did not find these changes, when several other studies of school-based 

mentoring have (see Herrera & Karcher, 2014; Wheeler, Keller & DuBois, 2010; also, Herrera et al., 

2007), is that these outcomes may have been experienced by the entire sample of youth, all of whom 

received standard CIS services. Research on the CIS program suggests these are primary outcomes 

of that program. Given this, it would be difficult to expect non-academically-focused youth 

mentoring to leverage additional change in such indicators of school engagement beyond what CIS 

already yielded for the sample as a whole (Somers & Haider, 2017). 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 80

V. IMPLICATIONS FOR POLICY, PRACTICE, AND FUTURE RESEARCH

Evidence of the long-term benefits of school-based mentoring for the study’s sample of 

predominantly low-income, Latino/a students was corroborated through complementary effects on 

both educational and social/behavioral outcomes. Supplementing the modest and mixed findings 

from the initial impact study (Karcher, 2008), this study suggests long-term effects of school-based 

mentoring may reflect sleeper effects that were not initially evident after nine months but required a 

longer period of time to take hold. As the first long-term follow-up study of school-based mentoring, 

and perhaps the first long-term follow-up on a no-waitlist, randomized, controlled trial of youth 

mentoring, it is important to consider the possibility that other studies of the short-term impact of 

youth mentoring provided similar underestimates of the general benefits of this approach in 

supporting youth development, reducing crime, and fostering educational advancement among lowincome and minority youth. 

Therefore, one implication is to invest in longer evaluation cycles in future research as well 

as to return to the other large-scale experimental tests in the field of youth mentoring to see if sleeper 

effects were also at play in the initial outcome studies. That is, what has been reported to date, 

including the largest study of school-based mentoring (funded by IES through the US Department of 

Education; Bernstein et al., 2009), which reported small impacts overall, may not have captured the 

best evidence of the value of this approach as a means for bettering individuals and society. 

Specifically speaking to the use of this study, however, it seems clear there are policy 

implications for including school-based mentoring as part of a broader strategy for agencies and 

organizations seeking to effect change in the expected trajectories of low-income and minority youth 

in terms of rates of educational persistence and reduced criminality in adulthood. The rationale for 

doing so, including possible specifics to consider, is noted next. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 81

For Federal and Local Investment in Mentoring as a Tool to Promote Higher Education 

The pursuit of almost any form of post-secondary education has meaningful consequences 

for the individuals themselves, but also for society; therefore, finding that there is compelling 

evidence that being enrolled in a mentoring program can extend the contribution beyond what other 

services may already do to facilitate future educational engagement is promising. 

The benefits of earning a bachelors’ degree are significant for individuals directly and for 

their children as well. Baum, Ma, & Payea (2013) reported the median weekly earnings of a fulltime, bachelor’s degree holder in 2011 were 64% percent higher than those of a high school graduate 

($1,053 compared to $638), and college-educated workers are more likely to be employed and to 

have jobs with benefits (e.g., vacation, employer-provided health insurance). Without a college 

degree, children born into the bottom income quintile have a 45% chance of remaining there as 

adults, whereas with a degree, they have less than a 20% chance of staying there. 

Earning an associate’s degree or having just engaged in some college classes has been shown 

to make an annual difference in salary (at the peak of one’s career) of almost $20,000 on average 

(see Figure 5). Therefore, even though this study did not measure post-secondary educational degree 

completion, just finding statistically non-negligible differences in rates of post-secondary enrollment 

can be expected to yield real-world differences in salaries earned for these individuals, resulting in 

more disposable income and greater tax revenue for the government. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 82

Source: from Fumia et al., 2016 

Figure 5. Annual Income Difference at Varying Levels of Post-Secondary Education 

The findings of this study about the direct benefits of school-based mentoring-program 

participation on adult educational pursuits, therefore, suggest further cost-benefit analysis is likely to 

reveal that providing school-based mentoring in this manner could be a highly cost-effective tool for 

promoting educational persistence. 

Also, it would be short-sighted to view benefits found in this study associating participation 

in a school-based mentoring program with fewer misdemeanors in the first five years after high 

school as insufficiently impactful to individuals and society to warrant policy initiatives. Such 

initiatives could include formally investing in school-based mentoring programs to lessen adult 

criminality as a standard prevention approach provided in schools. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 83

Although no effects of participation in a school-based mentoring program were found for 

rates of felony violent or property crimes in the first five post-high-school years (because the 

frequencies were too low to yield reliable analyses), the observed reductions in rates of 

misdemeanors are meaningful and important, not just statistically significant. In an article entitled 

“Why misdemeanors aren’t so minor,” Natapoff argues, 

“...we shouldn’t write off misdemeanors. The repercussions of a petty conviction can be 

anything but minor. These offenses are increasingly punished with hefty fines that lowincome defendants cannot pay.... A conviction of any kind can ruin a person’s job 

prospects. A petty conviction can affect eligibility for professional licenses, child custody, 

food stamps, student loans, and health care or lead to deportation. In many cities, a 

misdemeanor makes you ineligible for public housing.” (Natapoff, 2012, p. 1) 

As a whole, these findings suggest that policy initiatives to support school-based mentoring 

may yield substantial return on investment to the public. In terms of the ratio of the cost for school 

support programs like CIS to involve community volunteers as mentors relative to the long-term 

reductions in crime and increased educational advancement observed in this study, the investment 

seems likely to have a promising cost-benefit ratio for public funding. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 84

REFERENCES 

Avera, J., Karcher, M.J, & Zholu, Y. (2014, March 20). A Long-term Follow-up on Two 

Randomized Studies of Youth Mentoring: A Plan to Identify Essential Ingredients of 

Effective Mentoring Interactions. Poster presented at the Biennial Meeting of the Society for 

Research on Adolescence, Austin, Texas. 

Barlow, D. H. (2010). Negative effects from psychological treatments: A perspective. 

American Psychologist, 65(1), 13-20. doi: 10.1037/0022-006x.59.1.12 

10.1093/clipsy/bpg025 

Baum, S., Ma., J., & Payea, K. (2013). Education pays 2013: The benefits of higher education for 

individuals and society. New York: The College Board. 

Bernstein, L., Dun Rappaport, C., Olsho, L., Hunt, D., and Levin, M. (2009). Impact evaluation of 

the U.S. Department of Education’s student mentoring program. Washington, DC: National 

Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, 

U.S. Department of Education. 

Cavell, T., Karcher, M. J., & Elledge, L. C. (June 3, 2010). Are the effects of school-based 

mentoring mediated by mentor student relationship quality? Poster at the 18th Annual 

Meeting of the Society for Prevention Research. Denver, CO: Grand Hyatt. 

Corrin, W., Parise, L., Cerna, O., Haider, Z., & Somers, M-A. (2015). Case management for 

students at risk of dropping out: Implementation and interim impact findings from the 

Communities In Schools Evaluation. New York: MDRC. 

Darling, N. (2005). Mentoring adolescents. In D. L. DuBois & M. J. Karcher (Eds.), Handbook of 

youth mentoring. (pp. 177-190). Thousand Oaks, CA: Sage. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 85

Dishion, T. J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and 

problem behavior. American Psychologist, 54, 755-764. 

DuBois, D. L., Felner, R. D., Sherman, M. D., & Bull, C. A. (1994). Socio-environmental 

experiences, self-esteem, and emotional/behavioral problems in early adolescence. 

American Journal of Community Psychology, 22, 371-397. 

DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring 

programs: A meta-analytical review. American Journal of Community Psychology, 30, 157– 

197. 

DuBois, D. L. & Karcher, M. J. (2014). Youth mentoring in contemporary perspective. In D. L. 

DuBois, & M. J. Karcher (Eds.), Handbook of youth mentoring (pp. 3-14). Thousand Oaks, 

CA: Sage Publications. 

DuBois, D. L., Neville, H. A., Parra, G. R., & Pugh-Lilly, A. O. (2002). Testing a new model of 

mentoring. In G. G. Noam (Ed.-in-chief) & J. E. Rhodes (Ed.), A critical view of youth 

mentoring (New Directions for Youth Development: Theory, Research, and Practice, No. 93, 

pp. 21-57). San Francisco: Jossey-Bass. 

DuBois, D. L., Portillo, N., Rhodes, J. E., Silverthorn, N., & Valentine, J. C. (2011). How effective 

are mentoring programs for youth? A systematic assessment of the evidence. Psychological 

Science in the Public Interest, 12, 57–91. 

Fox, J. (2008). Applied regression analysis and generalized linear models. Thousand Oaks: Sage. 

Fumia, D., Nicolai, C., Nafziger, M., Hirsch, M., & Hoagland, C. (2016). Interventions to promote 

post-secondary attainment: A review of the evidence and benefit-cost analysis. (Document 

Number 16-12-2301). Olympia: Washington State Institute for Public Policy. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 86

Grossman, J. B., Chan, C. S., Schwartz, S. E. O., & Rhodes, J. E. (2012). The test of time in schoolbased mentoring: The role of relationship duration and re-matching on academic outcomes. 

American Journal of Community Psychology, 49, 43-54. 

Grossman, J. B., & Rhodes, J. E. (2002). The test of time: Predictors and effects of duration in youth 

mentoring relationships. American Journal of Community Psychology, 30, 199–219. 

Hamilton, M. A., & Hamilton, S. F. (2005). Work and Service-Learning. In D. L. DuBois & M. J. 

Karcher (Eds.), Handbook of youth mentoring. (pp. 348-363). Thousand Oaks, CA: Sage. 

Hamilton, S.F., & Hamilton, M.A. (1992). Mentoring programs: Promise and paradox. Phi Delta 

Kappan, 73, 546-550. 

Hansen, K. & Karcher, M. J. (2009, April). School-based mentoring match activities and relationship 

quality. Karcher, M. J. (Chair), Mediators and moderators of outcomes in school-based youth 

mentoring, poster symposium at the biennial convention of the Society for Research in Child 

Development, Denver. 

Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A 

regression-based approach. New York Guilford Press. 

Herrera, C. & Karcher, M. J. (2014). School-based mentoring. In D. L. DuBois, & M. J. Karcher (Eds.), 

Handbook of youth mentoring (2nd Edition; pp. 203-220). Thousand Oaks, CA: Sage 

Publications. 

Herrera, C., Grossman, J. B., Kauh, T. J., Feldman, A. F., McMaken, J. (with Jucovy, L. Z.). (2007). 

Making a difference in schools: The Big Brothers Big Sisters school-based mentoring impact 

study. Philadelphia, PA: Public/Private Ventures. 

Jessor, R., & Jessor, S. L. (1977). Problem behavior and psychological development: A longitudinal 

study of youth. New York: Academic Press. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 87

Karcher, M. J. (2008). The Study of Mentoring in the Learning Environment (SMILE): A 

randomized evaluation of the effectiveness of school-based mentoring. Prevention Science, 

9(2), 99-113. doi: 10.1007/s11121-008-0083-z. 

Karcher, M. J. (November 3, 2016). School-based mentoring as a wake-up call for teenagers: Longterm findings. National Mentoring Symposium. Banff, Alberta Canada. 

Karcher, M. J. (January 29, 2016). Long-term Effects of School–based Mentoring on Latino Youth. 

Panel participant: What the research says, Chair, Jean E. Rhodes. National Mentoring 

Summit. Library of Congress: Washington, DC. 

Karcher, M.J, & Avera, J., Zholu, Y., & Johnson, D. (2017, April 6). The Benefits of Latino/as 

Participation in the Communities in Schools Mentoring Program on Post-Secondary 

Education and Adult Criminality 10 Years Later. Poster presented at the Society for 

Research in Child Development Biennial Meeting, Austin, Texas. 

Karcher, M. J., Benne, K., Gil-Hernandez, Allen, C., D., Roy-Carlson, L., Holcomb, M., & Gomez, 

M. (2006, June). The Study of Mentoring in the Learning Environment (SMILE): A

functional approach to predicting mentor satisfaction from mentoring interactions. Poster 

symposium, 14th annual meeting of the Society for Prevention Research, San Antonio, TX. 

Karcher, M. J., Davis, C., & Powell, B. (2002). The effects of developmental mentoring on 

connectedness and academic achievement. The School Community Journal, 12, 36-50. 

Karcher, M. J. & Hansen, K. (2014). Mentoring activities and interactions. In D. L. DuBois, & M. J. 

Karcher (Eds.), Handbook of youth mentoring (2nd Edition; pp. 63-72). Thousand Oaks, CA: 

Sage Publications. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 88

Karcher, M. J., & Nakkula, M. J. (2010). Youth mentoring with a balanced focus, shared purpose, 

and collaborative interactions. New Directions for Youth Development (126), 13-32. doi: 

10.1002/yd.347 

Marshall, S. (2001). Do I matter? Construct validation of adolescents' perceived mattering to parents 

and friends. Journal of Adolescence, 24, 473-490. 

Morrow, K. V., & Styles, M. B. (1995). Building relationships with youth in program settings: A 

study of Big Brothers/Big Sisters. Philadelphia, PA: Public/Private Ventures. 

McCord, J. (1978). A thirty-year follow-up of treatment effects. American Psychologist, 33, 284-

289. 

Nakkula, M. J. & Harris, J. T. (2005). Assessment of mentoring relationships. In D. L. DuBois & M. 

J. Karcher (Eds.), Handbook of youth mentoring (pp. 100-117). Thousand Oaks, CA: SAGE.

Nakkula, M. J. & Harris, J. T. (in press). Assessment of mentoring relationships. In D. L. DuBois & 

M. J. Karcher (Eds.), Handbook of youth mentoring (2nd ed.).Thousand Oaks, CA: SAGE.

Natapoff , A. (2012). Why misdemeanors aren’t so minor. Slate Retrieved Dec. 12, 2017 from 

http://www.slate.com/articles/news_and_politics/jurisprudence/2012/0...sdemeanors_can_ha 

ve_major_consequences_for_the_people_charged_.html 

National Research Council. (2013). Reforming juvenile justice: A developmental approach. 

Committee on Assessing Juvenile Justice Reform (Bonnie, R. J., Johnson, R. L., Chemers, B. 

M. & Schuck, J. A.) & Committee on Law and Justice, Division of Behavioral and Social

Sciences and Education (Eds.). Washington, DC: The National Academies Press. 

Noam, G. G., Malti, T., K. & Karcher, M. J. (2014). Mentoring in developmental perspective. In D. 

L. DuBois, & M. J. Karcher (Eds.), Handbook of youth mentoring (2nd Edition; pp. 99-116).

Thousand Oaks, CA: Sage Publications. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 89

O’Connell, A. A. & Amico, K. R. (2010). Logistic regression. In G. R. Handcock & R. O. Mueller 

(Eds.), The reviewers guide to quantitative methods in the social sciences. New York: 

Routledge. 

Pryce, J. (2012). Mentor attunement: An approach to successful school-based mentoring 

relationships. Child and Adolescent Social Work Journal, 29(4), 1-21. 

Rhodes, J. E. (2002). Stand by me: The risks and rewards of mentoring today's youth. Cambridge: 

Harvard University Press. 

Rhodes, J. E., Grossman, J. B., & Resch, N. R. (2000). Agents of change: Pathways through which 

mentoring relationships influence adolescents’ academic adjustment. Child Development, 71, 

1662-1671. 

Somers, M-A. & Haider, Z. (2017). A quasi-experimental evaluation of Communities in Schools. 

New York: MDRC. 

Tolan P., Henry, D. H., Schoeny, M., & Bass, A. (2008). Mentoring interventions to affect juvenile 

delinquency and associated problems (the Campbell Report on Mentoring). Institute for 

Juvenile Research, University of Illinois at Chicago. 

Tolan, P. H., Henry, D. B., Schoeny, M. S., Lovegrove, P., & Nichols, E. (2014). Mentoring 

programs to affect delinquency and associated outcomes of youth at risk: A comprehensive 

meta-analytic review. Journal of Experimental Criminology, 10, 179-206. 

https://doi.org/10.1007/s11292-013-9181-4 

U.S. Department of Justice (2008). Crime in the United States: Uniform Crime Reports, 2007. 

Federal Bureau of Investigation: Washington, DC. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 90

U.S. Department of Justice (2009). Crime in the United States: Ten year arrest trends. Federal 

Bureau of Investigation: Washington, DC. Retrieved from 

http://www2.fbi.gov/ucr/cius2009/data/table_32.html 

Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H. (1997). A metaanalysis of outcome studies comparing bona fide psychotherapies: Empirically, “All Must 

Have Prizes.” Psychological Bulletin, 122(3), 203–215. doi:10.1037/0033-2909.122.3.203 

Wheeler, M. E., Keller, T. E., & DuBois, D. L. (2010). Review of three recent randomized trials 

of school-based mentoring: Making sense of mixed findings. SRCD Social Policy Report, 

24 (3). 

Final Technical Report: 2013-JU-FX-0008 
Final Technical Report: 2013-JU-FX-0008 91

APPENDICES 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 92

APPENDIX A: Mentor’s Activity Log Options 

Statistical TEAM 

Variable Framework 

Name Mentoring Activity Description Dimension 

redoe Casual conversation (Discussed sports, weekend activities, holiday plans, fun things 

to do in the community, neighborhood, etc.) 

Talk 

redof Conversation on social issues (Discussed current events in the news, poverty, 

neighborhood events, religion, cultural issues, etc.) 

Talk 

redog Conversation about relationships: • Family • Teachers • Friends •

Romantic Friend 

Talk 

redoh Listening & learning (Discussed mentee’s hobbies & interests, feelings, or mentee 

talked most of the time while mentor listened.) 

Talk 

redoj Sports, athletic activity, or outdoor game (activity) (Played basketball, soccer, catch, 

volleyball, tennis...) 

Play 

redok Creative activities (activity) (Engaged in drawing, arts and crafts, reading and 

writing for fun, photography, crafts, art projects, etc.) 

Play 

redol Indoor games (activity) (Board games, playing cards, chess, Uno, checkers, 

computer games, puzzles, etc.) 

Play 

redoa Academics (Discussed grades, school, testing, etc.) Learn 

redob Behavior (Discussed youth’s behaviors that were related to problems with peers, 

teachers, adults, specifically misbehavior.) 

Learn 

redoc Attendance, graduating and “stay-in-school” discussion Learn 

redod Future (Discussed college, careers, jobs, goals, dreams, etc.) Learn 

redoi Tutoring/Homework (activity) (Helped with homework, did tutoring, helped with 

reading, library, academic computer work.) 

Do 

Source: Karcher, M. J. (2008). The Study of Mentoring in the Learning Environment (SMILE): A 

randomized evaluation of the effectiveness of school-based mentoring. Prevention Science, 9(2), 99-113.* 

Mentor-Mentee Interaction Checklist (Karcher, 2005). Mentors described the content of their 

interactions and discussions with their mentees using a form provided by CIS that listed many common 

activities engaged in during school-based mentoring. The interactions tallied were the same ones as examined 

in an earlier study (DuBois, Neville, et al., 2002) that found that particular mentoring interactions and 

discussion content were highly predictive of whether or not mentees came to see their mentors as “significant 

adults” in later life. Using DuBois, Neville, et al.’s scales, a checklist was created that program site 

coordinators’ had mentors complete after each meeting. Prescriptive/Instrumental activities included 

discussion of youth’s behavior, activities related to homework or schoolwork, and discussion or participation 

in prevention curricula activities, such as skill building exercises. Developmental/Psychosocial activities 

included discussion of social issues, casual conversation, recreational activities (like sports), game play, and 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 93

mentor listening to the mentee talk or learning about his or her life (e.g., struggles or successes). There was 

space for mentors to write additional activities not listed. These data were entered into a database each 

month. At the end of the evaluation, the additional activities reported by the mentors were coded by two raters 

as either prescriptive/instrumental, developmental, or ambiguous, based on the definitions provided by 

Morrow and Styles (1995), will be used as indicators of either the developmental or prescriptive latent factors. 

References 

Karcher, M. J. (2005). Testing the effects of instrumental and developmental activities on outcomes in youth 

mentoring. Unpublished manuscript, University of Texas at San Antonio. 

DuBois, D. L., Neville, H. A., Parra, G. R., & Pugh-Lilly, A. O. (2002). Testing a new model of mentoring. 

In J. E. Rhodes (Ed.). A critical view of youth mentoring: No. 93 in the New directions for youth 

development series. San Francisco: Jossey-Bass. 

Karcher, M. J., Kuperminc, G., Portwood, S., Sipe, C., & Taylor, A. (2006). Mentoring programs: A framework to 

inform program development, research, and evaluation. Journal of Community Psychology, 34, 709-725. 

Morrow, K., & Styles, M. (1995). Building relationships with youth in program settings: A study of Big 

Brothers/Big Sisters. Philadelphia: Public/Private Ventures. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 94

APPENDIX B: Activity Coding 

Coding Mentoring Interactions 

Estimating total hours of mentoring and percent time of mentoring activities. Presented below is 

SPSS syntax used, linking specific activities (in parentheses) to higher order interaction categories of 

Play, Talk, Learn, and Do. 

LEARN: Percent Time in Instrumental Discussion 

COMPUTE TPctInsD = SUM(redoa,redob,redoc,redod)/RedoThrs . 

VARIABLE LABELS TPctInsD 'Percent of Time in Instrumental Discussions' . 

TALK: Percent Time in Developmental Discussion 

COMPUTE TPctDevD = SUM(redoe,redof,redog,redoh)/RedoThrs . 

VARIABLE LABELS TPctDevD 'Percent Time in Developmental Discussions' . 

PLAY: Percent Time in Developmental Activities 

COMPUTE TPctDevA = sum(redoj,redok,redol)/RedoThrs . 

DO (Homework): Percent Time in Instrumental Activities/Homework 

COMPUTE TPctInsA = ThrsInsA/RedoThrs . 

VARIABLE LABELS TPctInsA 'Percent Time Tutoring or Homework' . 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 95

APPENDIX C: Mentoring Experience and Relationship Quality Surveys 

Mentee Mattering Survey (Marshall, 2001). Mattering, a form of social identity, is the 

psychological tendency to view the self as significant to others. Based on the Perceived Mattering 

scale (Marshall, 2001) developed to assess how much youth feel they matter to their mothers, 

fathers, friends, and other important individuals, the Mentee Mattering Survey includes nine items 

translated from Marshall (2001) and which have demonstrated high reliability when used previously 

with Caucasian mentees in grades four and five (a = .93; Karcher, 2002), and the survey 

demonstrated good reliability (a = .75/.81) in the present study. 

not much somewhat a lot 

1. I am important to my mentor: 1 2 3 4 5 

2. I am needed by my mentor:

3. I am missed by my mentor when I am away:

4. When I talk, my mentor tries to understand what I am saying:

5. I am interesting to my mentor:

6. My mentor notices my feelings:

7. My mentor gives me credit when I do well:

8. My mentor notices when I need help:

9. I matter to my mentor:

Marshall, S. (2001). Do I matter? Construct validation of adolescents' perceived mattering to 

parents and friends. Journal of Adolescence, 24, 473-490. 

Scales of Social Support Applied to Mentors. Based on DuBois and Hirsch (1990), measures 

feeling valued and being motivated by mentor. 

Feel valued by mentor 

1bv My mentor cares about how well I am doing in school. 

2bv My mentor is very sure that I can do well in school and in the future. 

3bv My mentor cares about me even when I make mistakes 

4bv My mentor really listens and understands me. 

5bv My mentor looks out for me and helps me. 

6bv My mentor and I both have fun when we are together. 

Feel motivated by mentor 

1bm My mentor gives me useful advise in dealing with my problems. 

2bm My mentor has qualities or skills that I’d like to have when I’m older. 

3bm I learn how to do things from watching and listening to my mentor. 

4bm My mentor introduces me to new ideas, interests, and things to do. 

5bm My mentor pushes me to succeed at things I want to do. 

DuBois, D. L., & Hirsch, B. J. (1990). School and neighborhood friendship patterns of 

Blacks and Whites in early adolescence. Child Development. 61(2), 524-536. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 96

Connectedness to Mentor and Connectedness to Mentoring Program Scales 

Mentor connection. The mentor connection scale includes items that were intended to 

measure the degree to which the mentee viewed the mentor as providing empathy, praise, and 

attention. These are the three elements of Kohut’s model of self-psychology viewed as core the to 

development of self-esteem and social connectedness (see Karcher, Zambrano, & Holcomb, 2008; 

Lee & Robbins, 1998). Mentor Connection. The Mentor Connection scale (EPA) contains seven 

items. The items are designed to assess the level of connection a student has to his or her mentor. 

Each of the items, for both years, are well above the accepted cutoff of .5. 

MY Mentor... Responses: No/Never Sometimes Often Always 

(1) ... understands me well. 1 2 3 4 

(2)... says good things about me.

(3) ... likes to spend time with me.

(4) ... is rude to me. (Filler item to lessen response set)

(5) ... knows a lot about me.

(6)... likes how I am in mentoring.

(7) ... asks me questions about me and my life.

(8) ... is disrespectful to me. (Filler item)

(9)... accepts me for who I am.

(10)... makes me feel good about who I am.

(11)... listens to what I have to say.

Program connection. The program connection scale includes items that were intended to 

measure the degree to which the mentee views the program as providing a clear, consistent structure in 

which enjoyable activities are provided, and which allow the mentee to idealize the mentor through 

positive, structured experiences with the mentor. These are the second set of elements of Kohut’s model 

of self-psychology viewed as core the to development of self-esteem and social connectedness (see 

Karcher, Zambrano, & Holcomb, 2008; Lee & Robbins, 1998). In Kohut’s model of the bi-polar selfdevelopment (1971), this structure is viewed as the second pole of supportive developmental contents 

which afford opportunities to idealize the others present in this environment. The scale has six items. In 

both years, the factor loadings were above the .5 cutoff 

ABOUT THE PROGRAM Responses: No/Never Sometimes Often Always 

1. I like what we do together. 1 2 3 4 

2. I get to do things I like to do.

3. I like coming to meet my mentor.

4. I learn things about myself from my mentor

5. Mentoring is boring.

6. I enjoy what we do in mentoring.

Mentor Connection=1, 2, 3, 5, 6, 7, 9, 10, 11 

Program Connection= 1, 3, 4, 5, 6 

Filler Variables to Avoid Response Set Bias (omitted from all scales =M4, M8) 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 97

Connors Child Rating Scale: Global Index (Conners, Sitarenios, Parker, & Epstein, 1998). This survey, 

completed by parents prior to each youth being randomly assigned to conditions, reflects ten items which 

capture a range of problems behaviors, both internalizing and externalizing. Items include: The child is 

“Restless or overactive; Excitable, impulsive; Cries often and easily; and Mood changes quickly and 

drastically.” The scale demonstrated high reliability (a = .82). 

Conners, C. K., Sitarenios, G., Parker, J. D. A., & Epstein, J. N. (1998). The revised Conners' 

Parent Rating Scale (CPRS-R): Factor structure, reliability, and criterion validity. Journal of 

Abnormal Child Psychology, 26(4), 257-268. 

Social Skills Rating Scale (Elliott & Gresham). Assesses Cooperation, Empathy, Self-Control, 

Assertiveness and Responsibility. Has extensive reliability and validity data. 

Elliott, S. N., & Gresham, F. M. (1987). Children's social skills: Assessment and classification 

practices. Journal of Counseling and Development, 66, 96-99. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 98

APPENDIX D: Criminal Activity Coding 

Classification Code 

Criminal Part I Felony 

Violent: 

a Criminal homicide CRP1CHO 

b forcible rape CRP1RAP 

b robbery CRP1ROB 

b aggravated assault CRP1AGA 

Property Crime 

f buglary CRP1BUG 

f larceny CRP1LAR 

f motor theft CRP1MOT 

b arson CRP1ARS 

I drug trafficing place CRP1DRT 

d child pornography CRP1CPo 

Criminal Part II Misdemeanor 

MSD A bail jumping CRP2BAI 

MSD A stolen property CRP2STP 

MSD A driving under influence CRP2DUI 

MDS A drug offences CRP2DRU 

MDS A reckless driving CRP2REC 

MSD A resisting officer CRP2ROF 

MSD A neglecting a child CRP2NEG 

MSD A weapon offenses CRP2WOF 

MSD B vandalism CRP2VAN 

MSD B disorderly conduct CRP2DOC 

MSD B trespassing CRP2MSD 

MSD C simple assault CRP2SAS 

MSD C forgery CRP2FOR 

MSD C counterfeit CRP2COU 

MSD C fraud CRP2FRA 

MSD C gambling CRP2GAM 

MSD C offenses against family CRP2OAF 

MSD C prostitution CRP2PRO 

MSD C public drunkness CRP2PUB 

MSD A sex offenses CRP2SEX 

MSD C loitering CRP2LOI 

MSD C vagrancy CRP2VAG 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 99

CRIME CODING 

COMPUTE ViolentTot=CRP1CHO+CRP1RAP+CRP1ROB+CRP1AGA. 

VARIABLE LABELS ViolentTot 'Total Violent Crimes'. 

COMPUTE PropTot=CRP1BUG+CRP1LAR+CRP1MOT+CRP1ARS. 

VARIABLE LABELS PropTot 'Total Property Crimes'. 

COMPUTE MisdTot=CRP2STP+CRP2DUI+CRP2DRU+CRP2WOF+CRP2VAN+CRP2DOC+ 

CRP2SAS+CRP2FOR+CRP2COU+CRP2FRA+CRP2GAM+CRP2SEX+CRP2LOI+CRP2VAG. 

VARIABLE LABELS MisdTot 'Total Misdemeanor Crimes'. 

RECODE ViolentTot PropTot MisdTot (0=0) (1 thru 45=1) INTO Violentby21 Propby21 Misdby21. 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 100

APPPENDIX E: Zero-Order Correlations by Sex 

Zero-Order Correlations for Female Mentees Between Activities and Outcomes 

Arrested for 

Misdemeanor. 

by Age 21 

Enrolled in 

Postsecondary by 

Age 21 

Total CIS 

(ex. men) 

Total hours 

of 

Mentoring 

Enrolled in some 

post secondary 

Education by 21 

r 0.002 1 

p 0.977 

n 311 311 

Total CIS 

services (excl. 

mentoring) 

r -0.046 0.013 1 

p 0.425 0.815 

n 307 307 307 

Total hours of 

Mentoring 

r -0.059 .130* .245** 1 

p 0.301 0.023 0.000 

n 307 307 307 307 

Percent of Time 

in Instrumental 

Discussion 

r 0.042 -0.144 -0.100 -.240** 

p 0.634 0.104 0.260 0.006 

n 129 129 129 129 

Percent Time in 

Developmental 

Discussions 

r -0.163 .181* -0.122 0.010 

p 0.065 0.040 0.167 0.907 

n 129 129 129 129 

Percent Time in 

Developmental 

Activities 

r -0.025 0.043 .203* .255** 

p 0.781 0.627 0.021 0.003 

n 129 129 129 129 

Percent Time 

Doing Tutoring 

or Homework 

r .206* -0.126 -0.001 -0.079

p 0.019 0.156 0.990 0.376 

n 129 129 129 129 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 101

Zero-Order Correlations for Female Mentees Between Experiences and Outcomes 

Arrested for 

Misdemeanor. 

by Age 21 

Enrolled in 

Postsecondary 

by Age 21 

Total 

CIS (ex. 

men) 

Total hours 

of 

Mentoring 

Mentee feels valued r 0.087 0.087 0.083 .258** 

p 0.330 0.329 0.353 0.003 

n 128 128 127 127 

Feels motivated by 

mentor 

r 0.021 0.153* 0.030 .219* 

p 0.816 0.084 0.737 0.013 

n 128 128 127 127 

Matter to mentor r -0.011 0.057 0.168• .329** 

p 0.901 0.524 0.059 0.000 

n 128 128 127 127 

Connectedness to 

Mentor (Year 1) 

r 0.046 0.043 0.070 .298** 

p 0.603 0.626 0.434 0.001 

n 129 129 128 128 

Connectedness to 

Program (Year 1) 

r -0.023 0.071 0.075 .199* 

p 0.798 0.424 0.403 0.024 

n 129 129 128 128 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 102

Zero-Order Correlations for Male Mentees Between Activities and Outcomes 

Arrested for 

Misdemeanor. 

by Age 21 

Enrolled in 

Postsecondary by 

Age 21 

Total CIS 

(ex. men) 

Total hours 

of 

Mentoring 

Enrolled in some 

post secondary 

Education by 21 

r 0.012 1 

p 0.881 

n 155 155 

Total CIS 

services (excl. 

mentoring) 

r -0.105 -0.004 1 

p 0.193 0.961 

n 154 154 154 

Total hours of 

Mentoring 

r -0.113 -0.082 .271** 1 

p 0.161 0.313 0.001 

n 154 154 154 154 

Percent of Time 

in Instrumental 

Discussion 

r 0.054 -0.021 -0.075 -0.202

p 0.636 0.856 0.516 0.076 

n 78 78 78 78 

Percent Time in 

Developmental 

Discussions 

r -0.197 0.042 -0.070 -0.040

p 0.084 0.713 0.540 0.730 

n 78 78 78 78 

Percent Time in 

Developmental 

Activities 

r 0.110 0.083 0.088 0.166 

p 0.338 0.470 0.442 0.147 

n 78 78 78 78 

Percent Time 

Doing Tutoring 

or Homework 

r -0.028 -0.214 0.047 0.052 

p 0.806 0.060 0.683 0.652 

n 78 78 78 78 

Final Technical Report: 2013-JU-FX-0008 
 

 

 

 

 

 

 

 

Final Technical Report: 2013-JU-FX-0008 103

Zero-Order Correlations for Male Mentees Between Experiences and Outcomes 

Arrested for 

Misdemeanor. 

by Age 21 

Enrolled in 

Postsecondary 

by Age 21 

Total 

CIS (ex. 

men) 

Total hours 

of 

Mentoring 

Mentee feels valued r -.241* -0.166 0.178 0.204 

p 0.031 0.142 0.113 0.070 

n 80 80 80 80 

Feels motivated by 

mentor 

r -0.180 -0.060 0.191 .241* 

p 0.110 0.595 0.090 0.032 

n 80 80 80 80 

Matter to mentor r -.279* -0.045 0.129 0.134 

p 0.012 0.691 0.254 0.236 

n 80 80 80 80 

Connectedness to 

Mentor (Year 1) 

r -.235* -0.060 0.185 .283* 

p 0.036 0.599 0.101 0.011 

n 80 80 80 80 

Connectedness to 

Program (Year 1) 

r -.273* -0.113 .237* .236* 

p 0.014 0.319 0.035 0.035 

n 80 80 80 80 

Final Technical Report: 2013-JU-FX-0008