# Presentation: 688900

## Efficacy of A Mixed Methods Approach to Analyzing  Educational Enrichment Activities: a mixed methods approach

- Merna Villarejo			Deborah Kogan
- Amy Barlow			Brian Veazey
- Melanie Jones			Lina Mendez-Benavidez
- Jennifer Sweeney		Sarah Ovink
- 	School of Education
- University of California, Davis
*Funded by NIH NIGMS MORE program “Efficacy of Interventions”*

## Overall Research Question
	Does participation in undergraduate science enrichment activities encourage students to pursue biology-based professions and provide them with the tools to do so?
Mixed Methods Approach
Quantitative Study of College Achievement: [All UCD biology entrants 1995-99, n=6,878]Statistical methods (multivariate logistic regression) to find factors that correlate with persistence and performance in biology. 
Qualitative Study of BUSP Alumni [Study pool = graduates from 1992-2004 with GPA ≥ 2.7]
Surveys (n= 201) to define career path and factors important to career choices.
Interviews with subset of survey respondents (n=109) to discover motivation and influences on career choices. 

- 	Does participation in undergraduate science enrichment activities encourage students to pursue biology-based professions and provide them with the tools to do so?
**Mixed Methods Approach**
- Quantitative Study of College Achievement: *[All UCD biology entrants 1995-99, n=6,878]*Statistical methods (multivariate logistic regression) to find factors that correlate with persistence and performance in biology.
- Qualitative Study of BUSP Alumni *[Study pool = graduates from 1992-2004 with GPA ≥ 2.7]*
- Surveys (n= 201) to define career path and factors important to career choices.
- Interviews with subset of survey respondents (n=109) to discover motivation and influences on career choices.

**Notes:**
Earlier study of 1988-94 classes asked “Do URM students who participate in BUSP do better in college than non-participant minorities?  Clear yes answer
Statistical study
Here we have extended that approach to later classes 1995-99, with an important addition. Add comparison to white/Asian majority.  Not only are they doing better, but are they well enough prepared to do without affirmative action at the next step in their careers.
Statistical analyses of undergraduate attainment address the question of whether BUSP students have the tools to pursue a biology-based career.  
Survey and interviews
The survey goes beyond the UG years to ask what careers alumni actually choose and their ideas of the relative importance of different influences – family concerns, UG experiences etc.  Different study population than the statistical study.  Chose to look at all BUSP alumni 92-04 who had the potential to go on to graduate or professional school.  In statistical study, we use a graduating GPA=3.0 as that floor.  We started contacting alumni, we found that some with GPAs as low as 2.7 went on to MS to prove themselves and used that to get into PhD or MD programs.  Gave us a total population of 322 (out of ~1000)

## Overall Research Question
	Does participation in undergraduate science enrichment activities prepare and encourage minority students to pursue biology-based professions?

Study tools
Statistical analysis of college achievement
Alumni survey
Alumni interviews

- 	Does participation in undergraduate science enrichment activities prepare and encourage minority students to pursue biology-based professions?
- Study tools
- Statistical analysis of college achievement
- Alumni survey
- Alumni interviews

**Notes:**
Do these programs work?  NIH is funding these activities so you want to know whether it is money well-spent.  This talk looks at first two study tools; statistical analysis and survey.

## Biology Undergraduate Scholars Program (BUSP)at UCD, 1988-present

- Comprehensive
- academic enrichment in chemistry, math & biology
- advising
- research experience
-  Large scale
- 55-65 new freshmen each year
- Multi-year
- Pre-freshman summer bridge through sophomore year
- Largely underrepresented minority students (URM)

**Notes:**
Studying one such enrichment program at UCD.

## Slide 5

**Notes:**
The first step toward a professional biology career is an undergraduate science degree.  
How do BUSP participants compare to non-participants in college graduation outcomes?  
	In earlier work, we showed that BUSP students who entered 1988-94 outperformed non-participant URMs.  In this study we also compared them with the majority white/Asian group.  The data shows that BUSP students again outperformed non-participant URMs but also did as well or better than white/Asian students, even though BUSP students entered college with lower high school GPAs and SAT scores.  Conclusion: these students have the tools to succeed in a biology-based profession.

## Multivariate logistic regression shows that undergraduate research participation is associated with high graduation rates in biology majors and graduation with high grades in biology majors, for students of all races and ethnicities.

Causality???????

UG research → Biology interest
or
Biology interest → UG research

- Causality???????
- UG research → Biology interest
- or
- Biology interest → UG research

**Notes:**
We used same data set to look for variables that predicted success in biology.  There is a strong, positive association between BUSP participation and academic persistence and success, and of UR with persistence and performance in biology. However, statistics cannot be used to show causality.  An ethnographic study by Seymour et al suggests that summer research programs serving mostly rising seniors at largely white liberal arts colleges make no difference in ultimate career choice.  Students simply “clarify, refine and reinforce” their interests.  How can we learn about causality in our population?  Is it the same?

## BUSP Alumni Career Path Survey

- Track High Achieving BUSP Alumni
- 	Graduates from 1992-2004 with GPA ≥ 2.7 (n = 322)
- _Alumni Survey (respondents = 201) tells us:_
- What components of BUSP were most helpful in achieving academic success.
- Alumni’s post-college career paths.
- Some important factors in making career choices.

**Notes:**
Need a different kind of research to answer those questions, and to find out if students did indeed pursue a professional biology career.
Therefore, we tracked alumni and asked them to respond first to a survey, and then to participate in an in-depth telephone interview (n=106).

## Biomedical Careers in Research

- 19% of BUSP Alumni selected biomedical careers in research

**Notes:**
Compare and contrast doctoral level; then research professions

## Alumni credit supplemental instruction with helping them to excel academically.

**Notes:**
What components of BUSP were most helpful in achieving academic success?
Alumni valued supplemental instruction most highly
“ having those [supplemental classes] was so incredibly useful I can't even begin to tell you.   And I was like, 'Okay, I'm not an idiot; I actually know what I'm doing.' 
Female, Mexican-American, First Generation College Student, employed as Senior Clinical Trial Manager

## 70 % of respondents selected biomedical careers

**MEDICAL DOCTORS (34 %) **
**BIOMEDICAL PhDs **
**	****(12 %) **
**OTHER ALLIED HEALTH (10 %)  ***** ***
**ALLIED HEALTH DOCTORATE (7 %) **
***	***
**BIOMEDICAL TECHNICIANS (7%)**

**OTHER (21 %)**
**OTHER PHD (5 %) **
**UNDECIDED (4 %)**

**70 % of respondents selected ****biomedical careers**

**Notes:**
Alumni post-college career paths
57% are at the doctoral level, largest number are Medical Doctors.  Only 12% are Biomed PhDs.   Why are so few drawn to research careers?

## Reasons for not pursuing a PhD

- Medical Doctors
- 	Desire to directly serve the community, prestige of medicine
- Technicians
- College debt load
- Some lack academic qualifications
- Allied Health Doctorates
- Job security and availability
- Other careers better accommodate raising a family

**Notes:**
Some factors important in choosing, or not choosing,  a career as a PhD-level scientific researcher
Some students are drawn to research by their UG research experience, others are repelled.  Those for whom other aspects of a career are more important, tend to make other career choices, even if they are academically successful in their science studies.  (For example, students for whom "helping people" or "working in a profession with high social prestige" is a high priority are likely to make a different career choice (e.g. medical doctor or allied health professional).  Somewhat disturbing is our discovery that the career of science researcher is nearly universally unattractive, for a variety of practical reasons, even among those individuals who decide to become  researchers.  For example, most respondents believe that science researchers will have trouble balancing work and raising a family, that they will have difficulty finding stable employment, and that other careers would offer greater  financial rewards. It is not clear the extent to which these "negatives" deterred participants from pursuing careers as science researchers, even if they were attracted to a research career for other reasons.

## Reasons For Pursuing a Biomed PhD

- PhDs
- Have substantive interest in science
- Expressed joy in doing research
- Recognized practical problems but continued despite them

**Notes:**

Among the aspects of the research experience that appear to be positive "magnets" toward research
careers--compelling for a subset of study subjects--are substantive interest in science and joy in doing research.

## Out of 24 Biomedical When do Biomed PhDs: make their career choices?

- 5  	research was main interest at college entry.
- 8  	interested in research at college entry, but also 	considering other options.
- 11 	pre-med, changed to PhD after undergraduate 	research experience.

**Notes:**
At college entry, many in our target population are unfamiliar and/or uninterested in PhD-level research careers.  For a few, undergraduate research is transformative.  

“It open(ed) my mind to the different career options.  I came into college pre-med but after conducting research in a lab, I found that that is something that I really want to do as a career.”

## Policy implication:  

- At college entry, most BUSP participants are unfamiliar with the career options associated with the PhD.  Many start out on the pre-med path and only discover their research interest during college, largely as the result of participation in undergraduate research.
- Policy implication:
- To encourage more minorities pursue careers undergraduate
- research opportunities should be expanded beyond those already seriously interested in research to include “pre-med” students.

**Notes:**

The positive: 
Broad participation in undergraduate research attracts some students to science careers who would not otherwise have made this career choice.  
The negative:
Individuals choose research because they love it, despite the fact that they perceive that this career choice brings with it strong practical disadvantages.