Effects of Outdoor Education
Programs for Children in California

Submitted by:American Institutes for Research

Submitted to:
The California Department of Education
1430 N Street
Sacramento, CA 95814

January 31, 2005
“American Institutes for Research” is a registered trademark. All other brand, product, or company names aretrademarks or registered trademarks of their respective owners.

1791 ARASTRADERO ROAD|PALO ALTO, CA 94304-1337|TEL 650 493 3550|FAX 650 858 0958|WEBSITE WWW.AIR.ORG

Acknowledgements
The American Institutes for Research wants to extend thanks to the many individuals who
contributed to this project. We especially appreciate the assistance of the staff from the four
elementary schools and three outdoor science schools who participated in the study. The
following staff contributed to this report: Deborah (Montgomery) Parrish, Principal Investigator;
Gabriele Phillips, Project Director; Roger Levine, Managing Research Scientist; Hiro Hikawa
and Matt Gaertner, Research Analysts; Nadine Agosta, Research Associate; and Diana Doyal,
Administrative Associate. For more information about this report, please contact Gabriele
Phillips, at:American Institutes for Research1791 Arastradero Road
Palo Alto, CA 94304
650-843-8100

Effects of Outdoor Education Programs for Children in California

Table of Contents
Executive Summary ........................................................................................................................iii
Research Questions.............................................................................................................iii
Methodology .......................................................................................................................iii
Data Analysis ......................................................................................................................iv
Findings................................................................................................................................ v
Social and Personal Skills.................................................................................................... v
Stewardship of the Environment.......................................................................................... v
Knowledge and Understanding of Science Concepts .........................................................vi
Benefits for English Language Learners.............................................................................vi
Input from Teachers, Outdoor Education Staff, and Students ............................................vi
Conclusions........................................................................................................................vii
Chapter 1: Introduction .................................................................................................................... 1
Background .......................................................................................................................... 1
Environmental Education in California ............................................................................... 2
Evaluation Approach ........................................................................................................... 3
Research Questions.............................................................................................................. 5
Evaluation Design................................................................................................................ 5
Chapter 2: Methodology .................................................................................................................. 7
Participating Outdoor Education Programs ......................................................................... 7
Participating Elementary Schools ........................................................................................ 8
Data Collection Strategies.................................................................................................... 9
Data Collection Tools ........................................................................................................ 10
Survey Responses .............................................................................................................. 12
Data Analysis ..................................................................................................................... 12
Chapter 3: Research Findings ........................................................................................................ 15
Research Question #1: How does participation in outdoor education programs impact
students’ personal and social skills (e.g., self-esteem, cooperation, etc.)? ............................. 15
Student Assessments.......................................................................................................... 16
Parent Reports.................................................................................................................... 18
Teacher Reports ................................................................................................................. 18
Personal and Social Skills Subgroup Analyses.................................................................. 20
Research Question #2: How does participation in outdoor education programs foster
students’ stewardship of the environment and their appreciation of the importance of thewise use of natural resources?................................................................................................. 23
Student Assessments.......................................................................................................... 23
Parent Reports.................................................................................................................... 25
Stewardship of the Environment Subgroup Analyses ....................................................... 26
Research Question #3: How does the science instruction received through the outdoor
education program curriculum increase students’ knowledge and understanding of science
concepts? ................................................................................................................................. 28
Student Assessments.......................................................................................................... 29
Science Score Subgroup Analyses..................................................................................... 30

American Institutes for Research Page iEffects of Outdoor Education Programs for Children in California

Chapter 4: Perceived Benefits of Outdoor Education: Feedback from Sixth Grade Teachers andOutdoor Education Staff .............................................................................................. 33
Academic Impacts.............................................................................................................. 33
Social-Emotional Impacts.................................................................................................. 34
Chapter 5: Conclusions .................................................................................................................. 37
Personal and Social Skills.................................................................................................. 37
Stewardship of the Environment........................................................................................ 37
Knowledge and Understanding of Science Concepts ........................................................ 38
Benefits for English Language Learners............................................................................ 38
Other Perceived Impacts of the Program ........................................................................... 38
Recommendations.............................................................................................................. 39
References...................................................................................................................................... 41
Appendix A: Alignment of California Academic Content Standards for Science with Outdoor
School Curricula ................................................................................................... A-1
Appendix B: Data Collection Tools.............................................................................................B-1
Appendix C: Exploratory Analysis..............................................................................................C-1

American Institutes for Research Page ii

Effects of Outdoor Education Programs for Children in California

Executive Summary
The American Institutes for Research (AIR) conducted an evaluation to measure the impacts ofweek-long residential outdoor education programs for at-risk sixth graders in California. As
described by California Assembly Bill (AB) Number 1330, Chapter 663, the Outdoor
Environmental Education Program is designed to “foster stewardship of the environment and an
appreciation of the importance of the wise use of natural resources.” The program serves at-riskyouth and underserved demographic groups. AB 1330 called for the California Department of
Education (CDE) to administer an independent evaluation of the program to be conducted by
February 1, 2005 to examine the effects of outdoor experiences on students’ behavior andlearning. This report presents the findings from the AIR evaluation.
This study focused on 255 sixth-grade students from four elementary schools who attended three
outdoor education programs (also referred to as outdoor science schools) between September andNovember of 2004. The evaluation utilized a “delayed treatment design.” Within participating
elementary schools, sixth-grade children were divided, by classroom, into two groups.
Approximately half of each school’s sixth grade (one or more classrooms) attended outdoor
school between September and November of 2004 and served as the treatment group. The
remaining sixth grade classrooms were scheduled to attend outdoor school after the study’s datacollection period ended in December 2004, thereby serving as the control group during the study
period. In this manner, the study utilized a treatment and control design without denying any
child the opportunity to attend outdoor science school. The design provides a rigorous method to
identify the outcomes associated with participation in the program.
Research Questions
The specific research questions addressed in this study are as follows:
1. How does participation in outdoor education programs impact students’ personal andsocial skills (e.g., self-esteem, cooperation, teamwork)?
2. How does participation in outdoor education programs foster students’ stewardship of theenvironment and their appreciation of the importance of the wise use of natural
resources?
3. How does the science instruction received through the outdoor education programcurriculum increase students’ knowledge and understanding of science concepts?
Methodology
Participating Outdoor Science Schools. The CDE, in consultation with AIR, selected threeresident outdoor science schools as the target programs for the study. Geographically diverse, theschools are located in rural areas near Fresno, Los Angeles, and San Diego. The programs
primarily serve fifth- and sixth-grade students during week-long residential programs, and use
curricula that align with the California State Science Framework and the California AcademicContent Standards for science. The hands-on, inquiry-based curriculum is designed to help

American Institutes for Research Page iii

Effects of Outdoor Education Programs for Children in California

students understand the environment and the role of humans as participants in ecosystems, aswell as develop their skills, attitudes, knowledge and commitment concerning the natural world.
While instructional activities vary somewhat across the outdoor education programs participatingin the study, the content of the curricula is consistent, focusing primarily on ecology and earthscience.
Participating Elementary Schools and Students. A total of 255 students from four California
elementary schools participated in the study. The four schools serve mostly Hispanic children(ranging from 69 percent to 89 percent of the student population) and have a high proportion ofEnglish Learners (32 percent to 66 percent of students). Eighty-one to 100 percent of the children
in each school qualify for the free and reduced price lunch program.
Data Collection. Data collection included the use of surveys and site visits between September
and December of 2004. Data were only collected from children for whom informed consent had
been granted (via signed consent forms) by the child and parent. Parents also provided theirconsent prior to completing the parent surveys. Participation in the evaluation did not impact
students’ opportunity to attend the outdoor school in any way.
Surveys. Students, parents, and teachers were surveyed. Three rounds of surveys wereadministered: before the treatment group attended outdoor school (Round 1 pre-survey),
immediately after the treatment group returned from outdoor school (Round 2 first post-survey),
and six to ten weeks after the treatment group returned from outdoor school (Round 3 second
post-survey). Students were surveyed in all three rounds, and parents and teachers were surveyedin Round 1 and Round 3. Parents and teachers provided individual ratings on eight constructs forstudents in both the treatment and control groups. Student responses from Rounds 1 and 2 wereused to determine the immediate impacts of participation in outdoor education, while student,
parent, and teacher responses from Rounds 1 and 3 were analyzed to explore the longer-term
impacts of the program.
Site Visits. Research staff conducted one-day site visits to each of the three outdoor
schools during the same week the treatment group attended each program. Staff observed
instructional activities while on-site and conducted in-person interviews or focus groups with thesixth-grade teachers of participating students. Teachers were asked to provide input regarding theperceived benefits of outdoor school for students, how the program impacted specific groups ofstudents, and how the outdoor school experience was integrated into their classroom teaching. In
addition, AIR staff interviewed the principals of the outdoor schools to gather descriptive
information and solicit their input regarding the impacts of the program.
Data Analysis
Quantitative Analysis. After data collection was completed, the survey results were recorded aselectronic data files. Scales for five social and personal constructs, three environmental attitude
scales, and an overall science score, were developed from individual survey items. The reliabilityof these constructs (scales) was assessed by calculating Cronbach’s alpha, which measures theextent to which the scale items are measuring a common, underlying construct.

American Institutes for Research Page iv

Effects of Outdoor Education Programs for Children in California

Based on an analysis plan developed for the study, two independent sample t-tests were used todetect statistically significant differences between various student groups and subgroups (e.g.,treatment versus control groups, male versus female, Hispanic versus non-Hispanic students).
Paired-sample t-tests were employed to examine significant gain scores within groups. Similar
analyses were conducted for survey data from parents and teachers. The criterion used forstatistical significance was p<.05.
Qualitative Analysis. The qualitative data collected through the site visits, interviews, andsurveys were reviewed and common themes were identified across respondents. The qualitative
data were used primarily to provide context for the quantitative survey findings.
Findings
The following section provides an overview of the study’s findings.
Social and Personal Skills
Students and parents were surveyed to measure student-level changes across five related
constructs: conflict resolution, self-esteem, cooperation, leadership, and their relationship with
their teacher. Teachers rated each student on eight constructs: self-esteem, cooperation, conflict
resolution, leadership, relationship with peers, problem solving, motivation to learn, andbehavior in class. Findings included the following:•••

According to student assessments gathered immediately after program participation,
children who attended outdoor science school showed significant positive gains inconflict resolution. However, the difference between the treatment and control groups on
conflict resolution was not statistically significant at that point in time. Six to ten weeks
later, children who attended the program showed gains in cooperation and conflict
resolution that were significantly higher than the control group.
Teacher ratings provide evidence of a wide range of positive outcomes related toparticipation in outdoor science school. Teachers rated all children before the treatmentgroup attended outdoor school and six to ten weeks later. According to teacher ratings ofeach student, those children who attended outdoor science school showed statisticallysignificant positive gains on all eight constructs on which they were rated. In contrast,
the control group showed losses on seven of the eight constructs. Children who attended
outdoor science school showed significantly larger gains than the control group in six ofthe eight constructs. These gains were observed in self-esteem, conflict resolution,
relationship with peers, problem solving, motivation to learn, and behavior in class.
Parent ratings of their children did not reveal any significant differences in the fivesocial-emotional constructs between children who attended outdoor school and those
who did not.

Stewardship of the Environment•

According to student assessments gathered immediately after program participation,
children who attended outdoor school showed significant increases in one of the threeconstructs: concern about conservation. However, these increases were not significantly
larger than gains by the control group.

American Institutes for Research Page v

Effects of Outdoor Education Programs for Children in California••

At the six- to ten-week point, the control group showed significant losses in two of thethree constructs (attitude toward science and environmental behaviors), whereas thetreatment group did not show any significant losses.
According to parent reports, students who participated in the program had significantly
larger gains in environmental behaviors, compared to children who did not attend theprogram. In other words, parents of children who attended outdoor school observed
children engaging in positive environmental behaviors (e.g., recycling, etc.) at home,whereas a statistically significant finding was not observed for parents of the control
group.

Knowledge and Understanding of Science Concepts••

Children who attended outdoor school significantly raised their science scores by 3
points (27 percent), as measured by a pre- and post-survey administered immediately
upon their return to school.
The increase in science knowledge was maintained six to ten weeks following program
participation, with no significant loss in science scores.

Benefits for English Language Learners•

This study focused on 255 sixth-grade students, 58 percent of which were identified byteachers as English Learner (EL) students. According to teacher reports, among thosestudents who attended the program, EL students demonstrated gains in cooperation,
leadership, relationship with peers, and motivation to learn that were significantly larger
than the gains shown by non-EL students for those constructs.

Input from Teachers, Outdoor Education Staff, and Students
Elementary school teachers and outdoor science school staff overwhelmingly emphasized thepositive outcomes they observed among children who attended the program. These included
increased confidence and self-esteem, positive relationships among students, and reduced
discipline and behavior problems. Teachers reported that the program provided hands-on,
effective science instruction which served as a foundation for subsequent classroom instruction.
The program provided an “opportunity to shine” for all students, including those withdisabilities, special needs, or other at-risk factors. Students who attended outdoor school wereasked if the experience changed them. Of the eighty-three students who provided a response,
sixty-seven percent (56 students) reported a positive change of some sort. Students wrote: “Yes,because I learned more. I like science a lot because it helped me to protect the environment evenmore.” and “I think I have changed after going to Outdoor school. I felt less bored, learned a lot
about science, and made more friends.”

American Institutes for Research Page vi

Effects of Outdoor Education Programs for Children in California

Conclusions
Fifty-six percent of the treatment group reported that outdoor school represented the first timethey had spent time in a natural setting. Participation in outdoor school was associated withhigher ratings of conflict resolution skills and cooperation (longer-term student assessments), andenvironmental behaviors (parent reports). Strong evidence of the benefits of outdoor school isseen in teachers’ ratings of students – students who attended the program received significantly
higher ratings than children who did not participate in six of eight constructs: self-esteem,
conflict resolution, relationship with peers, problem solving, motivation to learn, and behavior inclass. Children who attended outdoor school significantly raised their science scores by 3 points(27 percent), as measured by a pre- and post-survey administered immediately upon their returnto school. The increase in science knowledge was maintained six to ten weeks following
program participation, with no significant loss in science scores. The positive outcomes
associated with students’ participation in the five-day outdoor science school are promising,
especially given the relatively short timeframe of the program.
It is important to note that this study focuses on 255 students enrolled in schools serving at-risk
populations, attending three outdoor science schools. Findings cannot be generalized to allstudents attending outdoor education programs in California, particularly given the range ofprograms that exist. However, this research indicates a large number of positive outcomes for atrisk children who attend resident outdoor science schools certified by the California Departmentof Education.

American Institutes for Research Page vii

Effects of Outdoor Education Programs for Children in California

American Institutes for Research Page viii

Effects of Outdoor Education Programs for Children in California

Chapter 1: Introduction
The American Institutes for Research (AIR) conducted an evaluation to measure the impacts ofweek-long residential outdoor education programs for at-risk sixth graders in California. As
described by California Assembly Bill (AB) Number 1330, Chapter 663, the Outdoor
Environmental Education Program is designed to “foster stewardship of the environment and an
appreciation of the importance of the wise use of natural resources.” The program serves at-riskyouth and underserved demographic groups. AB 1330 called for an independent evaluation of
the program to be conducted by February 1, 2005 to examine the effects of outdoor experienceson students’ behavior and learning.
The study focused on the impacts of three outdoor education programs (also referred to as
outdoor science schools) on participating sixth-grade students. It was designed to determine if
there were measurable changes in students’ attitudes and behaviors in relation to theenvironment, their knowledge of science concepts taught during these programs, and their socialand emotional skills (e.g., relations with peers, behavior, self-esteem). The study is unique in that
it utilizes a “delayed treatment design”, whereby assessments of students who participated in an
outdoor education program (treatment group) are compared to assessments of students whose
participation in the program had not yet occurred (control group). The study was conducted
between July 1, 2004 and January 31, 2005, for the California Department of Education (CDE),
with the support of private funding.
Background
Prior research studies have documented the benefits of outdoor education programs for students.
A 2004 meta-analysis of 150 outdoor learning research studies conducted between 1993-2003
found that there was substantial research evidence to suggest that outdoor adventure programs
are associated with positive outcomes for young people, including attitudes toward theenvironment, independence, confidence, self-esteem, locus of control, self-efficacy, personal
effectiveness and coping strategies; and interpersonal and social skills, such as social
effectiveness, communication skills, group cohesion and teamwork (Rickinson et al., 2004). A
1998 report conducted by the State Education and Environmental Roundtable (SEER)1, indicatedthat programs which use the “Environment as the Integrating Context (EIC)” approach support
higher outcomes for participating students, including higher grades and scores in reading,writing, and math (Lieberman & Hoody, 1998). According to the Roundtable, EIC programs arethose that utilize a set of six best practices, one of which is to “develop knowledge,
understanding, and appreciation of the environment, community, and natural surroundings”.
Many studies of environmental education have focused on qualitative data collection and/or prepost-measures to determine the impacts of such programs. This study used a randomized
treatment and control design to determine the impacts of three one-week resident outdoor scienceschool (ROSS) programs in California for participating at-risk sixth-grade students. The design
provides a rigorous means of identifying the outcomes associated with participation in outdoor
education programs.
1

SEER is a cooperative endeavor of 16 state departments of education, including California, working to improvestudent learning by integrating the environment into K-12 curricula and school reform efforts.

American Institutes for Research Page 1

Effects of Outdoor Education Programs for Children in California

Environmental Education in California
California outdoor science schools are “those in which learning experiences and activities in all
subject matter areas are conducted in a relatively natural environment on an extended three- to
five-day resident basis. Specialized instructors, in addition to the regular classroom teacher, areusually involved. Students have, through such programs, an opportunity to gain an understandingof their interaction and interdependence with the natural world, the urban-suburban world inwhich most of them live, as well as with each other and all human kind.”2 Outdoor scienceschools in California serve fourth- through eight-grade students, using curricula aligned with the
California Science Content Standards.
The California Outdoor School Administrators (COSA) association collaborates with the
California Department of Education to ensure that high quality residential outdoor environmental
education programs are conducted throughout the state by implementing a rigorous ROSS
certification procedure. The certification team is charged with the responsibility of making athorough inspection of the site, its administration and staffing, and the overall program which
serves the students. Steps in the process include: 1) a self-study and evaluation by the local site
administrator and staff; 2) a two-day on-site study and review by an accreditation teamconsisting of COSA members who have demonstrated professional competency in outdoor
school operation and administration, and 3) a final certification by the California Department of
Education to properly complete the process. As of October of 2004, there were 29 ROSScertified outdoor schools in California.
Each of the three outdoor education programs participating in this study has full ROSS
certification, as recognized by the CDE. The following criteria must be met by a program toqualify for full ROSS status:3
•
•
•
•
•
•

Its operation and/or administration must be by a Local Education Agency (LEA).
It provides a written curriculum which helps students understand the interrelationships ofthe living and physical components of the environment and the role of humans asparticipators in the ecosystem.
It has a written curriculum that develops skills, attitudes, knowledge, and commitment
concerning the wise use of natural resources and the protection and enhancement of theenvironment.
It is held at a site, away from the local community of service, at which a variety ofenvironmental phenomena may be observed and studied firsthand.
It has personnel, facilities, materials, and equipment to permit students to observe andstudy environmental phenomena and interrelationships to the best advantage.
It provides appropriate facilities and staff for the feeding and lodging of students andteachers while in attendance at the site.

2

See Johnson and Ivie (2003, page 74, Appendix B, “Resident Outdoor Education in California – A Historical
Perspective”).
3
California Education Code, Article 5, section 8760-8773, Authorization – Article 13.5 Fees Section 35335;
California Health and Safety Code, Division 13, Part 2.3, Section 18897.

American Institutes for Research Page 2

Effects of Outdoor Education Programs for Children in California••

It is a program that requires student participation for at least three consecutive nights.
The program primarily serves students in grades four through eight.

Evaluation Approach
The conceptual framework, or logic model, for the study is based on recent outdoor education
research findings4 and a series of discussions with key advisors from CDE and the threeparticipating outdoor education programs, including program administrators and teaching staff.
The logic model (Exhibit 1) shows the outdoor schools’ inputs, activities/strategies, outputs,
expected initial results, expected intermediate results, and expected long-term results. This typeof model is used to provide an explicit statement of the hypothesized connections between the
inputs, activities, and outputs of ROSS-certified programs and their outcomes or intended results.
Most importantly, logic models have the capacity to illuminate indirect effects by distinguishing
between initial, intermediate, and longer-term impacts.5 The initial impact of a program is theimmediate impact on the target group (e.g., increased student knowledge of scientific facts).
This, in turn, is expected to lead to an intermediate impact (e.g., increased understanding of
science and related academic concepts). The longer-term impacts are those that may be expected
over a longer period of time (e.g., improved academic scores in science and other subjects).
Although some or all of the hypothesized longer-term impacts are not measurable during thetime frame for this evaluation, they are included in the model so as not to lose sight of the notionthat intermediate impacts may lead to even more significant long-term results.
The research questions, performance indicators, and measurement methods for this study werethus based on the hypothesized relationships shown in the logic model. The study focused
primarily on the expected impacts of outdoor education programs on students’ social-emotional
skills and environmentally responsible behavior. Furthermore, we examined impacts on students’knowledge and understanding of the science concepts that comprise the outdoor education
program curriculum.

4

We primarily used the following two studies: Lieberman, Hoody, and Lieberman (2000) and North American Association forEnvironmental Education, and National Environmental Education & Training Foundation (2001).
5
United Way of America (1996).

American Institutes for Research Page 3

Effects of Outdoor Education Programs for Children in California

Exhibit 1: Logic model for outdoor education programs
InputsWhat a program has towork with

Program budget
Facilities
Staffing
Curriculum/Field
GuidesActivities/
Strategies

Outputs

Processes it uses

Teacher and student
pre-session
preparations
Experiential (handson) learning
Individual and group
activities
Paper/pencil tasks
Social activities (e.g.,cleaning up the bed,
washing dishes,
preparing meals)

Expected
Initial Results

Expected
Intermediate Results

Expected Long-term
Results

What it produces

Initial impact on target
group

Intermediate impact ontarget group

Benefit to students

One-week programfor outdoor-based
residential instructiontied to California K-12
content standards

Program participation
increases theknowledge of facts
and skills of students:- environmental ed- science- social studies- other
Experiential andresidential program
increases students’personal and social
skills (e.g., selfesteem, selfawareness, selfregulation,
cooperation, respect,friendship, leadership,
conflict resolution,
independence, selfreliance)

Increased
understanding of
science and related
academic concepts

Students show
improved academic
scores in science andother subjects.

Increased
appreciation forbiodiversity, ecology
of naturalenvironments, andconservation

Students are
motivated to pursue
careers in scienceand related fields.

Increased motivation
to learn
Increased levels ofcooperation andfriendship among
peers
Improved locus ofcontrol
Improved attitude
toward school•
Improved student
attendance•
Decreasedclassroom/school
disciplinary
actions•
Improved student
teacher
relationships.

American Institutes for Research Page 4

Students demonstrateenvironmentally
responsible behavior.
Students show
increased rate of highschool completion.
Students show
increase in thenumber of scienceelectives / AP classes
taken.

Effects of Outdoor Education Programs for Children in California

Research Questions
The specific research questions addressed in this study are as follows:
1) How does participation in outdoor education programs impact students’ personal andsocial skills (e.g., self-esteem, cooperation, teamwork)?
2) How does participation in outdoor education programs foster students’ stewardship of theenvironment and their appreciation of the importance of the wise use of natural
resources?
3) How does the science instruction received through the outdoor education programcurriculum increase students’ knowledge and understanding of science concepts?
Evaluation Design
This study focused on at-risk sixth-grade students from four elementary schools who attended
three outdoor education programs between September and November of 2004. The evaluation
utilized a “delayed treatment design.” Within participating elementary schools, sixth-grade
children were divided, by classroom, into two groups. Approximately half of each school’s sixth
grade (one or more classrooms) attended outdoor school between September and November of2004 and served as the treatment group. The remaining sixth grade classrooms were scheduled toattend outdoor school after the study’s data collection period ended in December 2004, thereby
serving as the control group during the study period. In this manner, the study utilized atreatment and control design without denying any child the opportunity to attend outdoor scienceschool. A full description of the study’s methodological approach is included in Chapter 2.
Findings, addressing each of the three research questions, are included in Chapter 3. Chapter 4
presents qualitative data gathered from elementary school teachers and outdoor education staff.
Chapter 5 includes a summary of findings and conclusion.

American Institutes for Research Page 5

Effects of Outdoor Education Programs for Children in California

American Institutes for Research Page 6

Effects of Outdoor Education Programs for Children in California

Chapter 2: Methodology
This chapter describes the methodology employed during the study. It details the participating
outdoor education programs and elementary schools, the various data collection strategies andtools, and the analyses that were carried out to address the research questions.
Participating Outdoor Education Programs
The CDE, in consultation with AIR, selected three ROSS-certified outdoor science schools as thetarget programs for the study. Geographically diverse, the schools are located in rural areas near
Fresno, Los Angeles, and San Diego. The programs serve fourth- and sixth-grade students during
week-long residential programs, and use curricula that align with the California State Science
Framework and the California Academic Content Standards for science (see Appendix A for alist of the specific academic content standards that align with the outdoor school programcurriculum). The hands-on, inquiry-based curriculum is designed to help students understand theenvironment and the role of humans as participants in ecosystems, as well as develop their skills,
attitudes, knowledge and commitment concerning the natural world. While instructional
activities vary somewhat across the outdoor education programs participating in the study, thecontent of the curricula is consistent, focusing primarily on ecology and earth science.
Across the three outdoor education programs, a total of approximately 31,000 children are served
each year, including children in fourth- through sixth-grade. In a typical year, the three outdoor
schools serve a combined total of approximately 19,000 sixth-grade students (two of the outdoor
schools serve approximately 8,000 sixth-grade students each and the third outdoor school serves
approximately 3,000 sixth-grade students). The outdoor science schools in this study do not
maintain eligibility criteria for participating elementary schools. That is, the outdoor schools
frequently serve elementary schools with at-risk student populations (such as those included inthis study). However, elementary schools representing any socio-economic population have theopportunity to attend the programs.
Students arrive at the program on Monday and live on site, in cabin groups, until they depart on
Friday. A typical day at outdoor school includes morning cabin chores, breakfast, a morning (orall day) “trail” involving science lessons/activities, lunch, an afternoon “trail” for additional
science lessons/activities, dinner, cabin time, and evening activities (e.g., a night hike, social
activity). Students are assigned chores and tasks throughout the week (e.g., setting up for meals,
clean-up). With the help of their cabin leader, students are responsible for keeping their cabinneat and clean, setting and cleaning tables during meals, and working cooperatively with theircabin mates to accomplish the tasks of daily living.
Instruction at outdoor school typically takes place in small groups "on the trail". Concepts thatare taught at the outdoor school are supported by hands-on experiences such as:••••

observing and identifying birds with binoculars and a field guide manual,
searching for and identifying aquatic insects in a stream or pond,
identifying species of trees and drawing conclusions on the "health" of the immediate
forest,hunting for wildlife clues such as animal tracks or scat,
American Institutes for Research Page 7

Effects of Outdoor Education Programs for Children in California

•
•
•
•

collecting mineral specimens and testing them for hardness, pH, and physicalcharacteristics,
looking for examples of natural recycling such as a decomposing log, a fungus or an
animal serving as a decomposer,
observing planets and distant galaxies through the telescope at the observatory, andviewing the night sky and learning about the constellations and theassociated mythology.

Combined with these types of learning activities, outdoor science schools also offer students a
range of social experiences. Students meet peers from different schools, communities,
ethnicities, cultures and religions. Throughout the week, they work together as a team to problem
solve, share meals and chores, and participate in evening social activities. Students takeresponsibility to dress, bathe, and keep themselves groomed without their parents’ supervision.
For many students, outdoor school is the first time they have been away from home for an
extended period.
Outdoor school teaching staff have bachelor’s degrees and many also maintain a California
teaching credential, although it is not required by the programs. Elementary classroom teachers
accompany their students to the program. In general, classrooms teachers join children on
lessons and activities, assisting the lead outdoor school teacher when needed, meeting withchildren informally or during structured times, and attending meals and social activities.
Participating Elementary Schools
Four California elementary schools participated in the study. The original study design called forthree elementary schools with four or more sixth-grade classrooms to participate, each school
attending one of the three outdoor education programs. This was accomplished for two of theoutdoor education programs. The third outdoor education program attempted to recruit anelementary school, located within their service area, with four sixth-grade classrooms. Because
most elementary schools in the program’s service area did not have a student population to fill
four sixth-grade classrooms, two elementary schools (schools 3 and 4 in Table 1) were invited by
the outdoor program to participate in the study. A total of 341 sixth-grade students currently
attend the four participating elementary schools. All sixth-grade children (and one of theirparents or guardians) at the four participating schools were asked to provide consent toparticipate in the study.
Characteristics of the four participating elementary schools are listed in Table 1. As shown, thefour schools serve mostly Hispanic children (ranging from 69 percent to 89 percent of thestudent population) and have a high proportion of English Learners (32 percent to 66 percent ofstudents). Eighty-one to 100 percent of the children in each school qualify for the free andreduced price lunch program.

American Institutes for Research Page 8

Effects of Outdoor Education Programs for Children in California

Table 1. Elementary school characteristics (2003-2004 school year)

School

Total
Enrollment
and 6th Grade
Enrollment*

Student Ethnicity**

1

815 total
studentsth
111 6 gradestudents

EnglishLearners

Free/Reduced
Lunch
Program

API
State
Rank

Hispanic = 80%
Filipino = 11%
Multiple or no response = 4%
White = 3%
African American = 1%
Asian = 1%

50%

100%

3

2

794 total
studentsth
110 6 gradestudents

Hispanic = 89%
White = 2%
African American = 2%
Asian = 1%
Filipino = 7%
Pacific Islander = 1%

66%

100%

3

3

693 total
students
70 6th gradestudents

Hispanic = 79%
Asian = 20%
Filipino = 1%
White = 1%

62%

94%

3

432 studentsth
50 6 gradestudents

Hispanic = 64%
Asian = 26%
White = 7%
African American = 1%
Filipino = 1%
American Indian or Alaska Native = 1%
Pacific Islander = 1%

32%

81%

4

4

Source: California Department of Education, 2003-2004 Data Quest*Sixth-grade enrollment for the 2004-2005 school year, as reported by each school principal.
**Totals may not sum to 100 percent due to rounding error.

Data Collection Strategies
Data collection included the use of surveys and site visits between September and December of
2004. Data were only collected from children for whom informed consent had been granted (viasigned consent forms) by the child and parent. Parents also provided their consent prior tocompleting the parent surveys. Participation in the evaluation did not impact students’opportunity to attend the outdoor school in any way.
Surveys. Students, parents, and teachers were surveyed. AIR staff administered three rounds ofsurveys: before the treatment group attended outdoor school (Round 1 pre-survey), immediately
after the treatment group returned from outdoor school (Round 2 first post-survey), and six to ten
weeks after the treatment group returned from outdoor school (Round 3 second post-survey)6.
Students were surveyed in all three rounds, and parents and teachers were surveyed in Round 1
and Round 3. Student responses from Rounds 1 and 2 were used to determine the immediate
impacts of participation in outdoor education, while student, parent, and teacher responses from
Rounds 1 and 3 were analyzed to explore the longer-term impacts of the program.
6

The Round 3 survey was administered to students in all schools during the same week, at the latest possible pointin the study, in order to determine the longer-term impacts of program participation. Because schools attended theoutdoor education programs at different times, the Round 3 survey was administered at a range of six to ten weeks
following program participation.

American Institutes for Research Page 9

Effects of Outdoor Education Programs for Children in California

Site Visits. Research staff conducted one-day site visits to each of the three outdoor schools
during the same week the treatment group attended each program. Staff observed instructional
activities while on-site and conducted in-person interviews or focus groups with the sixth-gradeteachers of participating students. Teachers were asked to provide input regarding the perceived
benefits of outdoor school for students, how the program impacted specific groups of students,and how the outdoor school experience was integrated into their classroom teaching. In addition,
AIR staff interviewed the principals of the outdoor schools to gather descriptive information andsolicit their input regarding the impacts of the program.
Data Collection Tools
Surveys were administered to students, parents, and elementary school teachers. Copies of thesurveys are included in Appendix B7.
Student Survey. The student survey was designed to collect information on five individual
social-emotional scales, on three scales regarding attitudes toward the environment, and (for thetreatment group only)8 on overall knowledge of science-related topics that are taught at theoutdoor schools. The social-emotional scales and two of the environmental scales were measured
using an 11-point scale (0 Strongly Disagree – 10 Strongly Agree). For example, statements
included “I feel good about myself” and “I like science.” The third environmental scale utilized atwo-point (yes/no) response option (e.g., “Do you separate things at home for recycling?). The
science items were measured through a series of nine multiple-choice questions and one openended question. Students who attended outdoor school also responded to one open-ended
question on the Round 3 survey to collect qualitative responses about the impact of outdoor
school.
Parent Survey. The parent survey contained the same set of items and rating scales included in
the student survey (revised to align with the parent’s perspective), excluding the science-related
items. Parent survey statements included “My child feels good about him/herself.” and “Mychild likes science.” The parent survey was available in English and Spanish.
Table 2 summarizes the scales (constructs) that were included in the student and parent surveys.

7

Items in the surveys that related to the social-emotional and environmental scales were based on a review ofexisting tools used in environmental education research, primarily the Life Effectiveness Questionnaire (Neill &Richards, 2003).
8
The science-related survey items were not administered to children who did not attend outdoor science school (thecontrol group), due to the reality that control group children did not necessarily have the opportunity to learn thesame academic (science) content during the week the treatment group students attended outdoor science school (oreven during the six-week period following the treatment group’s participation, after which the Round 3 surveys wereadministered to both groups). Thus, it would be unfair to compare treatment and control groups on the science
content that was known to have been covered during outdoor school, but not necessarily covered by the control
classrooms during the same week or in the six weeks following. In addition, the content of classroom instruction forchildren who did not attend outdoor education could not be rigorously controlled.

American Institutes for Research Page 10

Effects of Outdoor Education Programs for Children in California

Table 2. Student and parent survey scales
ScalesSocial-Emotional Scales
Self-esteem
Cooperation
Leadership
Conflict resolution
Relationship to teacher
Environmental Scales
Concern about conservation
Attitude toward science
Environmental behaviors
Science Knowledge Scale
Total science score

Number ofSurvey Item
Survey Items Response Options
5
2
1
3
2

11-point scale(strongly disagree tostrongly agree)3
2
16

11-point scale(strongly disagree tostrongly agree)
2-point scale (yes/no)10

Nine multiple choice items
and one open-ended
question, scored as “0”, “.5”,or “1” using a scoring rubric.9
Maximum score = 19 points

We measured the internal consistencies of the five social and personal and the threeenvironmental attitudes scales by Cronbach’s alpha. Items included in the social and personalscales and two of the environmental scales were measured using an 11-point scale (0 Strongly
Disagree – 10 Strongly Agree). Table 2 details the number of survey items used in each scale anda description of the response options. In sum, the numbers of individual survey items included inthese scales are: self-esteem = 5, cooperation = 2, leadership = 110, conflict resolution = 3,
relationship with teacher = 2, attitude toward science = 2, and concern about conservation = 3.
The environmental behavior scale was composed of 16 dichotomous (yes/no) questions bycounting the number of “yes” responses. In order to adjust the different numbers of items
included in these scales, we report Standardized Cronbach’s alpha in Appendix C. The “attitude
toward science” construct had a relatively low alpha coefficient (.519) in the first round of
survey data, however, this same construct shows higher coefficients in the two subsequent datacollection rounds. All of the other constructs showed an acceptable level of reliability.
Teacher Survey. Teachers completed a short assessment of each child, using the 0 to 10 rating
scale on eight constructs: self-esteem, relationships with peers, effective problem solving,
conflict resolution, cooperation and teamwork, effective leadership, motivation to learn, andbehavior in class. In the second teacher survey, teachers of the treatment group students also
responded to open-ended questions regarding how the outdoor school program impacted theirstudents.

9

The open-ended question on the science section of the survey stated, “List two new ideas you can do to take care of
the Earth.” Student responses were scored using a “0”, “.5”, and “1” rating system, based on a scoring rubric. “0”responses were incomprehensible or nonapplicable. “.5” responses were applicable but nonspecific. “1” responses
were applicable and included specific ways to take care of the Earth.
10
The leadership scale originally included two items; however, one of the items used a reverse question, whichcaused inconsistency with the other item. Thus, we decided to drop the reverse question item and the leadershipscale subsequently consisted of one item.

American Institutes for Research Page 11

Effects of Outdoor Education Programs for Children in California

Survey Responses
Table 3 shows the number of students, parents, and teachers for whom we asked for consent toparticipate in the study, the number of consenting participants, and the number of surveys
received from students, parents, and teachers in each round of data collection.
Table 3. Survey responses

Survey Respondents

Number ofParticipants

Number of Surveys Received,by Data Collection Round

Number ofConsenting
Participants

Round 1Round 2Round 3Students

341

255

245

216

236

Parents

341

257

212

n/a*

193

Teachers

14

14

14

n/a*

14

*Parents and teachers were only surveyed in Round 1 and Round 3 of the study.

Table 4 reflects the number of treatment and control children in the study, by data collection
round. In all but one school, children were randomly assigned to the treatment and control
groups at the classroom level. In this school, three lead teachers taught two groups of sixth-grade
children equally and the school did not consider the groups as separate classrooms. In this case,the school preferred to randomly select children on an individual level, rather than create
artificial classroom groups.
Table 4. Number of treatment and control student surveys received, by data collection round
Student Group

Round 1Round 2Round 3Treatment

125

127

119

Control

109

118

106

“Drop-outs” from Treatment Group*

0

10

11

*Some children originally assigned to the treatment group (attending outdoor school) did not participate in the
program (e.g., they were ill, their parents decided to not send their child, etc.).

Data Analysis
The following section describes the strategies used to analyze the quantitative survey data andthe qualitative interview and focus group information.
Quantitative Analysis. After data collection was completed, the survey results were recorded aselectronic data files. Scales for five social and personal constructs, three environmental attitude
scales, and an overall science score, were developed from individual survey items. The reliabilityof these constructs (scales) was assessed by calculating Cronbach’s alpha, which measures theextent to which the scale items are measuring a common, underlying construct. Appendix C
includes the exploratory analyses.
Based on an analysis plan developed for the study, two independent sample t-tests were used todetect statistically significant differences between various student groups and subgroups (e.g.,treatment versus control groups, male versus female, Hispanic versus non-Hispanic students).

American Institutes for Research Page 12

Effects of Outdoor Education Programs for Children in California

Paired-sample t-tests were employed to examine significant gain scores within groups. Similar
analyses were conducted for survey data from parents and teachers. The criterion used forstatistical significance was p<.0511.
Qualitative Analysis. The qualitative data collected through the site visits, interviews, andsurveys were reviewed and common themes were identified across respondents. The qualitative
data are used primarily to provide context for the quantitative survey findings, presented in
Chapter 3.
Generalizations from the Sample. This study focused on 255 students, from four elementary
schools, attending three ROSS-certified outdoor education programs in California. It is important
to note that the study sample was primarily composed of Hispanic children, a large minority ofwhom were English Learners. Given the scope of the study, research findings should not be
generalized to all sixth-grade students who attend outdoor education programs in California,
particularly given the fact that there are a range of outdoor education programs, privately andpublicly funded, with different objectives and structure within the state. However, as described in
the following chapter, findings from this study indicate positive outcomes for at-risk sixth-gradestudents who attend established, ROSS-certified outdoor education programs.

11

Many of these analyses were intended to be exploratory in nature. For example, while our major interest was
overall program impact, we also investigated whether the program might have differential impacts on certain
subgroups (i.e., males versus females; Hispanics versus non-Hispanics). One often adjusts the alpha level
downward to compensate for the increased probability of error when multiple statistical tests are performed on thesame data. Since the objective of our exploratory analyses was the identification of possible effects (to enablefurther focused research rather than the scientific demonstration of these impacts, we did not adjust the statisticalsignificance criterion to account for the number of statistical tests conducted. (See Pernerger, 1998, who further
argues that, even when the use of such adjustments might be considered, the most common of these [such as theBonferroni adjustment] are too conservative and increase the likelihood of Type II errors.) Nonetheless, when
Bonferroni adjustments would result in findings being deemed non-statistically significant, these findings arefootnoted.

American Institutes for Research Page 13

Effects of Outdoor Education Programs for Children in California

American Institutes for Research Page 14

Effects of Outdoor Education Programs for Children in California

Chapter 3: Research Findings
In this chapter, we present the study findings for each of the three research questions.
Quantitative student, teacher, and parent survey data are used to address each research question.
Qualitative information collected through the site visits, interviews, and open-ended survey
responses is presented in Chapter 4, to help provide context for the findings.
For each research question, we present data from student, parent, and/or teacher surveys.
Students, parents, and teachers were surveyed before the treatment group attended the outdoor
science schools and then six to ten weeks after the treatment group’s return. All students werealso surveyed the week following the treatment group’s program participation, in order todetermine the more immediate student-level impacts of the program. Therefore, student survey
responses are presented to explore both the immediate and longer-term impacts of program
participation. Table 5 shows the types of survey data that were used to address each of the threeresearch questions.
Table 5. Survey data used to address each research question
Data Source:
Immediate Impacts(Round 1 and 2
Surveys)

Data Source:
Longer-term Impacts(Round 1 and 3
Surveys)

RQ1: How does participation in outdoor
education programs impact students’ personaland social skills?

Student Survey
Parent Survey
Teacher Survey

Student Survey
Parent Survey
Teacher Survey

RQ2: How does participation in outdoor
education programs foster students’ stewardshipof the environment and their appreciation of theimportance of the wise use of natural resources?

Student Survey
Parent Survey

Student Survey
Parent Survey

RQ3: How does the science instruction received
through the outdoor education programcurriculum increase students’ knowledge andunderstanding of science concepts?

Student Survey

Student Survey

Research Question (RQ)

Research Question #1: How does participation in outdoor education programs
impact students’ personal and social skills (e.g., self-esteem, cooperation, etc.)?
To determine the impact of outdoor education programs on students’ personal and social skills,
students and parents were surveyed on five constructs: self-esteem, leadership, cooperation,
conflict resolution, and students’ relationship with their teacher. Teachers provided a global
rating (rather than responding to multiple survey items for each construct) for each child on selfesteem, relationships with peers, effective problem solving, conflict resolution, cooperation andteamwork, effective leadership, motivation to learn, and behavior in class.

American Institutes for Research Page 15

Effects of Outdoor Education Programs for Children in California

Student Assessments
Immediate Impacts of Outdoor Education. Table 6 shows the student assessment scores fromthe pre-survey to the first post-survey (immediate impacts of program participation) for thetreatment and control groups. It includes students’ average pre-survey scores, average postsurvey scores, the average gain, and percent gain. According to student data, children who
participated in outdoor school showed positive gains on all five constructs: self-esteem,
cooperation, leadership, conflict resolution, and relationship with teacher, immediately after
program participation. In contrast, children who did not attend the outdoor school showed losses
on two of the five constructs: cooperation and conflict resolution, although these losses were notstatistically significant. The control group also showed significant gains in leadership (p<.05).
Of the five constructs in which children who attended outdoor school showed immediate gains,
one construct showed statistically significant gains: conflict resolution (p<.05). However, thedifference between the treatment and control groups on conflict resolution was not statisticallysignificant at that time.
Table 6. Student assessments, (mean scores), pre-survey to first post-survey (immediate) gains in
self-esteem, cooperation, leadership, conflict resolution, and relationship with teacher
Group

N

Pre-survey

Post-Survey

Gain

Pct Gain

0.19
0.22

2.33%
2.69%

Self EsteemControl
Treatment

106
125

8.34
8.07

8.54
8.29
Leadership

Control
Treatment

102
118

7.04
7.53
6.61
7.11
Relationship with Teacher

0.49!
0.50

6.96%
7.56%

Control
Treatment

106
124

8.24
7.85

8.34
8.11
Cooperation

0.10
0.27

1.26%
3.39%

Control
Treatment

106
125

8.13
7.68

8.00
7.75
Conflict Resolution

-0.13
0.08

-1.57%
0.99%

Control
Treatment

106
125

7.70
7.08

-0.08
0.41!

-1.08%
5.77%

7.61
7.48

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between treatment and control group gain scores are indicated by asterisks:*p<.05; **p<.01; ***p<.001.

American Institutes for Research Page 16

Effects of Outdoor Education Programs for Children in California

Longer-Term Impacts of Outdoor Education. Students completed the same survey six to ten
weeks after the treatment group’s program participation. Table 7 shows the student assessmentscores from the pre-survey to the second post-survey (longer-term impacts of program
participation) for the control and treatment groups. It includes students’ average pre-survey
scores, average second post-survey scores, the average gain, and percent gain.
Table 7. Student assessments, (mean scores), pre-survey to second post-survey (longer-term)
gains in self-esteem, cooperation, leadership, conflict resolution, and relationship with teacher
Group

N

Pre-survey

2nd PostSurvey

Gain

Pct Gain

Self EsteemControl
Treatment

102
110

8.42
8.04

8.45
8.15
Leadership

0.03
0.11

0.34%
1.38%

Control
Treatment

101
103

7.11
7.27
6.41
6.99
Relationship with Teacher

0.16
0.58!

2.23%
9.02%

Control
Treatment

102
110

8.09
7.86

0.00
0.32

0.06%
4.10%

Control
Treatment

102
110

8.15
7.93
7.57
7.93
Conflict Resolution

-0.23
0.35*

-2.77%
4.68%

Control
Treatment

102
110

7.87
7.06

-0.66!!
0.46!***-8.33%
6.50%

8.09
8.19
Cooperation

7.21
7.52

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between treatment and control group gain scores are indicated by asterisks:*p<.05; **p<.01; ***p<.001.

Six to ten weeks following program participation, children who attended outdoor school showedsignificant positive gains on two of the five constructs: leadership (p<.05) and conflict resolution(p<.05). In contrast, the control group showed losses on two of the five constructs. One of thesecontrol group losses was significant (conflict resolution, p<.01).
Children who attended outdoor education had significantly higher gains in cooperation andconflict resolution, compared to children who did not participate in the program (p<.05 andp<.001, respectively). The treatment group showed its largest gain (9 percent) in the area ofleadership.
In sum, according to student assessment data, there were no significant differences between
children who attended outdoor science school and those who did not attend, immediately after
program participation. However, six to ten weeks later, children who attended the program
showed significantly higher gains in cooperation and conflict resolution, compared to children
who did not participate.

American Institutes for Research Page 17

Effects of Outdoor Education Programs for Children in California

Parent Reports
Parents completed surveys before the treatment group attended outdoor school and six to ten
weeks later. Parent survey data are presented in Table 8. It includes average pre-survey scores,
average post-survey scores, the gain, and percent gain.
Table 8. Parent reports, (mean ratings), pre-survey to post-survey gains in self-esteem,
cooperation, leadership, conflict resolution, relationship with teacher
Group

N

Pre-survey

Post-Survey

Gain

Pct Gain

Self EsteemControl
Treatment

69
92

8.46
8.35

8.51
8.34
Leadership

0.05
-0.02

0.57%
-0.23%

Control
Treatment

68
92

7.80
7.67
7.27
7.49
Relationship with Teacher

-0.13
0.22

-1.70%
3.03%

Control
Treatment

69
92

8.43
8.53

0.05
-0.26

0.60%
-3.06%

-0.07
-0.11

-0.83%-1.25%

0.14
0.19

1.79%
2.43%

8.48
8.27
Cooperation

Control
Treatment

68
92

8.47
8.45

8.40
8.35

Conflict Resolution
Control
Treatment

69
92

7.84
7.76

7.98
7.95

Parent ratings of their children did not reveal any significant differences in the five socialemotional constructs between children who attended outdoor school and those who did not.
According to parent reports, children who attended outdoor school showed positive gains on twoof the five constructs (leadership and conflict resolution), although these gains were notsignificant. In addition, the treatment group showed losses on three of the five constructs (selfesteem, relationship with teacher, and cooperation), yet none of these losses were statisticallysignificant.
Teacher Reports
Teachers rated all children before the treatment group attended outdoor school and six to ten
weeks later. Teacher survey data is presented in Table 9, which includes ratings on eight
constructs for the control and treatment groups, with the average pre-survey rating, average postsurvey rating, the average gain, and the percent gain.

American Institutes for Research Page 18

Effects of Outdoor Education Programs for Children in California

Table 9: Teacher ratings, (mean ratings), pre-survey to post-survey gains in self-esteem,
cooperation, leadership, conflict resolution, relationship with peers, problem solving, motivation
to learn, and behavior in class
Group

N

Pre-survey

Post-Survey

Gain

Pct Gain

Self-EsteemControl
Treatment

115
123

7.18
6.49

Control
Treatment

115
118

Control
Treatment
Control
Treatment

7.04
7.11
Leadership-0.14
0.63***!!!

-1.94%
9.65%

5.79
6.41
6.56
7.04
Relationship with Peers

0.62!!
0.48!!

10.66%
7.36%

115
123

7.64
7.14
7.13
7.96
Motivation to Learn-0.50!!
0.83***!!!

-6.60%
11.63%

115
123

7.10
7.15

-0.20
0.31*!

-2.82%
4.32%

6.90
7.46
Cooperation

Control
Treatment

115
123

7.17
7.17
7.40
7.76
Conflict Resolution

-0.01
0.37!

-0.12%
4.95%

Control
Treatment

115
123

6.52
6.59

-0.05
0.77***!!!

-0.80%
11.73%-0.16
1.28***!!!

-2.37%
20.44%-0.20
0.26*!

-2.79%
3.40%

6.47
7.36
Problem Solving

Control
Treatment

115
123

6.62
6.24

6.46
7.52
Behavior in Class

Control
Treatment

108
123

7.31
7.66

7.10
7.92

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between treatment and control group gain scores are indicated by asterisks:*p<.05; **p<.01; ***p<.001.

Teacher ratings of students provide strong evidence of the positive outcomes associated withparticipation in outdoor education. Students who attended outdoor science school showedstatistically significant positive gains on all eight constructs. In contrast, children who did notattend the program showed losses on seven of the eight constructs, with one of these losses
significant (relationship with peers). The control group showed only one significant gain, in thearea of leadership.
Children who attended outdoor science school showed gains in six constructs that weresignificantly larger than the gains shown by children who did not: self-esteem (p<.001), conflict
resolution (p<.001), relationship with peers (p<.001), problem solving (p<.001), motivation tolearn (p<.05), and behavior in class (p<.05). The largest gain for children who attended outdoor
school (20 percent) was in the area of problem-solving.

American Institutes for Research Page 19

Effects of Outdoor Education Programs for Children in California

Although the control group showed larger gains than the treatment group in leadership, thecontrol group’s pre-survey scores were significantly lower than the treatment group’s pre-survey
scores for that construct. It is reasonable to assume that the control group had more room to growin this area, and that the gain differential is artificially large. Likewise, the treatment group
showed significantly lower pre-survey scores in self-esteem and relationship to peers than thecontrol group. However, the control group showed losses on these constructs.
It is important to point out that teacher ratings may have been affected by the teachers’
knowledge of which children participated in the outdoor science school and which did not. It is
possible that teachers generally have favorable attitudes towards programs such as the outdoor
science schools their students attend. Thus, although teachers of students in both the treatment
and control groups were scheduled to send their students to outdoor science school at some point
during the 2004-2005 school year, the teacher ratings of students, indicating a number of positive
outcomes associated with program participation, should be interpreted with this caveat in mind.
Personal and Social Skills Subgroup Analyses
Analyses were conducted on the student survey data to determine if there were significant
differences between the participating elementary schools on the five social-emotional constructs.
For the purposes of analysis, Schools 3 and 4 were combined and listed in the tables as School 3(as noted in Chapter 2, Schools 3 and 4 were smaller schools that were recruited to attend thesame outdoor education program). In addition, differential effects for subgroups within thetreatment groups (EL status, gender, and race) were explored. In any construct where one
group’s gain was significantly larger than another group (e.g. male versus female, etc.), we tested
the differences in the pre-survey scores. This would tell us whether or not one group hassignificantly more room to grow than another group, and whether it could be reasonable toassume that the demonstrated gain differential was artificially large. If we found the pre-survey
scores to be significantly different, we ran a regression model controlling for the pre-survey
scores.
Between-school Differences. Among the five social-emotional constructs, significant shortterm between-school differences in gains were found in the construct of leadership, as shown in
Table 10. Short-term gains by students at School 1 were significantly lower than gains by
students at Schools 2 and 3, and pre-test scores across schools were not significantly different.
Table 10. Between-school differences in leadership, pre-survey to first post survey (short-term)
gains, by school
Leadership

School 1 (N=35)
School 2 (N=33)
School 3 (N=50)

Pre-survey

1st Post-Survey

7.20
5.91
6.66

6.37
7.06
7.66

Gain: Pre to 1st
Post surveys-0.83*
1.15
1.00!!

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between schools’ gain scores are indicated by asterisks: *p<.05; **p<.01;***p<.001.

American Institutes for Research Page 20

Effects of Outdoor Education Programs for Children in California

Significant long-term between-school differences in gains were found in the construct of selfesteem, as shown in Table 11. Long-term gains by students at School 1 were significantly lower
than gains by students at Schools 2 and 3, and pre-test scores across schools were notsignificantly different.
Table 11. Between-school differences in self-esteem, pre-survey to second post survey (longerterm) gains, by school
Self Esteem

School 1 (N=29)
School 2 (N=32)
School 3 (N=49)

Pre-survey

2nd Post-Survey

8.24
7.72
8.13

7.56
8.08
8.54

Gain: Pre to 2nd
Post surveys-0.68*
0.36
0.42!

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between schools’ gain scores are indicated by asterisks: *p<.05; **p<.01;***p<.001.

Subgroup Effects. Based on student reports, we did not detect any significant differences
between EL students and non-EL students, male and female students, or race groups on any of
the five social-emotional constructs.
We also examined teacher reports of EL students versus non-EL students to detect any subgroupdifferences. Among students who attended outdoor science school, data from teacher reports
indicate that English Learner (EL) students benefited more than non-EL students in some areas.
As shown in Table 12, EL students showed significant gains on all of the eight constructs thatwere measured, whereas non-EL students showed significant gains in two constructs (conflict
resolution and problem solving). EL students demonstrated gains in cooperation, leadership,
relationship with peers, and motivation to learn that were significantly larger than the gains
shown by non-EL students in those constructs. Both EL and non-EL students showed theirlargest gains (22 percent and 15 percent, respectively) in the area of problem solving. In sum,
among students who attend outdoor science school, these findings suggest the program is
especially beneficial to EL students across a number of social and personal skills.

American Institutes for Research Page 21

Effects of Outdoor Education Programs for Children in California

Table 12. Teacher reports, (mean ratings), English Learners (EL) versus Non-English Learners,(Non-EL) pre-survey to post-survey gains in self-esteem, cooperation, leadership, conflict
resolution, relationship with peers, problem solving, motivation to learn, and behavior in class
Group

N

Pre-survey

Post-Survey

Gain

Pct Gain

0.70!!!
0.39

10.97%
5.91%

Self-Esteem
EL
Non-EL

87
33

6.39
6.67

7.09
7.06
Leadership

EL
Non-EL

82
33

6.54
7.23
6.58
6.48
Relationship with Peers

0.70!!!*-0.09

10.63%-1.38%

EL
Non-EL

87
33

6.97
8.03
7.39
7.58
Motivation to Learn

1.07!!!**
0.18

15.35%
2.46%

EL
Non-EL

87
33

7.17
7.12

0.59!!!**
-0.45

8.17%
-6.38%

7.76
6.67
Cooperation

EL
Non-EL

87
33

7.36
7.93
7.42
7.24
Conflict Resolution

0.57!!*
-0.18

7.81%
-2.45%

EL
Non-EL

87
33

6.62
6.33

0.80!!!
0.73!!

12.15%
11.48%

7.43
7.06
Problem Solving

EL
Non-EL

87
33

6.23
6.24

7.61
7.18
Behavior in Class

1.38!!!
0.94!!!

22.14%
15.05%

EL
Non-EL

87
33

7.70
7.55

8.05
7.52

0.34!!
-0.03

4.48%-0.40%

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between EL and non-EL gain scores are indicated by asterisks: *p<.05; **p<.01;***p<.001.

Summary of Research Question #1 Findings. Participation in outdoor education was
associated with significantly higher ratings of cooperation and conflict resolution (longer-term
student assessments). According to teacher ratings, children who attended outdoor school
showed gains in six of the eight constructs measured (self-esteem, conflict resolution,
relationship with peers, problem solving, motivation to learn, and behavior in class) that weresignificantly larger than the gains shown by the control group. Parent data did not reveal any
significant differences in the five social-emotional constructs between children who attended
outdoor school and those who did not.12
12

For the student data, if Bonferroni adjustments are applied, the significant differences between the treatment andcontrol groups for the construct of cooperation become insignificant. For the teacher ratings of students, if
Bonferroni adjustments are applied, the significant differences between the treatment and control groups for
motivation to learn and behavior in class become insignificant. In addition, significant differences between EL andnon-EL students for leadership, relationship with peers, motivation to learn, and cooperation become insignificant.

American Institutes for Research Page 22

Effects of Outdoor Education Programs for Children in California

Research Question #2: How does participation in outdoor education programs
foster students’ stewardship of the environment and their appreciation of theimportance of the wise use of natural resources?
Students and parents were surveyed to determine the impact of program participation on
students’ stewardship of the environment and their appreciation of the wise use of natural
resources. Three scales were used to address this research question: concern about conservation,
environmental behaviors, and attitude toward science. Following are some examples from theitems that made up each of these constructs:•••

Concern about conservation− It worries me when I see people use too much water.
− It makes me happy when people recycle used bottles, cans, and paper.
Environmental behaviors− Do you separate things at home for recycling?
− Do you leave the refrigerator door open while you decide what food to take out ofit?
Attitude toward science− When I grow up, I want to be a scientist.
− I like science.

Data from student and parent surveys are included below. Teachers did not rate their students onthese constructs due to concern for balancing teachers’ burden with maximizing their input on
students’ social and emotional skills, classroom behavior, and motivation toward learning. In
addition, we would expect that students’ appreciation of the environment and behaviors such as
recycling and energy conservation would be observed and reported best by parents and students.
Student Assessments
Immediate Impacts of Program Participation. Table 13 shows students’ gains from the presurvey to the first post-survey on three scales: concern about conservation, environmental
behaviors, and attitude toward science. The treatment group showed a significant gain inconcern about conservation (p<.05), although it was not significantly higher than the control
group. While participating students showed a greater gain in their attitude toward science (5.81
percent) compared to students who did not attend the program (.64 percent), the difference
between the two groups was not statistically significant. In addition, while the treatment group
showed a decrease in environmental behaviors, it was not statistically significant compared to thecontrol group. Table 13 shows the average pre-survey and first post-survey rating, the average
gain, and the percent gain for the treatment and control groups.

American Institutes for Research Page 23

Effects of Outdoor Education Programs for Children in California

Table 13. Student assessments, (mean scores), pre-survey to first post-survey gains (immediate
gains) in concern about conservation, attitude toward science, and environmental behaviors
Group

N

Pre-survey

Post-Survey

Gain

Pct Gain

Concern about Conservation
Control
Treatment

106
125

6.89
6.81
6.11
6.62
Attitude Toward Science

-0.08
0.51!

-1.14%
8.28%

Control
Treatment

106
123

5.91
5.24

0.04
0.30

0.64%
5.81%

-0.40
-0.10

-4.38%-1.18%

5.94
5.55

Environmental Behaviors
Control
Treatment

105
125

9.13
8.84

8.73
8.74

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between treatment and control group gain scores are indicated by asterisks:*p<.05; **p<.01; ***p<.001.

Longer-Term Impacts. Students completed the same survey six to ten weeks later. Table 14
shows students’ gains from the pre-survey to the second post-survey on three scales: concern
about conservation, environmental behaviors, and attitude toward science. Six to ten weeks after
program participation, students who attended outdoor school did not show any significant losses
or gains in the three scales. In contrast, the control group showed significant losses on two of thethree scales: attitude toward science and environmental behaviors.
Based on student assessments, these findings suggest that six to ten weeks following
participation, outdoor science school did not result in significant gains for students. However, thelosses observed among students who did not attend the program suggest outdoor school may
have some effect on students’ behaviors and attitudes related to the environment and science.

American Institutes for Research Page 24

Effects of Outdoor Education Programs for Children in California

Table 14. Student assessments, (mean scores), pre-survey to second post-survey (longer-term)
gains in concern about conservation, attitude toward science, and environmental behaviors
Group

N

Pre-survey

2nd PostSurvey

Gain

Pct Gain

Concern about Conservation
Control
Treatment

102
110

6.65
6.21
6.09
5.87
Attitude Toward Science

-0.44
-0.22

-6.61%-3.58%

Control
Treatment

102
108

6.05
5.19

-0.80!!!
-0.25-13.17%-4.72%-0.65!
-0.11-7.12%-1.29%

5.25
4.95

Environmental Behaviors
Control
Treatment

102
115

9.09
8.79

8.44
8.68

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between treatment and control group gain scores are indicated by asterisks:*p<.05; **p<.01; ***p<.001.

Parent Reports
Parents rated their children on the same three scales used in the student survey: concern about
conservation, attitude toward science, and environmental behaviors. Parent data are presented inTable 15.
Table 15. Parent reports, (mean ratings), pre-survey to post-survey gains in concern about
conservation, attitude toward science, and environmental behaviors
Group

N

Pre-survey

Post-Survey

Gain

Pct Gain

Concern about Conservation
Control
Treatment

67
91

6.42
6.69
6.65
6.81
Attitude Toward Science

0.27
0.16

4.18%
2.37%

Control
Treatment

69
92

6.43
5.89

-0.43
0.07

-6.76%
1.11%-0.46
0.90!*-4.99%
9.42%

6.00
5.96

Environmental Behaviors
Control
Treatment

67
89

9.27
9.54

8.81
10.44

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between treatment and control group gain scores are indicated by asterisks:*p<.05; **p<.01; ***p<.001.

American Institutes for Research Page 25

Effects of Outdoor Education Programs for Children in California

According to parent reports, children who went to outdoor school showed positive gains in all
three scales: their concern about conservation, attitude toward science, and environmental
behaviors. Of these, students who participated in the program had significantly larger gains inenvironmental behaviors (p<.05), compared to children who did not attend the program.
Stewardship of the Environment Subgroup Analyses
Analyses were conducted on the student survey data to determine if there were significant
differences across the participating elementary schools on the three environmental scales(concern about conservation, attitude toward science, and environmental behaviors). For the
purposes of analysis, Schools 3 and 4 were combined and shown as School 3 in the tables. In
addition, differential effects for subgroups within the treatment groups (EL status, gender, andrace) were explored. In any construct where one group’s gain was significantly larger thananother group (e.g. male versus female, etc.), we tested the differences in the pre-survey scores.
This would tell us whether or not one group has significantly more room to grow than anothergroup, and whether it could be reasonable to assume that the demonstrated gain differential was
artificially large. If we found the pre-survey scores to be significantly different, we ran aregression model controlling for the pre-survey scores. For these analyses, we include a table forany significant findings.
Between-school Differences. Students at School 1 gained significantly less long-term (six to ten
weeks following program participation) than students at School 3 in concern about conservation.
Student gains in School 2 were not significantly different from gains by students at other schools.
These results are shown in Table 16.
Table 16. Between-school differences, pre-survey to second post-survey (longer-term) gains inconcern about conservation, by school
Concern About Conservation

School 1 (N=29)
School 2 (N=32)
School 3 (N=49)

Pre-survey

2nd Post-Survey

5.66
6.30
6.21

4.40
5.67
6.88

Gain: Pre to 2nd
Post surveys-1.25*-0.64
0.67** Difference between schools is significant at p<.05

In addition, School 1 showed a significant long-term loss (six to ten weeks following program
participation) in environmental behaviors. Students at School 1 gained significantly less longterm in this construct than students at School 3. School 2 student gains in environmental
behaviors were not significantly different from gains by students at other schools. These results
are shown in Table 17.

American Institutes for Research Page 26

Effects of Outdoor Education Programs for Children in California

Table 17. Between-school differences, pre-survey to second post-survey (longer-term) gains inenvironmental behaviors, by school
Environmental Behaviors

School 1 (N=31)
School 2 (N=32)
School 3 (N=52)

Pre-survey

2nd Post-Survey

7.61
8.97
9.38

5.87
9.09
10.10

Gain: Pre to 2nd
Post surveys-1.74!!*
0.13
0.71*

Statistically significant changes in scores, within school, are indicated by exclamation points: !p<.05; !!p<.01;!!!p<.001.
Statistically significant differences between schools are indicated by asterisks: *p<.05; **p<.01; ***p<.001.

Subgroup Effects. There were no significant differences between males and females on any of
the three scales (concern about conservation, attitude toward science, and environmental
behaviors). In regard to subgroup analyses for race, we found there was no significant differencein short-term gains between Hispanics and Non-Hispanics. However, Non-Hispanics gained
significantly more than Hispanics in just one of the three areas assessed, concern about
conservation, from the pre-survey to the second post-survey. These results are shown in Table
18.
Table 18. Student assessments, Hispanics versus Non-Hispanics, pre-survey to second postsurvey gains in concern about conservation
Pre-survey
Hispanic (N=75)
Non-Hispanic (N=33)

2nd Post-Survey

Concern About Conservation
6.27
5.49
5.69
6.94

Gain: Pre to 2nd
Post surveys-0.77!
1.25!**

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between Hispanic and Non-Hispanic gain scores are indicated by asterisks: *p<.05;**p<.01; ***p<.001.

In regard to subgroup analyses for English Learner students versus non-English Learner students,
we found that Non-English Learners gained significantly more, from the pre- to second-post
survey, than English Learners in one scale: attitude toward science. These results are displayed inTable 19. There were no short-term (pre-survey to the first post-survey) differences between thetwo groups.
Table 19. Student assessments, non EL-students versus EL students, pre-survey to second postsurvey gains in attitude toward science
Gain: Pre to 2nd
Pre-survey
2nd Post-Survey
Post surveys
Attitude Toward Science
EL (N=78)
5.19
4.51
-0.68
Non-EL (N=29)
5.22
5.97
0.74*
Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between EL and Non-EL gain scores are indicated by asterisks: *p<.05; **p<.01;***p<.001.

Summary of Research Question #2 Findings. According to student assessments, children who
attended outdoor education showed a significant increase in concern about conservation,
American Institutes for Research Page 27

Effects of Outdoor Education Programs for Children in California

immediately after program participation. However, this gain was not significantly different thanthe control group. Six to ten weeks following program participation, there were no significant
differences between the treatment and control groups. The control group, at the six-to-ten week
point, showed significant losses in two of the three constructs (attitude toward science andenvironmental behaviors), whereas the treatment group did not show any significant losses.
Parent reports indicated that children who attended outdoor school showed significantly higher
gains in environmental behaviors, compared to children who did not attend. In other words,
parents of children who attended outdoor school observed children engaging in positiveenvironmental behaviors (e.g., recycling, etc.) at home, whereas a statistically significant finding
was not observed for parents of the control group13.

Research Question #3: How does the science instruction received through theoutdoor education program curriculum increase students’ knowledge andunderstanding of science concepts?
Students who attended the outdoor science school were surveyed on a set of ten science-related
questions. Survey items were adapted from existing assessment tools used by outdoor education
programs and from assessment questions tied to ACPE: A Child’s Place in the Environment, aseries of six environmental education curriculum guides for elementary school teachers that
integrates science, English-language arts, and selected children's literature, and that culminateswith student projects which enhance their environment and provide experiences in service
learning14. A copy of the student survey, including the ten science-related questions, is attachedin Appendix B.
The science-related survey items were not administered to children who did not attend outdoor
science school (the control group), due to the reality that control group children did notnecessarily have the opportunity to learn the same academic (science) content during the week
the treatment group students attended outdoor science school (or even during the six-week period
following the treatment group’s participation, after which the Round 3 surveys wereadministered to both groups). Thus, it would be unfair to compare treatment and control groups
on the science content that was known to have been covered during outdoor school, but notnecessarily covered by the control classrooms during the same week or in the six to ten weeks
following.
In addition, the content of classroom instruction for children who did not attend outdoor
education could not be rigorously controlled. In other words, it was beyond the scope of thisevaluation to monitor classroom instruction during the week that the treatment group attended
outdoor school or during the following six to ten weeks. For example, some control groupstudents might have received instruction specifically focused on ecology, while others did not.

13

If Bonferroni adjustments are applied to the student data, the significant differences between EL students and nonEL students for attitude toward science become insignificant. For the parent reports, the significant differences
between the control and treatment groups for environmental behaviors become insignificant.
14
ACPE and its assessment questions were developed by the Lake County Office of Education, with funding andsupport from the Office of Environmental Education within the CDE.

American Institutes for Research Page 28

Effects of Outdoor Education Programs for Children in California

If the science items had been administered to the treatment and control groups within theparameters of this study and significant differences were found between the two groups, we
could not have attributed those findings to program participation without additional controls. For
example, if the treatment group showed a significant gain in science knowledge, compared to thecontrol group, we would not be able to attribute those gains to participation in outdoor scienceschool, without rigorously controlling for classroom instruction related to the science concepts
assessed. Similarly, if there were no significant differences found between the two groups, we
could not report that outdoor science school had no impact on students’ science knowledge,
given that the study did not control for classroom science instruction for both groups. Withinthese limitations, the approach we chose was to administer pre- and post-surveys to the treatmentgroup, acknowledging that the findings would not be entirely conclusive with regard to theimpact of the program on children’s understanding and knowledge of science topics. Treatmentgroup students were administered pre- and post-tests to determine gains (or losses) in theirknowledge and understanding of the science concepts that were actually taught at the week-longoutdoor school programs.
In addition, the same science test was administered to the treatment group before their
participation in outdoor science school, immediately after they returned from the program, andsix to ten weeks later. By the third administration of the science test, students may have become
familiar with the survey items, which could have affected their science scores. These “practiceeffects” should also be considered as possible confounds in the interpretation of results.
Student Assessments
Immediate Impact of Program Participation. Table 20 shows the immediate impact of
outdoor science schools for treatment group children on their knowledge and understanding of
science concepts.
Table 20. Immediate gains (mean scores) in science knowledge for children who attended outdoor
education programs
Pre-survey

1st Post-Survey

Gain

Pct Gain

Science Scores (N=125)
Mean
Std Dev

10.6
3.3

13.5
2.9

2.9!!!
3.5

27.1%
--

Statistically significant changes in scores are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.

The total science score is based on ten science-related questions in the student survey. Scores
range from 0 to 19. Children who attended outdoor education programs improved their science
scores significantly (p<.001) from the pre- to the first post-survey. The average immediate gain
was about 3 points, or 27 percent.

American Institutes for Research Page 29

Effects of Outdoor Education Programs for Children in California

Longer-term Impacts of Program Participation. Students completed the same science survey
six to ten weeks after they participated in the outdoor education program. Results are displayedin Table 21. Only children who completed all three student surveys were included for thisanalysis.15
Table 21. Immediate and longer-term gains (mean scores) in science knowledge for children who
attended outdoor education programs
Pre-survey

Mean
Std Dev

1st Post-Survey 2nd Post-survey

10.6
3.4

13.7
2.9

Gain: Pre- to 1st
Post-surveys

Gain: Pre- to 2nd
Post-surveys

Science Scores (N=115)
13.2
3.1!!!
3.2
3.6

2.6!!!
3.7

Statistically significant changes in scores are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.

As shown in Table 20, the gain from the pre- to the first post-survey was statistically significant.
Although the mean science score minimally reduced (-.05 points) from the first post-survey tothe second post-survey, the reduction was not statistically significant. In other words, students
maintained a significant gain in science scores from the pre-survey to the second post-survey, six
to ten weeks following program participation.
Science Score Subgroup Analyses
Analyses were conducted to determine if there were significant differences between science
scores across schools. In addition, differential effects for subgroups within the treatment groups(EL status, gender, and race) were explored. A table is included for any significant findings.
We examined differences in gain scores across schools and found no significant results.
Difference in gains among the four schools was not statistically significant (by Analysis of
Variance), meaning the degree of gain was about the same across all four schools. Amongstudents who attended outdoor school, there was no significant difference in any of the gainscores for science (short-term and long-term) between male and female students or EL students
and non-EL students.
Among students who attended outdoor school, Hispanic students increased their total science
scores significantly more from the pre- to the first post-survey than non-Hispanic students.
However, Hispanic student science scores decreased significantly more than non-Hispanic
student scores from the pre- to second post-survey.

15

In this case, the total sample size (N) is lower than what was used for the analyses of the immediate impacts ofprogram participation because data for 10 children were missing (e.g., the child moved, did not respond to thesurvey, etc.). Using the sample size of 115 children, rather than 125, also slightly impacted the post-survey mean
and the pre- to the first-post-survey gain in Table 15, compared to what is shown in Table 14.

American Institutes for Research Page 30

Effects of Outdoor Education Programs for Children in California

Table 22. Science scores, (mean scores), Hispanic versus Non-Hispanic students
Pre-survey

1st PostSurvey

2nd Postsurvey

Gain: Pre- to 1st
Post-surveys

Gain: Pre- to 2nd
Post-surveys

Science Scores
Hispanic (N=77)

10.5

14.1*

13.4

3.6* !!!

2.9 !!!

Non-Hispanic(N=35)

10.9

12.7

13.2

1.8 !!

2.3 !!

Statistically significant changes in scores, within group, are indicated by exclamation points: !p<.05; !!p<.01; !!!p<.001.
Statistically significant differences between Hispanic and Non-Hispanic gain scores are indicated by asterisks: *p<.05;**p<.01; ***p<.001.

Summary of Research Question #3 Findings. Children who attended outdoor education
programs improved their science scores significantly from the pre- to the first post-surveys. The
average immediate gain was about 3 points, or 27 percent. Although the mean science scoreminimally reduced (-.05 points) from the first post-survey to the second post-survey, thereduction was not statistically significant. In other words, students maintained a significant gainin science scores from the pre-survey to the second post-survey, six to ten weeks following
program participation.16

16

If Bonferroni adjustments are applied to the student science data, the significant differences between Hispanic andnon-Hispanic science scores become insignificant.

American Institutes for Research Page 31

Effects of Outdoor Education Programs for Children in California

American Institutes for Research Page 32

Effects of Outdoor Education Programs for Children in California

Chapter 4: Perceived Benefits of Outdoor Education: Feedback from
Sixth Grade Teachers and Outdoor Education Staff
This section summarizes responses from teachers and outdoor education staff gathered throughopen-ended survey items and interviews. It provides a context through which to interpret thesurvey findings presented in Chapter 3.
Elementary school teachers described the benefits of participation in the outdoor education
program for sixth-grade students, including the academic and social-emotional effects ofprogram participation and its impact on children with disabilities and special needs. Outdoor
school principals provided their perspectives regarding program characteristics which may
contribute to positive outcomes for students.
Overwhelmingly, teachers emphasized the positive impacts of outdoor science school for theirstudents. One sixth-grade teacher stated, “Outdoor school is priceless to a child’s education. It
supports our efforts to educate our students.” Teachers and outdoor school staff reported that the
program offered the first opportunity for many children to spend time in a natural environment.
A teacher commented, “Many children have never seen a setting like outdoor school. [They]never get away from the concrete.” Statements from other teachers echoed the same thought,“Most of my students have never been out of their neighborhood. Outdoor science school opens
them up to the world outside. I have seen many of these students get excited about learning(discovering) after they attend outdoor science school.” The following section summarizes
comments from in-person discussions with teachers and from the thirteen teachers who
responded to open-ended survey questions. Input gathered from three outdoor school principals
also is provided.
Academic Impacts
According to elementary school teachers, outdoor science school provides an in-depth week ofhands-on science instruction and a foundation to support subsequent classroom instruction.
Teachers described the program as an effective means to motivate students and help them makeconnections between classroom learning and the real world. Twelve of the thirteen teachers17
who provided responses on the teacher survey reported that the students learned science concepts
while at the program. Comments from teachers included the following:
•
•
•
•

17

“Students have a tangible science experience” and “explore nature and people in adifferent structural setting. It allows students to see education as something other thantexts and tests.”“It [outdoor school] provided language for my English Learners and reinforced
vocabulary for other students.”“Students are outdoors and doing physical as well as mental work. [The program is an]
opportunity for them to shine.”“Many students are auditory/verbal or kinesthetic learners. Many who do not do well in aclassroom setting, excel at outdoor science school. They are able to gain a wealth of
background knowledge we use for nearly every other academic area.”

One teacher reported that her students “did not learn enough science” during the one-week program.

American Institutes for Research Page 33

Effects of Outdoor Education Programs for Children in California

Teachers reported using outdoor science school to leverage classroom instruction. One teacher
described the program as a “jumping off point” for subsequent activities in the classroom,
connecting outdoor school with writing projects, social studies, and science lessons. For
example, children in her class write an adventure story, using their outdoor school experience as
background and context. Similarly, a lesson in ancient history might include the question, “Whatdo you think people need to live in this time?”, building from students’ understanding of what
animals need to survive in their environment. One teacher commented, “It [outdoor school]aligns with the curriculum. It is an integral part of the curriculum.”
Outdoor science school staff were asked to identify factors about the program which support
student learning. They emphasized: 1) the novelty of the environment for students (56 percent of
the students in the treatment group reported that outdoor school was the first time they spent time
outdoors in a natural setting), 2) the high-interest, hands-on, inquiry-based nature of thecurriculum, and 3) the wide range of opportunities students have for success at the program.
According to one outdoor science school principal, “kids are being confronted with sciencecurriculum that is hands-on and all around them and real. When we teach science in theclassroom, a lot of it is video oriented, text oriented, but the student is not immersed in it. Just by
virtue that they [the students] are living in the outdoors, which is the setting they are studying,
they are embedded in the curriculum itself.” Children spend a week learning to “observe, ask
questions, look for details, look for evidence that supports their thinking, organizing – these are
the fundamental scientific processes.”
Outdoor school also is less prescriptive than a typical school setting. Because there is “no clockon the wall,” outdoor teaching staff can extend activities based on children’s interest and needs.
While outdoor school teaching staff follow a schedule, it is somewhat less restrictive than that of
the regular school day (e.g., outdoor staff can extend a lesson for an extra ten minutes if children
are interested and, as one outdoor school principal described, “at that ‘ah-ha’ moment” oflearning).
Social-Emotional Impacts
In addition to academic benefits, elementary school teachers described the social and emotional
impacts of outdoor science school for their students. Overwhelmingly, teachers emphasized how
the program supports students’ feelings of self-esteem, fosters relationships among peers, andreduces behavior problems. Teachers also highlighted the benefits of the program for children
with disabilities or special needs and for at-risk students.
Self-esteem. Teachers and outdoor education staff described how participation in the program
helps to support students’ self-esteem, by bolstering their independence and confidence, and byproviding a wide range of opportunities for children to succeed. Outdoor school staff are trained
to recognize students for their positive participation in activities and their accomplishments(small and large) throughout the day. Students’ demonstration of respect toward one another,teaching staff, and the environment is also rewarded. One outdoor school principal stated, “Theenvironment is set up for children to succeed and feel good about what they can do. The smaller
teacher-student ratio definitely helps, children pick up on that and think ‘hey this is anenvironment where everyone gets noticed, and I can shine here.’ When [students] do somethingright, they get recognized. One success builds on another...answering questions or making

American Institutes for Research Page 34

Effects of Outdoor Education Programs for Children in California

observations, exhibiting some good table manners at meal time, [and] helping their cabin matesscore points.”
Another outdoor school principal described how the program supports students’ self-esteem andgeneral social growth. The outdoor school uses an “ICARE” motto (Integrity, Cooperation,
Attitude, Respect, and Esteem) throughout all aspects of the program. Outdoor school staff talkabout these concepts every morning with the students and “ICARE” posters are posted
throughout the outdoor school site. If a child has a behavior problem, staff use the “ICARE”
concepts as a context to talk through the problem with the student.
Many of the teachers commented on the confidence they observed among their students. One
teacher, referring to one of her students, stated, “[At the program] is the first time I saw him
smile or raise his hand.” The program provides “countless enrichment activities, and theopportunity to be independent, explore, and build self-esteem,” stated another teacher. “Students
find success possible at science school. They realize that they can be successful at ‘smart’ things
so when they get back to school, their self-esteem is greater. They can handle things better.”Outdoor staff and elementary school teachers also described how student labels and stereotypes
fade at the program. Each child is new to the outdoor school teaching staff; thus the staff do notknow which student is regularly disciplined for behavior problems in the classroom or whichchild has an Individualized Education Program (IEP). Outdoor school staff and teachers
repeatedly used the phrase “fresh start” to describe children’s experience at outdoor school andupon their return to the classroom. “Children feel better about themselves because they know
their teacher saw them at outdoor school doing well,” said one outdoor science school principal.
Relationships among Peers. Students interact with one another throughout each day of the
program, living, eating, socializing, and learning together. They are deliberately combined withstudents from other schools in a variety of small groups. In this manner, as one teacher
characterized, “Pre-conceived notions [of children] are left behind – other children don’t knoweach other.” Other teachers talked about children who have difficulty creating positive peer
relationships within the regular school setting. “It’s like getting a second chance in making
friends. A fresh start. [Students] build interdependence....with classmates they seldom interact
with [in school].” For some children, “the first friend they ever make is at outdoor school.”
Several teachers commented on the importance of exposing their students to other children fromvarying backgrounds. Children “have to get along with other children. Not just other children in
their class but other children in their school and children from other schools. Sometimes thatmeans children of different [socioeconomic backgrounds]”. A teacher reported, “Students engage
with other students from diverse communities. This enables them to be exposed to a variety ofsocial settings. For some, it enables them to experience different emotions.”
Behavior and Discipline. Teachers identified several characteristics of the outdoor school that
may impact students’ behavior. Respect for staff and other children is a priority of the program,and this message is articulated to students throughout the week. The three outdoor scienceschools in this study maintained a higher staff to child ratio than a typical sixth-grade classroom.
Program staff can interact with more children on an individual basis, and the smaller group sizes

American Institutes for Research Page 35

Effects of Outdoor Education Programs for Children in California

allow for close supervision. On average, learning groups at outdoor science school are composedof 15 to 17 children, led by an outdoor education teacher and assisted by a “cabin leader” (ahigh-school counselor who attended the program as a fifth- or sixth-grade student). In addition,
an elementary school teacher often accompanies the group on their lessons.
The outdoor school schedule also helps to reduce behavior problems among students. According
to teachers, many children lack a regular schedule at home (e.g., homework, bed times, chores).
The outdoor science schools maintain specific standards of behavior that are expected fromchildren (e.g., a “3 strikes” policy before parents are called and children have to leave), as well
as a variety of chores that each child is expected to complete (e.g., cabin clean-up, helping withmeals). One teacher stated, “The activities at outdoor school reinforced respect andresponsibility.”Children with Disabilities and Special Needs and At-Risk Students. According to several
teachers, the program puts children with disabilities and special needs on equal footing with theirpeers, leaving behind the classifications that can emerge within the classroom setting. During a
focus group, one teacher challenged others to try to identify the special education students fromother students, pointing out that the program works as an “equalizer.” Teachers were asked to
identify particular benefits of outdoor science school for at-risk children. The outdoor school
provides “structure, discipline, motivation, and determination,” and is a “positive experience forstudents who are at different levels of education.” Echoing the comments regarding children withdisabilities or special needs, one teacher reported that at-risk students are “not labeled or grouped
the way they are in the regular school setting. They are able to shine.”
In sum, elementary school teachers and outdoor school staff identified a range of positive
outcomes they observed among students who attended the program. Many of these outcomes
were measured as part of this study (e.g., relationship with peers, self-esteem) and the results arepresented for the treatment and control groups in other sections of this report. Other outcomes,
such as the specific impact of the program for children with disabilities or special needs, orconcepts such as the “fresh start” afforded by outdoor school, were not directly measured, but
program and elementary school staff alike repeatedly referred to these as important additional
benefits for students. The overwhelming support by elementary school teachers for the program
also corroborates the findings from this study that outdoor education programs are effective in
multiple ways – for teaching science concepts, supporting positive social-emotional
development, and motivating students to think and act more responsibly as stewards of theenvironment.

American Institutes for Research Page 36

Effects of Outdoor Education Programs for Children in California

Chapter 5: Conclusions
This chapter summarizes the major research findings of the study.
Personal and Social Skills•••

According to student assessments gathered immediately after program participation,
children who attended outdoor science school showed significant positive gains inconflict resolution. However, the difference between the treatment and control groups on
conflict resolution was not statistically significant at that point in time. Six to ten weeks
later, children who attended the program showed gains in cooperation and conflict
resolution that were significantly higher than the control group.
Teacher ratings provide evidence of a wide range of positive outcomes related toparticipation in outdoor science school. Teachers rated all children before the treatmentgroup attended outdoor school and six to ten weeks later. According to teacher ratings ofeach student, those children who attended outdoor science school showed statisticallysignificant positive gains on all eight constructs on which they were rated. In contrast,
the control group showed losses on seven of the eight constructs. Children who attended
outdoor science school showed significantly larger gains than the control group in six ofthe eight constructs. These gains were observed in self-esteem, conflict resolution,
relationship with peers, problem solving, motivation to learn, and behavior in class.
Parent ratings of their children did not reveal any significant differences in the fivesocial-emotional constructs between children who attended outdoor school and those
who did not.

Stewardship of the Environment•••

According to student assessments gathered immediately after program participation,
children who attended outdoor school showed significant increases in one of the threeconstructs: concern about conservation. However, these increases were not significantly
larger than those of the control group.
At the six- to ten-week point, the control group showed significant losses in two of thethree constructs (attitude toward science and environmental behaviors), whereas thetreatment group did not show any significant losses.
According to parent reports, students who participated in the program had significantly
larger gains in environmental behaviors (p<.05), compared to children who did notattend the program. In other words, parents of children who attended outdoor school
observed children engaging in positive environmental behaviors (e.g., recycling, etc.) at
home, whereas a statistically significant finding was not observed for ratings by parentsof children in the control group.

American Institutes for Research Page 37

Effects of Outdoor Education Programs for Children in California

Knowledge and Understanding of Science Concepts••

Children who attended outdoor school significantly raised their science scores by 3
points (27 percent), as measured by a pre- and post-survey administered immediately
upon their return to school.
The increase in science knowledge was maintained six to ten weeks following program
participation, with no significant loss in science scores.

Benefits for English Language Learners•

This study focused on 255 sixth-grade students, 58 percent of which were identified byteachers as English Learner (EL) students. According to teacher reports, among thosestudents who attended the program, EL students demonstrated gains in cooperation,
leadership, relationship with peers, and motivation to learn that were significantly larger
than the gains shown by non-EL students for those constructs.

Other Perceived Impacts of the Program
Elementary school teachers and outdoor science school staff overwhelmingly emphasized thepositive outcomes they observed among children who attended the program. These included
increased confidence and self-esteem, positive relationships among students, and reduced
discipline and behavior problems. Teachers reported that the program provided effective, handson science instruction which served as a foundation for subsequent classroom instruction.
Among other benefits, elementary school teachers and outdoor school staff repeatedlyemphasized how outdoor science school provides a “fresh start” for students. While this study
did not collect quantitative measures of this concept, feedback from teachers and staff suggest
the program offers effective strategies for supporting children who may struggle with labels or
stereotypes that might emerge within a regular school setting (e.g., children with special needs,
children with classroom behavior problems, etc.). The program provided an “opportunity toshine” for all students.
Students who attended outdoor school were asked if the experience changed them. Of the eightythree students who provided a response, sixty-seven percent (56 students) reported a positivechange in terms of science content learned, new friends, and appreciation for or stewardship ofthe environment. Typical responses from students included the following:
•
•
•
•
•
•

“Yes, because I learned more. I like science a lot because it helped me to protect theenvironment even more.”“I do more work on my own!”“Because I have a lot of friends now in other schools and I learned about nature more.”“Yes, I recycle more because I realized how important it is. I also don't litter.”“Yes, I try not to waste so much water and I try to recycle all the recyclable things. I alsocare more about nature than I did before.”“I've changed by learning that soils have worms and plants give us oxygen. I've learned a
lot at outdoor science school. I had fun and met new friends and I am kind of gettingused to science.”

American Institutes for Research Page 38

Effects of Outdoor Education Programs for Children in California•“I think I have changed after going to outdoor school. I felt less bored, learned a lot
about science, and made more friends.”

In sum, teachers, outdoor education staff, and participating students characterized outdoor
science school as a positive social-emotional and learning experience, with impacts observed
during the program week and subsequently in the classroom setting.
Recommendations•

The CDE, outdoor science schools, and elementary schools may wish to explore
additional strategies to bolster the positive outcomes associated with participationin the program. Given the relatively short length of the program (five days), schools
and families play an important role in reinforcing outdoor science school instruction
once children return to school and their homes. Research has clearly demonstrated theimportance of family-school connections in supporting student learning. To extend thebenefits of outdoor science school, public schools might consider developing or building
on existing strategies to integrate outdoor school curricula and teaching approaches into
their classroom instruction. Family members also play a critical part in reinforcing thelessons learned at outdoor school at home (e.g., conserving water, recycling, etc.).
Strategies to create strong connections between families and outdoor school (e.g., family“environmental stewardship” action plans, parent handouts describing activities forfamilies and children to do together at home, etc.) could help to bolster and sustain theimpact of the five-day program.

•This study suggests promising areas for future research regarding the benefits ofoutdoor learning experiences for at-risk students. This study explored a wide range ofpossible benefits associated with participation in environmental education programs.
Future, more targeted research that builds on the promising benefits found in this study,
such as in the area of problem-solving skills, may yield more detailed information about
the ways in which the specific outcomes associated with the program are best achieved.
Other recommendations for future research include a longitudinal study, to examine thelong-term impacts of participation in outdoor school for students. While the control
group for this study will attend outdoor school in the winter or spring of the 2004-2005
school year (after this study is completed), possible longitudinal research strategies
include tracking the sample of students over time (identifying a matched sample ofchildren who never attended outdoor science school as a comparison group), or costeffectiveness studies that compare varying approaches to the provision of outdoor school
experiences over time. Indeed, findings from this study suggest that outdoor education
programs may contribute to academic learning and social development for at-risk
students in ways that cannot be duplicated within the classroom environment. The
preliminary positive results from this study, along with the potential for long-term
benefits for students (i.e., improved academic scores in science and related subjects,
increased motivation to pursue science courses and/or careers, enhanced appreciation for
the environment, etc.) make a strong case for continued support, possible expansion, andfurther study of outdoor education programs for at-risk students.

American Institutes for Research Page 39

Effects of Outdoor Education Programs for Children in California

American Institutes for Research Page 40

Effects of Outdoor Education Programs for Children in California

References
Johnson, P., & Ivie, P. (2003). Guide for Self-Appraisal and Certification of Resident Outdoor
Science School Programs. (4th ed.). Costa Mesa, CA: Orange County Department of
Education, California Outdoor School Administrators.
Lieberman, G.A., & Hoody, L.L (1998). Closing the Achievement Gap: Using the Environment
as an Integrating Context for Learning. San Diego, CA: State Education andEnvironment Roundtable. http://www.seer.org/extras/execsum.pdf
Lieberman, G.A., Hoody, L.L., & Lieberman, G.M. (2000). California student assessment
project - The effects of environment-based education on student achievement. San Diego,
CA: State Education and Environment Roundtable,
Neill, J. T., Marsh, H. W., & Richards, G. E. (2003). The Life Effectiveness Questionnaire
Development and Psychometrics. Sydney University of Western Sydney.
North American Association for Environmental Education, & National Environmental Education& Training Foundation. (2001). Using environment-based education to advance learning
skills and character development. Washington, DC: Author.
Perneger, T.V. (1998, April). What’s wrong with Bonferroni adjustments? British Medical
Journal, 316, 1236-1238.
Rickinson, M., Dillon, J., Teamey, K., Morris, M., Choi M.Y., Sanders, D., & Benefield, P.
(2004, March). A Review of Research on Outdoor Learning. London: National
Foundation for Educational Research and King’s College London.
United Way of America. (1996). Measuring program outcomes: A practical approach.
Alexandria, VA: Author.

American Institutes for Research Page 41

Appendix A:
Correlation of the Sixth-Grade Science Content Standards
for California Public Schools with Curriculum at Certified
Outdoor Science Schools

A-1

A-2

Correlation of the Sixth-Grade Science Content Standards for California Public Schools
with Curriculum at Certified Outdoor Science Schools
Source: ROSS Certification Guide, California Outdoor Science School Administrators1
Students that attend ROSS certified outdoor science schools will gain knowledge, skills andexperiences in many curricular areas, and in particular science. Learning experiences andcurriculum vary from outdoor school to outdoor school. The following are examples of learning
opportunities that relate to the California State Science Standards.
At the Fall 2001 COSA Meeting, members voted to adopt this document to be used during ROSS
certification to ensure that at least 9 of the 14 standards are being addressed during a 5-day
program, and at least 7 of the 14 standards are being addressing during a 4-day program.
Investigation and Experimentation
1. Investigation and experimentation is a focus of state science standards at all grade levels.
Scientific progress is made by asking meaningful questions and conducting careful
investigations.
•
•
•
•
•
•
•
•

classify objects
measure and use appropriate tools
observe and identify change
develop a prediction or hypothesis
perform investigations
record data
formulate conclusions
communicate results of investigation

Life Sciences
2. Organisms in ecosystems exchange energy and nutrients among themselves and with theenvironment. (grade 6, #5). Examples of outdoor education activities include:• Students observe and identify (and in some cases, hold and/or touch) organisms in
the field and discuss the roles of these organisms in the ecosystem.
• Students describe (role play, and sing about) how energy entering ecosystems as
sunlight is transformed by producers into chemical energy throughphotosynthesis, and then from organism to organism in food webs.
• Students communicate how populations of organisms can be categorized by thefunctions they serve in an ecosystem.
• Students see first hand how the number and type of organisms an ecosystem cansupport depends on the resources available and abiotic factors such as quantity oflight and water, range of temperature, and soil composition.
1

The original document included in the ROSS Guide included Science Content Standards for Grades 4 – 7. This
document only includes excerpts for Grade 6.

A-3

Investigation and Experimentation
3. Investigation and experimentation is a focus of state science standards at all grade levels.
Scientific progress is made by asking meaningful questions and conducting careful
investigations.
•
•
•
•
•
•
•
•

classify objects
measure and use appropriate tools
observe and identify change
develop a prediction or hypothesis
perform investigations
record data
formulate conclusions
communicate results of investigation

Life Sciences
4. Organisms in ecosystems exchange energy and nutrients among themselves and with theenvironment. (grade 6, #5). Examples of outdoor education activities include:• Students observe and identify (and in some cases, hold and/or touch) organisms in
the field and discuss the roles of these organisms in the ecosystem.
• Students describe (role play, and sing about) how energy entering ecosystems as
sunlight is transformed by producers into chemical energy throughphotosynthesis, and then from organism to organism in food webs.
• Students communicate how populations of organisms can be categorized by thefunctions they serve in an ecosystem.
• Students see first hand how the number and type of organisms an ecosystem cansupport depends on the resources available and abiotic factors such as quantity oflight and water, range of temperature, and soil composition.
Earth Sciences
12. Plate tectonics explains important features of the Earth’s surface and major geologic events(grade 6, #1)• Students observe and hike mountains, or other natural areas, which are products of plate
tectonics and mountain building.
• Students observe past fault and/or volcanic activity (where possible), participate ingeology-based games and activities and discuss plate tectonics.
13. Topography is reshaped by weathering of rock and soil and by the transportation anddeposition of sediment (grade 6, #2)• Students see and feel rocks and geologic formations shaped by streams, rivers, waves,wind, etc.
• Students see erosion that has taken place due to weathering (rain storms, wind, etc.) or
utilize a model or activity to demonstrate this process.

A-4

14. Sources of energy and materials differ in amounts, distribution, usefulness, and the timerequired for their formation. (grade 6, #6)
•
•
•
•
•

Students are involved in activities such as separating recyclable items from trash and will
discuss the energy and resources used to make new items versus reusing and recycling
old ones.
Students learn about nonrenewable versus renewable resources through discussions andactivities.
Students participate in resource conservation activities (turning off lights, heaters, closing
doors, saving water, reducing food waste, etc.).
Students participate in activities that illustrate how habitat loss is driven by people’s
needs for nonrenewable and renewable resources.
Through activities and discussions, students will understand how human activity affects
resources.

A-5

Appendix B:

Data Collection Tools• Teacher Survey• Student Survey• Parent Survey

B-1

B-2

Teacher Survey
Please write your name and school _____________________________________________________________________________The purpose of this survey is to gather information on students in six different areas:
•
•
•
•
•
•
•
•

self esteem,
problem solving,
conflict resolution,
cooperation and teamwork,
leadership,
relationships with peers,
motivation to learn, andbehavior in class

For each student listed in the table, please read each sentence and indicate how much you disagree or agree, using a scale from 0 to 10. A “0” means
you strongly disagree and a “10” means you strongly agree. A “5” means you don’t feel very strongly one way or the other.
Rate students based on your observations of them at this point in time. Do not discuss this survey with other staff or students. Please write yourratings in the boxes under each student’s name. Do not provide ratings for students whose names do not appear in the table - we are only collecting
information on students for whom we have both student and parental consent.

Strongly
Disagree0

Neutral

1

2

3

4

5

Strongly
Agree
6

7

8

9

10

Please complete this survey the week of XXXXXXX.
It is very important that teachers assess each student during this time period. Thank you for your help! If you have any questions, please call
Gabriele Phillips at 650-843-8144, or email at gphillips@air.org.
Return this survey with the student and parent surveys in a pre-paid fed-ex label provided to the principal.

1Strongly
Disagree0

Neutral
1

2

3

4

5

6

7

8

9

Strongly
Agree
10

ExampleMaria Stevens

One student per column. Please continue on thefollowing pages.

1 Self-Esteem – This student exhibits a high level of selfesteem.
2 Relationships with Peers – This student gets along wellwith others in class.
3 Effective Problem Solving – This student uses
appropriate and effective problem-solving techniques.

648

4 Conflict Resolution – This student effectively heads off
and resolves interpersonal and group conflicts.

4

5 Cooperation and Teamwork – This student cooperates
well with his or her classmates.

4

6 Effective Leadership – This student shows strong
leadership skills.

7

7 Motivation to Learn – This student is motivated tolearn.

6

8 Behavior – This student is generally well-behaved inclass.

82

Student Survey
Print your name: ___________ _________________________________________
This survey is part of a study about outdoor science school. Your answers will NOT affect your school grades in any way.
The survey begins on the next page. Read each sentence. Then think about how much you agree or disagree with each sentence. Circle anumber, from zero to ten. “Zero” means you strongly disagree with the sentence. “Ten” means you strongly agree with the sentence. “5”means you do not feel strongly one way or the other – you are neutral about the sentence.
Your teacher will talk to you about these examples.
Example 1.
Strongly
DisagreeMy dog is very playful.

0

Neutral

1

2

3

4

5

Strongly
Agree

6

7

8

9

10

John marked “8” because his dog is a puppy and likes to chase balls. But his puppy sleeps a lot, so it is not always playful.
Example 2.
Strongly
Disagree

I like spaghetti.

0

Neutral

1

2

3

4

5

Strongly
Agree

6

7

8

9

10

Michael circled “5” because when his mom cooks spaghetti for dinner, he does not really care. Spaghetti is just okay, he doesn’t really likeit or dislike it.
Practice. Read the sentence. Circle a number from 0 to 10. Have a group discussion about your answer.
Strongly
Disagree

I like to play sports.

0

Neutral

1

2

3

4

5

Strongly
Agree

6

7

8

9

10

1

Strongly
Disagree

NeutralStrongly
Agree

1.

I feel good about myself.

0

1

2

3

4

5

6

7

8

9

10

2.

I am good at cooperating with my classmates.

0

1

2

3

4

5

6

7

8

9

10

3.

I have a lot to be proud of.

0

1

2

3

4

5

6

7

8

9

10

4.

I am a good leader.

0

1

2

3

4

5

6

7

8

9

10

5.

I am good at figuring out how to solve my disagreementswith other people.

0

1

2

3

4

5

6

7

8

9

10

6.

I feel comfortable talking to my teacher.

0

1

2

3

4

5

6

7

8

9

10

7.

Most things I do turn out well.

0

1

2

3

4

5

6

7

8

9

10

8.

I like it when someone else is the leader and I am one of
the followers.

0

1

2

3

4

5

6

7

8

9

10

9.

I like science.

0

1

2

3

4

5

6

7

8

9

10

10. I have confidence in myself.

0

1

2

3

4

5

6

7

8

9

10

11. I solve problems with my friends by talking things out
with them.

0

1

2

3

4

5

6

7

8

9

10

12. I like cooperating with others.

0

1

2

3

4

5

6

7

8

9

10

13. I am happy with the way I can do most things.

0

1

2

3

4

5

6

7

8

9

10

14. I try to avoid unnecessary arguments with other people.

0

1

2

3

4

5

6

7

8

9

10

15. I get along well with my teacher.

0

1

2

3

4

5

6

7

8

9

10

16. When I grow up, I want to be a scientist.

0

1

2

3

4

5

6

7

8

9

10

17. It worries me to think how much energy is wasted in theworld.

0

1

2

3

4

5

6

7

8

9

10

18. It worries me when I see people use too much water.

0

1

2

3

4

5

6

7

8

9

10

19. It makes me happy when people recycle used bottles,
cans, and paper.

0

1

2

3

4

5

6

7

8

9

10

2

Read each question and answer by circling “Yes” or “No”.
20. To save energy, do you turn off lights at home when you don’t need to use them?

Yes

No

21. Does it upset you to think that many people don’t care about the environment?

Yes

No

22. To save water, do you try to use as little water as possible when you take a shower or bath?

Yes

No

23. Would you be willing to ride the bus, light rail, or trolley to more places to cut down on air
pollution?

Yes

No

24. Do you get upset when you think of the things people throw away that could be recycled?

Yes

No

25. Have you talked with your parents about problems with the environment?

Yes

No

26. Do you read stories that are about the environment?

Yes

No

27. Do you leave the refrigerator door open while you decide what food to take out of it?

Yes

No

28. Do you get angry about the damage that pollution does to the environment?

Yes

No

29. Have you asked your family to recycle things you use (like bottles, cans, or paper)?

Yes

No

30. Have you ever asked others what you can do to help reduce pollution?

Yes

No

31. Do you worry about problems with the environment?

Yes

No

32. Would you ask your friends to recycle things like bottles, cans, or paper?

Yes

No

33. To save water, do you turn off water in the sink while you brush your teeth?

Yes

No

34. Do you separate things at home for recycling?

Yes

No

35. Would you be willing to write letters asking people to help cut down pollution?

Yes

No
3

Read each question and answer it the best you can. If you don’t know the answer to a question, leave it blank.
1. Circle the 7 natural resources that support life on earth.
Technology

Water

Sun

HousesSoil

Animals/Wildlife

Gravity

Minerals

ErosionEvaporation

Money

Cars

Plants/Forests

Air

2. Name three cycles that occur in nature.

n ____________________________
o ____________________________
p____________________________

3. What is a producer? Choose one.
A. An organism that eats only live plants
B. An organism that eats only dead things
C. An organism that decomposes things
D. An organism that makes its own food
4. Circle the word below that best fits the following definition: “A change that living things go through so they fit better with theirenvironment.”
Erosion

Adaptation

Condensation

Examination
45. What role does sunlight play in photosynthesis? Choose one.
A. It makes the plant warm.
B. It provides energy
C. It provides oxygen
D. It makes chlorophyll
6. What gas do plants give off during the process of photosynthesis that is used by animals? Choose one.
A. Oxygen
B. WaterC. Carbon monoxide
D. Sugar
7. How do cars and other vehicles affect the environment? Choose one.
A. Most produce air pollution.
B. Most create new homes for wildlife.
C. Most use renewable energy sources.
D. Most are beautiful and fun to drive.
8. What can people do to reduce the amount of air pollution? Choose one.
A. Drive more.
B. Never carpool with others.
C. Never use public transportation.
D. Walk or ride a bike to school or work.
9. What is NOT an action that people do to act responsibly toward the environment? Choose one.
A. Conserve energy
B. Prevent pollution
C. Recycle waste
D. Drop litter on the ground5

10.

11. We all have a responsibility to take care of the Earth. List two new ideas about how you can care for the Earth.

n__________________________________________________________________________
o___________________________________________________________________________

6

Now tell us about you…
About how many kids in your class are your friends? ____________________________________
About how many kids in your class are your good friends? _______________________________Are you a boy or a girl?

{ Boy{ GirlWhich of the following best describes you? You make pick more than one.

{ Hispanic or Latino{ Black or African American{ White{ Asian{ Native Hawaiian or Other Pacific Islander{ American Indian or Alaska Native
Thank You!

7

ID Label
Parent SurveyYour child’s elementary school is participating in a study about outdoor schools in California. The study is called the Outdoor Science
School Programs Evaluation. This survey is an opportunity for you to tell us what your child thinks about him or herself and some of thethings he or she does at home. Everything is confidential. Your name and your child’s name will never be used. The information from thissurvey will help us determine how outdoor school helps sixth grade students. Please fill out the survey this week and give it to your child toreturn to his or her teacher.
Directions: The survey begins on the next page. Read each sentence. Then think about how much you agree or disagree with eachsentence. Circle a number, from zero to ten. “Zero” means you strongly disagree with the sentence. “Ten” means you strongly agree withthe sentence. “5” (neutral) means you do not feel strongly one way or the other. Think about your child who is enrolled in sixth grade
when answering the questions.
Examples:Strongly
DisagreeMy child enjoys school0

Neutral

1

2

3

4

5

Strongly
Agree

6

7

8

9

10

Mary circled “9” because her daughter enjoys school a lot.
Strongly
DisagreeMy child likes spaghetti.

0

Neutral

1

2

3

4

5

Strongly
Agree

6

7

8

9

10

Teresa circled “5” because when she cooks spaghetti for dinner, her son Michael does not really care. Michael thinks spaghetti is just okay,
he doesn’t really like it or dislike it.

Please fill out the survey this week. Then give it to your child to bring back to his or her teacher.
If you have any questions, please call Gabriele Phillips at 650-843-8144 or email at gphillips@air.org. Thank you!

1

What is your name? ________________________________________________________________________________________
What is the name of your sixth-grade child? ___________________________________________________________________
Strongly
Disagree

NeutralStrongly
Agree

1.

My child feels good about him/herself.

0

1

2

3

4

5

6

7

8

9

10

2.

My child is good at cooperating with his/herclassmates.

0

1

2

3

4

5

6

7

8

9

10

3.

My child feels he/she has a lot to be proud
of.

0

1

2

3

4

5

6

7

8

9

10

4.

My child is a good leader.

0

1

2

3

4

5

6

7

8

9

10

5.

My child is good at figuring out how tosolve his/her disagreements with otherpeople.

0

1

2

3

4

5

6

7

8

9

10

6.

My child feels comfortable talking to his/herteacher.

0

1

2

3

4

5

6

7

8

9

10

7.

My child feels most things he/she does turnout well.

0

1

2

3

4

5

6

7

8

9

10

8.

My child likes to be the leader rather than afollower.

0

1

2

3

4

5

6

7

8

9

10

9.

My child likes science.

0

1

2

3

4

5

6

7

8

9

10

2

Strongly
Disagree

NeutralStrongly
Agree

10. My child has confidence in his/herself.

0

1

2

3

4

5

6

7

8

9

10

11. My child solves problems with his/her friends
by talking things out with them.

0

1

2

3

4

5

6

7

8

9

10

12. My child likes cooperating with others.

0

1

2

3

4

5

6

7

8

9

10

13. My child is happy with the way he/she can domost things.

0

1

2

3

4

5

6

7

8

9

10

14. My child tries to avoid unnecessary
arguments with other people.

0

1

2

3

4

5

6

7

8

9

10

15. My child gets along well with his/her teacher.

0

1

2

3

4

5

6

7

8

9

10

16. When my child grows up, he/she wants to bea scientist.

0

1

2

3

4

5

6

7

8

9

10

17. My child worries about how much energy iswasted in the world.

0

1

2

3

4

5

6

7

8

9

10

18. My child worries when he/she sees people
use too much water.

0

1

2

3

4

5

6

7

8

9

10

19. It makes my child happy when people recycle
used bottles, cans, and paper.

0

1

2

3

4

5

6

7

8

9

10

3

Read each question and answer by circling “Yes” or “No”
20. To save energy, does your child turn off lights at home when he/she does not need to use them?

Yes

No

21. Does it upset your child to think that many people don’t care about the environment?

Yes

No

22. To save water, does your child try to use as little water as possible when he/she takes a shower or bath?

Yes

No

23. Would your child be willing to ride the bus, light rail, or trolley to more places to cut down on air pollution?

Yes

No

24. Does your child get upset when he/she thinks of the things people throw away that could be recycled?

Yes

No

25. Has your child talked with you about problems with the environment?

Yes

No

26. Does your child read stories that are about the environment?

Yes

No

27. Does your child leave the refrigerator door open while he/she decides what food to take out of it?

Yes

No

28. Does your child get angry about the damage that pollution does to the environment?

Yes

No

29. Has your child asked your family to recycle things you use (like bottles, cans, or paper)?

Yes

No

30. Has your child ever asked others what he/she can do to help reduce pollution?

Yes

No

31. Does your child worry about problems with the environment?

Yes

No

32. Would your child ask his/her friends to recycle things like bottles, cans, or paper?

Yes

No

33. To save water, does your child turn off water in the sink while brushing his/her teeth?

Yes

No

34. Does your child separate things at home for recycling?

Yes

No

35. Would your child be willing to write letters asking people to help cut down on pollution?

Yes

No

Please give this survey to your child to return to his or her teacher. Thank you!

4Appendix C
Exploratory Analysis

C-1

C-2

We measured the internal consistencies of the five social and personal and the threeenvironmental attitudes scales by Cronbach’s alpha. Items included in the social and personalscales and two of the environmental scales were measured using an 11-point scale (0 Strongly
Disagree – 10 Strongly Agree). The numbers of individual survey items included in these scales
1
are: self-esteem = 5, cooperation = 2, leadership = 1 , conflict resolution = 3, relationship withteacher = 2, attitude toward science = 2, and concern about conservation = 3. The environmental
behavior scale was composed of 16 dichotomous (yes/no) questions by counting the number of“yes” responses. In order to adjust the different numbers of items included in these scales, wereport Standardized Cronbach’s alpha.
The “attitude toward science” construct had a relatively low alpha coefficient (.519) in the firstround of survey data, however, this same construct shows higher coefficients in the two
subsequent data collections rounds. All the other constructs showed an acceptable level ofreliability.
Student Surveys
Standardized Cronbach's Alpha()
Constructs
Round 1 Round 2 Round 3
Self-Esteem
0.754
0.838
0.864
Cooperation
0.637
0.787
0.756
Leadership------Conflict Resolution
0.682
0.736
0.809
Relationship with Teacher
0.717
0.790
0.805
Attitude Toward Science
0.519
0.691
0.667
Concern About Conservation
0.767
0.861
0.923
Environmental Behaviors
0.637
0.778
0.854

Parent Surveys

Constructs
Self-Esteem
Cooperation
Leadership
Conflict Resolution
Relationship with Teacher
Attitude Toward Science
Concern About Conservation
Environmental Behaviors

Standardized
Cronbach's
Alpha ()
Round
Round
1
2
0.865
0.898
0.746
0.818
0.754
0.735
0.790
0.730
0.732
0.830
0.592
0.725
0.889
0.908
0.880
0.890

1

The leadership scale originally included two items; however, one of the items used a reverse question, whichcaused inconsistency with the other item. Thus, we decided to drop the reverse question item and the leadershipscale subsequently consisted of one item.

C-3