Ready for Lift-Off, Orange County?:
How a National-Local Partnership Keeps STEM Programming
Innovative, Relevant, and Engaging
August 8, 2023
Abby Bergman and Dr. Dawn McDaniel

SummaryThis brief report examines how a national-local partnership in Orange County, California was leveraged toadapt and deliver new, high-quality science, technology, engineering, and mathematics (STEM) learning
experiences. As part of the Imagine Science network of communities, Imagine Science Orange County(ISOC) is an example of the power of youth-serving organizations across the community coming together tocreate learning experiences for youth historically underrepresented in STEM. Through stakeholderinterviews, a document review, and examining self-report data from over 600 youth, we explore ISOC’s
innovative approach to partnership, curriculum adaptation, and continuous quality improvement. Data from2019 to 2022 shows ISOC programs increased youth STEM career interest, STEM engagement, and STEM
identity, and that these increases were statistically significant. Finally, we summarize the key findings, lessons
learned, and future plans for ISOC.
Out-of-school time (OST) settings often feel distinct from classrooms. They have the potential to offer youth
learning experiences that are more relevant and rewarding than those offered in most school settings. OST settings
are also learning environments where there is more time for youth to build relationships with adults (Hein, 2009;
Noam & Shah, 2014; Yohalem & Wilson-Ahlstrom, 2010). Research has shown that OST settings enhance youth’s
in-school academic achievement and improve their overall wellbeing (Kuperminc et al., 2019). For youth from
historically underserved communities, these types of learning environments are especially critical for youth’s
engagement and success in science, technology, engineering, and math (STEM). For example, the relationships
youth develop in OST settings are critical to youth establishing a clear and positive STEM identity and having
hopeful expectations for what they can achieve in STEM in the future (Catalano et al., 2004).
OST STEM programs are often facilitated by adults who come from a variety of backgrounds, and with a range ofexperiences: “their familiarity with teaching, classroom management–-and, most importantly, the [content]–-varies”(Cooper, 2013). Further, when it comes to providing high-quality STEM learning opportunities, many OST staff
likely have “no positive STEM experiences on which to draw” (Groome & Rodríguez, 2014). This lack of
exposure to STEM limits OST staff’s ability to lead high-quality STEM programming. Hill (2012) summarizes adilemma common for OST staff: “while school teachers are increasingly called upon to become more proficient insubject matter, we expect OST staff to improve student outcomes…without adequate subject matter training.”
Therefore, professional development for OST staff has become, “the missing link to promoting STEM topics"(Coalition for Science After School, 2008). Program content, if delivered effectively, drives youth engagement in away such that “OST staff members can do more with their background knowledge in youth development when
they pair it with content-specific expertise” (Cooper, 2013).
Indeed, OST staff often deliver STEM program content in more flexible or localized ways, than would be seen in aclassroom. Fitzsimons and colleagues (2020) provide guidance on best practices for this type of curriculum
adaptation, which often accounts for youth’s context and community. In describing their “framework approach,”these authors share how curricular content can be adapted so that the learning experience becomes more relevant,meaningful and responsive to, as well as respectful of, the learner (Fitzsimons et al., 2020). These types ofcurricular adaptations are likely essential to encouraging engagement in youth historically underrepresented in
STEM.
In this report, we explore how a national-local partnership developed through the Imagine Science network (seeNote 1) supported the Orange County community in adapting and delivering new, high-quality STEM learning
experiences for youth historically underrepresented in STEM. Through stakeholder interviews, a document
review, and examining youth data, we explore this community’s innovative approach to partnership, curriculum
adaptation, and continuous quality improvement. Then, we summarize the key findings, lessons learned, bestpractices, and future plans for Orange County.

Note 1. Imagine Science
Imagine Science is a network linking youth-serving organizations in communities across the country to offer “anunprecedented partnership and new way of doing business for its four governing organizations” (Clark et al.,2017). Combined, the four national youth-serving organizations that comprise Imagine Science (Girls Inc., Boys
& Girls Clubs of America, YMCA, and the National 4-H Council) have the capacity to reach 18 million youth,
many of whom are historically underrepresented in STEM. In addition to this large reach, the Imagine Science
network aims to offer innovation in its science, technology, engineering, and mathematics (STEM) programming,enhance the existing capacity and strategies for organizations to connect with underserved youth, and promote thedelivery of engaging high-quality STEM experiences.
Imagine Science’s national-local partnerships allow for combining resources (such as, curricula, staff expertise)
across these large youth-serving organizations and communities. Through national trainings and resources
sharing, all communities adopt a similar continuous quality improvement process and common program measures.
For example, all communities collect data using PEAR’s Common Instrument Suite – Student Survey measuring
STEM engagement, STEM career interest, and STEM identity. This has allowed for the creation of a sharedframework and language through which these communities describe their successes.
Also, while communities may offer a variety of different types of STEM programming, they all share the sameessential program elements:
•
•
•
•
•

Dosage: At least 15 hours of STEM programming,
Target Population: Initiative serves all age groups with arequirement that at least 50% of participants are in Grades 4-8,
Content: Consistent with National Standards,
Research: Demonstrated evidence of efficacy, andSetting: Fit for delivery within youth development settings.

Each community exerts local control, making programmatic and financial decisions within the network’s
guidelines and standards. Communities are also responsible for maintaining high-quality STEM programming andprofessional development. They have increased capacity to do this by leveraging the national network ofcommunities and resources across organizations with the community itself.

2

Imagine Science Orange County
The Orange County, California community is a prime example of Imagine Science’s guiding principle that youth
benefit the most when youth-serving organizations collaborate effectively together. This community has had across-organization partnership since 2015 including the 4-H of Orange County, Boys & Girls Clubs of Garden
Grove, Boys & Girls Clubs of Huntington Valley, and Girls Inc. of Orange County. Imagine Science Orange
County (ISOC) was one of three pilot communities of Imagine Science, and one of its first county-wide models. It
currently serves 840 youth from the Garden Grove, Fountain Valley, Huntington Beach, Westminster, Anaheim,
Santa Ana, and Costa Mesa communities.
Local leadership from each of these youth-serving organizations convene to share knowledge, resources, lessons
learned, and best practices across the community. Dr. Gil Noam, Founder of PEAR, who has witnessed theevolution of this partnership described that: “From the beginning, [this initiative] was values-driven and…topdown.” Therefore, while many essential elements of this work occur and are controlled at the local level, thisinitiative also has the support and standards that are set by the national leadership at each organization and thenational network of communities. This national support streamlines knowledge- and resource-sharing, trustbuilding, and collaboration at the local level. It also empowers local leadership to carry out their shared goal ofproviding high-quality STEM programming to youth who have been historically underrepresented as the“communities know their youth, and their challenges, best” (Clark et al., 2017).
While all organizations work together to improve the quality of STEM programming, each organization has unique
resources to bring to the partnership. Currently, ISOC is comprised of four representatives: Heather White, Ryan
Brenes, Rita Jakel, and Anna Reynolds. Noam noted in describing this group, “they are really excited about eachother, respectful and trusting… [and there is an] openness to learn from each other.” They each have started totake on more specialized and defined roles in their approach to their combined work. These roles have shifted
over time, and they might continue to evolve as different representatives join the collaboration (Figure 1).
Figure 1. Collective Approach

For example, Rita Jakel, Community Education Specialist from 4H of Orange County, was a science teacher for over three
decades. She has access to the National 4-H Council’s STEM
curriculum kits. Annually, 4-H holds a STEM Challenge where4-H programs across the country compete to develop themed
curriculum kits. These STEM curriculums are known for being
innovative and relevant and provide 4-H with up-to-date content
to use in their STEM programming. Jakel combines resources
available from these STEM challenge kits, along with other peerreviewed 4-H curricula. Then, she shares them with the other
ISOC representatives.

Jakel also draws from the National 4-H Council’s STEM
curriculums to create and deliver trainings for the STEM programfacilitators from Boys & Girls Clubs or Girls Inc. who will
facilitate these STEM programs. In March 2023, Jakel lead a
STEM Facilitation & Curriculum Training where 83% ofattendees reported that they had been working at their site for less than two years. In describing these trainings,
Jakel explained that the facilitators, “relate really well with the youth in the program, but a lot of them are nottrained as educators…they may not have come with a STEM focus, background, or education – they may even be
afraid of STEM.” Therefore, Jakel plays a key role in ensuring STEM facilitators feel comfortable and confident indelivering the content of STEM programs and how they are delivered.
3During the training, attendees practice the scientific process, they attempt the activities they will later facilitate withyouth and use their own experiences with failure to determine how to modify and troubleshoot activities. After the
March 2023 training, a survey of attendees showed that most attendees found that the training was relevant to theirwork, endorsed that it gave them an opportunity to practice the lessons they would be facilitating, and, finally,
reported that it increased their confidence with STEM concepts.
Heather White, Senior Director of Program Impact, from Boys & Girls Clubs of Garden Grove, Ryan Brenes,
Director of Operation, from Boys & Girls Clubs of Huntington Valley, and Anna Reynolds, STEM Coordinator,
from Girls Inc. of Orange County are responsible for program delivery. Their organizations are successfully
reaching youth that are often under-served in STEM programs. They also work with STEM facilitators and youth
day-to-day, so they are tightly connected to the needs of youth in their community.
Figure 2. Continuous Improvement Model

Select High Quality
Curriculum

Deliver Updated
Curriculum

Adapt to Imagine
Science Hrs & Grades

Girls Inc. has traditionally integrated inquiry-based learning into its
approach to STEM learning. Reynolds applies this approach notjust to her work with youth, but also to her approach to continuous
program improvement. She has spearheaded ISOC’s process ofpiloting, testing, and improving the 4-H STEM curriculum that has
been brought to the group (see Figure 2). She described how sheuses her own work with youth and the experience of other ISOC
representatives in an iterative process.

In one instance, Reynolds decided to revamp one activity, so thematerials were better suited to her site. Specifically, she added thespace lander activity in place of the communications tower activity
Pilot Curriculum
as an engineering challenge that was better suited for her students.
This modification made its way into the standard curriculum for the
next site to use in their delivery of the lesson. She explained: “As I
was running [the STEM curriculum], I was practicing some
activities. I also was able to bring in a new activity…because I saw that one of the activities didn’t quite mesh forwhat was working with my group.” She goes on to say, “Then I brought that back to our collaborative as a group,
which is something that we added into our curriculum.”
Refine Based on
Delivery & Data

Train Faciliators on
Curriculum andIncrease Relevance& Appeal of
Activities for
Faciliators and YouthWhite and Brenes play a critical role in the continuous improvement process. Like Reynolds, they deliver and takepart in providing feedback, adapting the STEM programming, and scaling the programming across their sites.
Also, Girls Inc. and the Boy & Girls Clubs are responsible for collecting youth data. Since 2019, ISOC has
collected surveys from 825 youth, representing over one-sixth of the total Imagine Science youth survey data.

Launching Space Quest
In January 2022, Imagine Science brought communities across their network together through a webinar to share
best practices in curriculum development. ISOC participated in this webinar and heard about how 4-H agentsfrom Imagine Science Greater Houston and Imagine Science Lancaster County had developed a curriculum called
Space Quest by combining curricula from two related 4-H STEM Challenge kits: Galactic Quest and Mars Base Camp.
These 4-H STEM Challenge kits are aligned to National Standards, and included the expertise of NASA scientists.
These 4-H agents had also created and added extension activities to the curriculum so that the entire program met
Imagine Science’s goal of 15 hours of programming. To make the curriculum accessible to all communities in the4Imagine Science network, a resource folder was shared with everything needed to run the program, as well as
information on how to prepare staff to lead Space Quest.
Imagine Science’s Space Quest curriculum enables youth to learn about
and explore space (Figure 3). In the Space Quest program, participants
take on the role of “Space Cadets” participating in a Space Academy as
they complete various missions. In each mission, Space Cadets learn
about space “STEMinistas” (i.e., famous STEM figures), explore thetechnology and resources needed for missions, and tackle authentic
obstacles STEM professionals encounter in space exploration (Figure4). Throughout the curriculum, youth participants are practicing an
inquiry-based learning model asking themselves “What?” “So what?”and “Now what?” as they proceed in activities.

Figure 3. “Space Cadets,” Boys &Girls Clubs of Garden Grove, 2022

Nav Deol-Johnson, Imagine Science National Program Operations
Manager, observes how each Imagine Science community is able totake the curriculum and deliver all of it or part of it to best fit the needsof the youth they serve. She describes that this process is often very“collaborative” across the network, while at the same time it allows room for localization to a particularcommunity. According to Deol-Johnson: “…every community looks a little bit different even though they all[have the same curriculum]; it [doesn’t] look the same because they add supplements.”
Figure 4. “Space
Cadets,” Boys & Girls
Clubs of Garden Grove,
2022

ISOC first implemented Space Quest in Spring 2022. The STEM curriculum needed tofit each program format used within ISOC sites: (a) camps that occur during a short,condensed school break, (b)summer camps in which recreation is more central thanacademics, and (c) semester- long after school programs in which participant
attendance might vary across days. Space Quest was first piloted in a camp-type settingwith Girls Inc. facilitators. Each session was 6-hours for the duration of a 4-day
school break. After delivering in this format, the curriculum was refined and modified
to be delivered at Boys & Girls Clubs. In the summer of 2022, two Boys & Girls
Clubs of Garden Grove sites delivered Space Quest. The curriculum continued to berefined after this delivery so that it was even further improved by the fall. By the fallof 2022, the program was implemented at two Boys & Girls Clubs Huntington Valleysites. In total, the Space Quest was gradually delivered at a total of five Imagine Science
Orange County sites in three formats: as a Girls Inc. Camp, Boys & Girls Clubs
summer programming, and Boys & Girls Clubs academic year programming.

A post-program questionnaire was sent to past Space Quest participants. It consisted ofeight multiple-choice and open-ended questions about their program experience (i.e., “What was your favorite partof Space Quest?”) and takeaways (i.e., “What is one thing you learned during Space Quest that you still use in yourscience classes at school?”). In March 2023, nine youth who participated in Space Quest at either Girls Inc. in Spring
2022 or Boys & Girls Clubs of Garden Grove in Fall 2022 completed this questionnaire. Of the youth who
disclosed their gender identity, three were boys and five were girls. As of the time they completed thequestionnaire, five respondents were in fifth grade and four were in sixth grade. Respectively, these youth werelikely in fourth and fifth grades while they were in Space Quest.
Most of the youth named the “Rover Activity” as the one that they most wanted to do again. In this activity,
youth applied the entire engineering design process: they brainstormed, designed, built, tested, evaluated, and5redesigned. Three of these youth also listed “Engineer” as the
career that most excites them based on what they learned inSpace Quest. In their explanations, they noted how Engineers
make their ideas a reality: “I’d get to make an abundance of
unknown things, that kids like me dreamed of” and, “I can learn
and experiment new things and the things I can imagine.”[Space Quest] was wonderful,chaotic, fun, and basically the bestclub I’ve ever experienced.”–5th Grader at Boys & Girls Clubs of Garden Grove

Beyond the opportunities to practice STEM skills and form STEM career aspirations, Space Quest enabled youth tocollaborate and develop friendships: “[Space Quest] would help them cooperate with their peers and gain new
knowledge.” A few respondents mentioned that their favorite parts were interacting with other participants. The
words “fun” and “exciting” were each mentioned 11 times in relation to their experience in the program,highlighting that Space Quest participation afforded recreational benefits in addition to the academic and social ones.

Common Instrument Suite Findings: Space Quest and All ISOC STEM Programming
ISOC collects data from youth using the Common Instrument Suite – Student Survey (CIS-S). The CIS-S is ayouth self-report survey and the version that ISOC uses measures three scales: STEM engagement, STEM identity,
and STEM career interest. The reliabilities for these scales were high, ranging from 0.80 to 0.93 (Allen et al., 2019).
ISOC administers the CIS-S at the end of spring, summer and fall programming in a Retrospective Pretest-Posttest(RPP) format. The CIS-S was reported on a 4-point Likert scale from “Strongly Disagree” to “Strongly Agree.”Overall Youth Demographics
Of the 825 Imagine Science Orange County youth who took the CIS-S between 2019 and 2022, 602 met Imagine
Science’s evaluation inclusion criteria (i.e., were in 4th-8th grades and participated in at least 15 hours of Imagine
Science STEM programming) (Table 1). The majority (55%) of youth identified as girls and 41% as boys. For
grade distribution, fifth graders made up 37% of
Table 1. Imagine Science Orange County (n = 602) Sample
the sample, followed by sixth/seventh graders
Description, 2019-2022(32%), and fourth graders (27%). The remainder
Space Quest
Overall
Variable(4%) were eighth graders. Over a third (34%) of# of Youth (%)# of Youth (%)
Year
the sample identified as Latino or Hispanic. The
2019
N/A
173 (29%)
other main race/ethnicity groups represented in
2021
N/A
231 (38%)
the sample included Asian, Asian-American2022
45 (100%)
198 (33%)(20%); White, Caucasian (17%); andGendermultiracial/multiethnic (10%). The remainderBoy
18 (40%)247 (41%)Girl
26 (58%)332 (55%)(11%) included youth whose race/ethnicity wasGrade
not listed and those who preferred not to answer.
Fourth8 (18%)
164 (27%)
Over half (53%) of youth speak a language otherFifth
15 (33%)222 (37%)
than English at home. This large percentage
Sixth/Seventh
22 (49%)194 (32%)
mirrors Orange County’s diverse population as
Eighth0 (0%)
22 (4%)
Race/Ethnicity
nearly one-third of the population is comprised
Asian, Asian13 (33%)
119 (20%)of immigrants (United States Census Bureau,
American2022).
Latino or Hispanic
18(45%)
203 (34%)

Space Quest Demographics
In 2022, 265 youth took the CIS-S across Orange
County; 198 of these youth met the Imagine
Science program inclusion criteria, meaning they
were in 4th-8th grades and participated in at least
80% of the 15 hours of STEM programming. Of

White, CaucasianMultiracial/ethnic
Prefer not toanswer/not listed
Primary Language
English
Non-English

0 (0%)
3 (8%)
6 (15%)103 (17%)59 (10%)64 (11%)3 (8%)
34 (90%)226 (38%)321 (53%)

6

these, 45 youth participated in Space Quest programs (Table 1). The majority (58%) of Space Quest participants
identified as girls and 40% as boys. Youth in this sample were in fourth through eighth grades, with nearly half(47%) in sixth grade, a third (33%) in fifth grade, and the remainder (18%) in fourth grade. The largest
race/ethnicity identity represented in the sample was Latino or Hispanic (45%). Other major race/ethnicity groups
included Asian, Asian-American (33%) and multiracial/multiethnic (8%). These top race/ethnicity categories arealigned with the county demographics as there are sizeable Latino and Asian communities (United States Census
Bureau, 2022). Thirteen percent of youth preferred not to disclose their race/ethnicity identity. Nearly all (90%) ofyouth spoke a language other than English at home.

STEM Outcomes
To examine the changes in STEM outcomes from before and after the STEM programming, a “difference score”
for each scale was calculated by subtracting the scale’s retrospective-pre mean from its retrospective-post mean.
Then, these difference scores were analyzed to see if they differed significantly from zero. If the p-value of a given
scale was below 0.05, its mean difference was significant and unlikely due to chance. We first examined theoutcomes of all ISOC programming. A one-sample t-test revealed that, on average, Imagine Science Orange
County youth reported statistically significant positive change on all three CIS-S scales (p’s < 0.001): STEM careerinterest, STEM engagement, and STEM identity (Figure 5). We repeated this analysis for youth who participatedin the Space Quest program. Similar to the pattern we saw with all ISOC programming, we found that Space Quest
youth reported statistically significant positive change on all three CIS-S scales (p’s < 0.001): STEM career interest,
STEM engagement, and STEM identity (Figure ). To examine the magnitude of these differences, we computed
effect sizes using Cohen’s d for all ISOC STEM programs and the Space Quest program, respectively, and found a
medium-to-large effect for the different programs (Cohen, 2009): STEM career interest (d = 0.79, 0.69), STEM
engagement (d = 0.91, 0.84), and STEM identity (d = 0.72, 0.54).
Figure 5. Averages on STEM Outcomes Youth in Space Quest and All STEM Programs, Imagine Science
Orange County, 2019-2022

Space Quest Youth, 2022 (n = 45)
4.00
3.00

All Youth, 2019-2022 (n = 602)
4.00

3.05
2.66

3.20
2.80
2.31

2.61

2.00

3.00

2.68

3.11

2.74

3.14
2.43

2.80

2.00
1.00

1.00
STEM Career
STEM
Interest* Engagement*
Pre

Post

STEM
Identity*

0.00
STEM Career
STEM
Interest* Engagement*
Pre

STEM
Identity*

Post

We compared ISOC effect sizes to those found in a meta-analysis of 15 studies on measuring STEM interest in
OST programs (Young et al., 2017). These U.S.-based programs served K-12 students and were comprised ofsummer camps, after school programs, and academic fairs. Compared to effect sizes found in a meta-analysis ofhigh-quality, OST STEM programming (Young et al., 2017), these effect sizes are similar to the medium effect sizeof STEM programs that had an academic and social-emotional focus, rather than the small effect size of STEM
programs that had just an academic focus. OST STEM programming often includes social-emotional elements,
7

such as time to develop relationships with adults and peers and opportunities for collaboration and teamwork.
Research shows that these programs have a greater impact on youth engagement and interest in STEM than STEM
programs in schools (Young et al., 2017).

Discussion
Through a mixed-methods approach, we explored how the Orange County community operates within the
Imagine Science network. We observed how this national-local partnership allowed for the adaptation and deliveryof new, high-quality STEM learning experiences. Imagine Science, and ISOC in particular, leverages what works
best in integrating STEM learning into youth development, a powerful combination that has been shown toincrease STEM outcomes (Allen et al., 2019).
When OST organizations offer high-quality learning experiences that are widely accessible and have sustainable
participation, they can maximize their impact on the academic, social-emotional, and health and wellness outcomesof youth (Little et al., 2008). As such, the youth that participated in ISOC programs reported significant positivechange across all measured outcomes. Further, the effect sizes of these positive changes are similar to establishedbenchmarks of high-quality OST STEM programs (Young et al., 2017).
The ISOC collaboration enhances the Orange County community’s capacity for high-quality OST STEM
programming. They do this through a data-driven continuous quality improvement process and by utilizing bestpractices in sharing content expertise, training, and curricular resources (Groome & Rodríguez, 2014). This joint
effort allowed “professionals across projects … to generate and carry out creative solutions and strategies thatmaximize[d] benefit beyond that which each entity could accomplish” (Project Exploration & Coalition for Science
After School, 2009). As one funder of Imagine Science described the partnership, ISOC represents: “the best-caseexample of collaboration they have seen.”
The way ISOC worked together in adapting Space Quest showcased this approach. However, ISOC’s goal goesbeyond any one curriculum. As Reynolds described, they want to “build a toolbox of curricula that we can choose
from, so that we can sustain the collaboration we already have and also have variety” in STEM programming.
Indeed, this group is already piloting Imagine Science’s next STEM program – a water curriculum (see Note 2)that has been shared across the Imagine Science network. The spring ISOC pilot sites will be sharing their
feedback with the network this summer.
As Noam, described: ISOC has a “transformational aspiration so that [the work they do] actually penetrates eachindividual organization.” As we have seen with Orange County, the impact of an effective local partnership canreverberate beyond any one community. The Imagine Science’s network helps elevate innovation that occurs
through local partnerships to its national network, which then brings the work back to local communities acrossthe country. Therefore, among this network, innovative and best practices spread quickly. In consequence,
communities are better equipped to deliver high quality STEM programming and engage youth who have been
historically underrepresented in STEM.

Acknowledgements
We are grateful to Imagine Science for supporting this report. Thank you to the Orange County Imagine Science
community, their administrators and facilitators, and the families and young people who participated by completingsurveys. We also thank the team at PEAR Inc. for their work in data collection, management, analysis, andreporting.

8References
Allen, P. J., Chang, R., Gorrall, B. K., Waggenspack, L.,
Fukuda, E., Little, T. D., & Noam, G. G. (2019). Fromquality to outcomes: A national study of afterschool STEM
programming. International Journal of STEM Education, 6(1),37. https://doi.org/10.1186/s40594-019-0191-2
Catalano, R. F., Berglund, M. L., Ryan, J. A. M., Lonczak, H.
S., & Hawkins, J. D. (2004). Positive Youth Development in
the United States: Research Findings on Evaluations of
Positive Youth Development Programs. The ANNALS of theAmerican Academy of Political and Social Science, 591(1), 98–124.
https://doi.org/10.1177/0002716203260102
Clark, J., Sirangelo, J., Council, H., Washington, K.,
Vredenburgh, J., Chesire, J., & Mission, B. (2017). Imagine
Science: A Coalition of National Youth Organizations
Bringing STEM to Scale. STEM Ready America, 11.
Coalition for Science After School. (2008). Staff capacity andprofessional development for after-school STEM: A summary of key
research. Coalition for Science After School.
https://www.informalscience.org/sites/default/files/20140624_2009_Staff_Capacity_and_Professional_Development.p
df
Cohen, J. (2009). Statistical power analysis for the behavioral
sciences (2. ed., reprint). Psychology Press.
Cook-Endres, T., Taylor, A., & Penuel, B. (2014). Using
curriculum adaptation as a strategy to help teachers learn about NGSSand developing aligned instructional materials (No. 5; STEM
Teaching Tools Briefs).
https://stemteachingtools.org/assets/landscapes/STEMTeaching-Tool-5-Curriculum-Adaptation.pdf
Cooper, B. (2013). Teaching the What as Well as the How.
Fitzsimons, S., Coleman, V., & Greatorex, J. (2020). Context
matters—Adaptation guidance for developing a local
curriculum from an international curriculum framework.
Research Matters, 30, 12–18.
Groome, M., & Rodríguez, L. M. (2014). Collaborating toStrengthen STEM Programming in a Citywide System.
Hein, G. (2009). Learning Science in Informal
Environments: People, Places, and Pursuits. Museums &Social Issues, 4(1), 113–124.
https://doi.org/10.1179/msi.2009.4.1.113

Hill, S. (2012). Leap of faith: A literature review on the effects ofprofessional development on program quality and youth outcomes.
National Institute on Out-of-School Time.
https://www.academia.edu/85279257/Leap_of_Faith_A_Li
terature_Review_on_the_Effects_of_Professional_Develop
ment_on_Program_Quality_and_Youth_Outcomes?f_ri=1601
Kuperminc, G. P., Seitz, S., Joseph, H., Khatib, N., Wilson,
C., Collins, K., & Guessous, O. (2019). Enhancing Program
Quality in a National Sample of After-school Settings: The
Role of Youth–Staff Interactions and Staff/Organizational
Functioning. American Journal of Community Psychology, 63(3–4),
391–404. https://doi.org/10.1002/ajcp.12329
Little, P., Wimer, C., & Weiss, H. B. (2008). After School
Programs in the 21st Century: Their Potential and What it
Takes to Achieve It. Harvard Family Research Project, 10, 1–12.
Noam, G. G., & Shah, A. M. (2014). Informal Science andYouth Development: Creating Convergence in Out-ofSchool Time. Teachers College Record: The Voice of Scholarship in
Education, 116(13), 199–218.
https://doi.org/10.1177/016146811411601311
Project Exploration, & Coalition for Science After School.
(2009). A watershed moment: The first National Conference on
Science and Technology in Out-ofSchool Time.
https://search.issuelab.org/resource/a-watershed-momentthe-first-national-conference-on-science-in-out-of-schooltime.html
United States Census Bureau. (2022). Quick Facts: Orange
County, California.
https://www.census.gov/quickfacts/orangecountycalifornia
Yohalem, N., & Wilson-Ahlstrom, A. (2010). Inside theBlack Box: Assessing and Improving Quality in Youth
Programs. American Journal of Community Psychology, 45(3–4),
350–357. https://doi.org/10.1007/s10464-010-9311-3
Young, J. R., Ortiz, N., & Young, J. L. (2017). STEMulating
Interest: A meta-analysis of the effects of out-of-school timeon student STEM interest. International Journal of Education in
Mathematics, Science and Technology, 5(1), 62.
https://doi.org/10.18404/ijemst.61149

9

Note 2. Future Efforts
Following the success of their Space Quest curriculum, ISOC has spearheaded the development of anotherprogram, Water Guardians, as a means to expand their offerings of high-quality, STEM programming and continue
to engage the youth their sites serve in a novel and relevant way. This curriculum explores a different content areaof STEM but maintains much of the same structural elements and development process.
Like Space Quest, the process of developing Water Guardians began with a 4-H STEM Challenge curriculum kit,
Explorers of the Deep, and other resources, such as Rain 2 Drain, and Water Wizards, at its core. Supplemental
activities were then added to the curriculum to fit Imagine Science’s 15+ hour criteria. ISOC sourced some ofthese activities from other Imagine Science communities as the ISOC leaders know that these materials are alreadyvetted according to Imagine Science standards. Though the launch of the 4-H STEM Challenge kit in late 2022
was the initial impetus for Water Guardians, a water-based curriculum quickly became locally relevant to thecommunities that ISOC serves. In the ‘22/‘23 winter, southern California experienced particularly heavy rainfall
and, in some areas, significant flooding. Then, the activities that ISOC had been collecting about rain gardens,
flood barriers, and flood control became even more topical and applicable.
By March, ISOC had structured and finalized the Water Guardians curriculum. Jakel then created and delivered a
training for future curriculum facilitators. Water Guardians is currently being piloted at a few ISOC sites. Feedback
from the facilitators will be collected and used to refine the curriculum as it is implemented at more sites in thesummer.
Beyond its efficiency, ISOC’s quarter-long process of adaptation and then further refinement of curricula allows it
to be agile and deliver new STEM programming that is responsive to the current lived experiences of theparticipating youth. As Jakel mentions, “it’s a set pattern now. Everyone that worked on [Water Guardians] feels
like we generated a pretty good product. There’s a possibility [to apply this process] in the future for other topics
nationally with our 4-H educator network through Imagine Science.” Water Guardians illustrates another instancein which an Imagine Science community is using an established process of cross-organization collaboration. Suchcollaboration efficiently generates new and engaging STEM programming using resources already available within
the network.

10