NASA Advisory Council

Science, Technology, Engineering, and Mathematics (STEM)

Engagement Committee

NOTES

February 17, 2022

STEM Engagement Committee Members

Present: Dan Dumbacher (Committee Chair), Jamarius Reid (President, Student Government 

Association), Norman Fortenberry (Executive Director), Ray Mellado (Founder & Chairman), 

Tara Strang, Kristen De Vivo (Executive Director), Darryl Williams (Senior Vice President of 

Science and Education)

Others Present: Kris Brown, Mike Kincaid, Lisa Stewart, Bev Girten (Executive Secretary), 

Eleasah Banks, Kelly Calagna, Anne Marie Demme, Ashley Bishop, Rick Gilmore

Opening Comments

Mike Kincaid welcomed everyone and noted the virtual meeting is public. Dan Dumbacher

provided introductory comments. The committee members introduced themselves and 

Jamarius Reid was recognized as the first student representative through a competitive 

selection process. Mr. Kincaid provided an overview of the agenda.

Dr. Beverly Girten, Committee Executive Secretary, welcomed everyone and noted the meeting 

is a virtual meeting via Webex and is taking place in accordance with the Federal Advisory 

Committee ACT (FACA). Dr. Girten also requested for everyone to keep mic muted when not 

speaking and noted that the meeting is being recorded. 

Landscape

Mike Kincaid gave an overview of NASA’s Office of STEM Engagement’s vision, mission 

statement and strategic goals. Mr. Kincaid explained that NASA is in the center of the STEM 

ecosystem and noted the alignment between the OSTEM and OSTP/OMB Priorities. He also 

reminded the committee that events across the agency continuously drive the landscape we 

work in. Mr. Kincaid mentioned that during her first National Space Council Meeting, Madame

Vice President requested the prioritization of using space to engage students in STEM.

Overview of STEM Engagement

Kris Brown described NASA’s STEM engagement efforts as a continuum of learning 

opportunities and products for students to first attract them into STEM and then support them 

as they participate in the STEM pipeline. Attracting students into STEM is being achieved by 
employing evidence-based opportunities and ensuring accessibility of all efforts. Ms. Brown 

detailed the three primary areas of focus for K-12 efforts; broadening student participation, 

continue building productive strategic partnerships and networks, and expanding NASA’s 

contributions in engaging K-12 students. 

Mike Kincaid emphasized the importance of reaching students through teachers. NASA 

effectively supports the educational community by providing standard aligned lessons, guided 

educational activities, the NASA Express newsletter, and professional development 

opportunities. He introduced NASA CONNECTS, a program pilot targeted for Spring 2022 where 

educators can communicate with other educators, ask questions, collaborate, and share best 

practices. Students can engage directly with NASA through challenges and experiences offered 

on nasa.stem.gov such as the Artemis Student Challenges. The Artemis Student Challenges 

group together opportunities for students that contribute to the Artemis missions. NASA’s 

Internship program is well received, accepting 27,000 applicants during the Summer of 2021.

Mr. Kincaid shared program updates for Space Grant, EPSCoR, MUREP, and NextGen STEM. Mr. 

Kincaid included that within our MUREP program, OCEAN was developed with a focus to open 

opportunities for MSI’s to participate in NASA’s market. NextGen STEM, the program focused 

on k-12 efforts has undergone significant growth. Within NextGen STEM, there is TEAM II, lead 

by Dr. Bev Girten, gives out about 3 awards per year totaling $1,000,000 over a three year span. 

Many of the opportunities are funded by mission directorate contributions. It was reported that 

almost 30% went to minority students and 41% went to women. The NASA OSTEM 2021

Highlights report will be distributed before the full NAC meeting at the end of the month and 

contains a snapshot of successes from the past year. Public engagement has seen steady 

growth and the time spent on the nasa.stem.gov is up 17% between FY20 and FY21.

NAC Topics Review

Dan Dumbacher presented a review of the previous NAC meeting in October 2020. Mr. 

Dumbacher highlighted the concern for what kind of equity issues that could stem from COVID. 

He congratulated NASA STEM for the quick response in getting NASA STEM at home materials 

available to the public addressing some of the challenges during early stages of the pandemic. 

Mr. Dumbacher also commended NASA’s strategic goals. While the alignment with the NASA 

mission is clear, he suggested the prioritization of opportunities for minorities and underserved 

communities. OSTEM has built a great positive partnership with the National Science 

Foundation with the INCLUDES program and are open to other potential partnerships within 

the MUREP program. It was noted that there may be an opportunity for NASA to play a 

convening role in reinventing the education ecosystem. Discussion centered on the potential 

ability to leverage partnerships and capabilities of both NASA and the National Science 

Foundation to influence learning in the virtual environment. 

Mr. Dumbacher reported a set of findings from the previous NAC Meeting. Mr. Dumbacher

then reported the recommendations that was taken to the NAC that required attention and 
noted that all of the recommendations were passed along to the NASA advisory council and 

well received. 

Mr. Mellado inquired if NASA can share slides and data with other agencies in the federal 

government. Mr. Kincaid responded that yes, it can be shared and encouraged Mr. Mellado to 

do so, stating that; the more we can connect, the more the federal government is as well as our 

students. 

Kristin De Vivo asked Mr. Dumbacher to elaborate on the thought behind reinvention of the 

ecosystem concept present in his slides. Mr. Dumbacher responded stating they did not get into 

the details, methodologies, or solutions, just identified the need and will look to address the 

thought further in future meetings. Mr. Kincaid added that since it’s been 18 months since the 

last meeting we wanted to share the goals and where we were previously. He stated for new 

attendees that an added benefit of reviewing past findings and recommendations is that it 

illustrates an example of what the group will have a new version of by the end of the meeting.

Dr. Girten reminded the committee what was presented was in an informal abbreviated way to 

the full NAC and never went forward officially, they weren’t put on the appropriate forms, so 

the council did not consider them. They are aware of them, but no formal action has been 

taken yet. If you want action then we will need to add them into the new recommendations 

and findings we develop today. Mr. Kincaid reiterated the 3 priorities of STEM, Broadening 

Participation, K-12 Efforts, and Partnerships.

Broadening Student Participation

Mike Kincaid reaffirmed the importance of and commitment to DEIA to NASA. Mr. Kincaid

reviewed the purpose and overarching goals of this NASA Stem focus area. A team was 

developed to create a broadening student participation plan whose objectives will track the 

effects on our programs’ evaluation and approach. Mr. Kincaid explain the importance of 

reviewing past action, evaluating all efforts, not just MUREP and considering how to leverage 

partnerships. Mr. Kincaid reviewed MUREP’s vision and it’s 4 pillars of investment and 

engagement. Mr. Kincaid then notes the executive orders that serve as a foundation for what 

NASA does. Mr. Kincaid rhetorically inquired how to make a difference in engagement needed

to aid each level of awareness, degree commitment, degree persistence, and degree 

attainment. 

Mr. Kincaid shared data from the NASA Internship team breaking down the percentages of 

ethnic, racial, and gendered groups in the FY20 and FY21 intern groups. In reviewing the FY21 

interns that were managed by the Office of STEM Engagement, the data is more diverse than 

what you’d find in our nation’s representative breakdown of diversity in STEM degrees. The 

applicant pool has changed drastically between FY20 and FY21 with almost 12,000 more 

students applying (a 73% increase). In terms of underserved populations, we determine at 
whether they are applying, if they are being selected, and how they perform in those roles. 

There was a discussion explaining the percentages and the way the data was presented. Mr. 

Dumbacher asked if the overall number of applications went up and the number of applicants is 

higher than the percent increase for those who were selected, what does Mr. Kincaid think the 

driver is. Mr. Kincaid responded by stating that we have to look at if the overall number, 73%, 

the numbers of the applicants went significantly up. So if the whole group grew by 14%, 

compared to this number, the only group that did not increase was the non-minority white 

students. Norman Fortenberry suggested having the chart show the percentage of the total 

pool instead of the percentage of the increase. Mr. Kincaid agreed that would be useful and 

then referenced the slide with the pi chart. For comparative purposes, Mr. Fortenberry stated

he would like to see the same data for applicants selected (the current chart is applicants as a 

whole). 

Darryl Williams asked when looking at the applicant pool, the stratification in terms of who are 

applying, and if there is data on awareness mechanisms. In other words, how are students 

finding out and learning about this and what are the strategies that different places use to 

make students aware of how these opportunities exist. Mr. Kincaid agreed that it would be 

helpful to know how this, he is unaware of whether the intern team currently tracks that data 

point. Kristin De Vivo agreed in the benefit of tracking that metric to figure out how to influence 

DEI across opportunities based on how students are hearing about them. 

Jamarius Reid stated when looking at the data what stands out is the difference in the Asian 

group and white group. The white student group has maintained, and the Asian student group 

has increased. But he was curious to know how that is reflected in the other minority groups. 

Mr. Kincaid pointed out that Black and Hispanic groups have also significantly increased

explaining the slide data. Mr. Reid explained what PAXC is and its contribution to creating a 

community for interns at NASA. Mr. Kincaid adds that you’re more likely to stay at a job where 

there are more people that are like you, groups like PAXC provided this connection agency wide 

for students to relate to other people in a similar situation. 

Mr. Dumbacher asked to discuss inequalities in the selection process and if there had been 

progress. He observed there was an increase of 99% in American Indian or Alaskan Native 

students applying then to only 11% increase in being selected. Mr. Kincaid responded clarifying 

how those percentages actually compare, as it is not exactly 99% to 11% but rather, 99% to 73% 

for applications and then went up in selection by 14%, so 14% should be compared to the 11%.

Mr. Kincaid agreed on the importance of discussing inequities and potential biases. He 

explained how mentors select their interns and that there is always room for improvement. Kris

Brown commented on bigger picture of the internship program and looking ahead. This team is 

comprised of leaders within OSTEM’s internships programs with respect to practices and 

approaches. One focus is the DEI piece and allowing them training opportunities, community, 

and other opportunities. Another element is training awareness for the mentor community 
who are working with these students. Also, awareness around their selection processes and 

how they look at their applicants and what they’re looking for. This has been powerful for the 

community at large. The community is also working on best practices for intake process 

including selection processes. NASA has an internships outcome study that has been looking at 

value proposition for interns in terms of what they’ve taken away. 

Mr. Dumbacher commented the importance of being very intentional in the selection process. 

Mr. Kincaid agreed. Mr. Fortenberry broke down the data from 2020-2021 to clarify the 

numbers for the group one last time. Ms. De Vivo advised that a factor analysis could be applied 

to see what factors are driving the selection process. Ms. Brown shared that NASA has used 

Space Grant who sponsors interns through funding, but in addition to that, has been working 

with advocacy groups and associations that are devoted to these constituency groups. She 

commented that it would be interesting to see if OSTEM could do some qualitative analysis and 

have the committee weigh in on whether there are other vehicles or avenues that should be 

used.

K-12 Efforts

Kris Brown shared the critical need for investments in K-12 and notes that all prior dialogue 

regarding broadening student participation applies to this audience. It is important to consider 

how to attract, engage, and support students involved in stem and what incremental learning 

activities can be provided to support students. Ms. Brown discussed the drivers to design a K-12 

foundation and the K-12 framework. NASA continues to develop unique learning opportunities 

to attract students formally and informally such as the pilot NASA CONNECTS which allows 

NASA to build relationships with teachers. OSTEM is focused on outcomes and the ability to

measure progress and then take that evidence to impact future programs and opportunities. 

One of the building blocks is partnerships and the systematic approach to build them within 

networks of the larger STEM ecosystems. 

Mr. Dumbacher opened the floor to questions or comments. Mr. Reid mentioned that there 

was nothing stated about innovation. He asked if OSTEM can create an opportunity to explore

this generations’ values and that it might be beneficial. Mr. Fortenberry shared there might be 

an opportunity for analysis from a social scientist point of view and the research that is 

available. Kris advised that the educational tools and platforms have done work in this area by 

taking on personas to see how they would navigate the space that is out there. Mr. Williams

mentioned that there might be an opportunity to do more work with the National Science 

Foundation as they have a lot of rich evidence-based data on the communities of practice. Mr. 

Dumbacher ask if Ms. Brown had communicated with the National Science Teachers 

Association to which she confirmed that OSTEM has a relationship with them and that is a great 

avenue to pursue. Ms. Brown also informed the group that OSTEM has a recent hire coming in

from NSF who is bringing thoughts on how to engage with K-12 formal education which will 

help position us to work with NSF and their studies. 
Ms. De Vivo provided praise for the work going into this effort. She mentioned that it looks like 

the team is looking at the missions for the end goals of the learning. If so, that does not 

necessarily align with the goal of setting students up to be a part of this diverse workforce. If 

we were to look at an opportunity to broaden the approach to think about the overall 

development for students in STEM tracks. Ms. Brown explained that NextGenSTEM has gone 

through the methodical analysis that is outcome driven and the learning opportunities that are 

available. K-12 studies are looking at how we shift the work that we have done. The profile of 

the work completed has shifted within the last couple of years based on that evidence. 

Partnerships

Mr. Kincaid shared OSTEM’s current partnerships and goals to expand reach with LEGO and 

STEM NEXT among other partners. He highlighted notable accomplishments in FY2021 including 

12 new agreements, 17 informal agreements, and 94 different organizations OSTEM has 

worked with between April 2020 and May 2021. The Artemis Summit Meeting is an example of 

how OSTEM engages partners to realize high priority agency goals and messaging. These kinds 

of events garner high visibility and support.

Measuring Success

Tara Strang, NASA OSTEM’s assistant program manager of the performance assessment and 

evaluation team, shared an overview of the strategic performance framework and explained

how the strategy is built. The team uses a strategy performance framework to identify a 

performance assessment and then evaluates and addresses their identified learning agenda. 

The goal of the learning agency development and implementation is to generate and refine a 

process to look systematically across performance and evaluation activities, identifying gaps in 

knowledge and establishing a research agenda that generates knowledge to fill these gaps. Ms. 

Strang then provided a high level overview of the contributions to NASA’s missions and work 

from FY19-21 that was done in support of the questions, to what extent OSTEM’s investments 

contribute to NASA’s missions and work, how OSTEM’s investments broadened participation of 

historically underrepresented and underserved groups in the STEM fields in NASA STEM 

engagement activities, to what extent enhancements to OSTEM performance assessment and 

evaluation have been implemented, and what the effective strategies are that best support and 

measure OSTEM investments’ abilities to spark K-12 student’s STEM interests. The new OSTEM 

Gateway will be used to help track student’s engagement over a longer period.

Mr. Fortenberry asked, given the target audience K-12, how is anonymity respected? Ms. 

Strang shared that Gateway only registers students who are 16 and older. Mr. Kincaid then 

shared that the back end of Gateway is Salesforce, which is a tool that pulls the information 

together. He noted that it will still be difficult to help pull data from classrooms and ages 
younger than 16. 

Ms. De Vivo commented there will still be trouble tracking based on terms of use and asked if 

there will be a chance to see the logic model for K-12. Ms. Strang advised that it was created as 

a part of this assessment and will follow up with those results. Mr. Dumbacher asked if there 

were key metrics that are being tracked. Ms. Strang responded that there are different metrics 

supporting each learning question. The research and development products created with 

student support, paper publications and presentations also offer a long-term metric of 

increasing diversity. Ms. Strang noted the program tracks diversity on a yearly basis. 

Mr. Dumbacher asked how open they are open to sharing this information with the larger 

community to help facilitate what metrics are being used. Ms. Strang responded that there is 

an interagency working group to verify transparency and accountability and her team lead Rick 

Gilmore will be attending the upcoming roundtable focusing on this. Mr. Kincaid shared that 

there is an interagency group under OSTP that OSTEM contributes to. Mr. Kincaid advised that 

we are more than open to dive deeper into this in a future meeting with this as a focus. 

Ms. De Vivo asked what Ms. Strang was referring to when mentioning that the outcome 

measure for K-12 is engagement. Ms. Strang responded that they look at the number of 

applicants and participants. Ms. De Vivo then asked if there is a teacher certification path, 

incentivizing to participate through a special training opportunity that way you can see how 

many teachers you certify each year. Mr. Kincaid shared that approximately 5 years ago there 

was a badging process for teachers who had done work with OSTEM, but he was not sure of the 

status. Ms. De Vivo encouraged certification over an itemized badging process. Mr. Kincaid

agreed and as OSTEM explores what NASA CONNECTS contributes, what might the option for 

continuing education for teachers look like. Ms. De Vivo noted that the mediator between our 

resources and students is the teacher. Teachers are not prepared at this time, so whatever 

OSTEM could do to professionalize teaching to support this initiative will then prepare the 

students. Mr. Kincaid advised that OSTEM follow up with Ms. De Vivo individually to spend 

some time discussing this certification option. 

Mr. Williams comments on the need to look at the continuum of formal and informal 

education. He asked; looking at equity and inclusion, a lot of students we are trying to reach are 

engaging in out of school activities so how do we prepare these informal educators?

Finding & Recommendations

Dr. Girten reminds the committee of the changes since the last meeting. Findings are points 

that to take forward to the full NAC that don’t have to have a response or action from the 

administrator. A recommendation goes forward and a response or action is required from the 
administrator. There is a one tier system so everything from a committee will go forward to the 

full NAC and then the full NAC will decide what goes forward to the administrator. All 

recommendations should be a high level item that would warrant a response from the 

administrator. 

Findings

The committee finds that there continues to be evidence that progress is being made honing in 

on the strategic goals and vision for OSTEM. What was presented was very succinct. There is 

intentionality around identifying and making resources available for educators. Regarding 

internships data, there was a deep discussion on the marked progress in reaching a more 

diverse pool of students but there are still questions about what it is that is being effective. The 

committee looks forward to having a future discussion regarding the infrastructure for 

recruiting for internships, including service member and veterans. The progress in building 

partnerships has been significant. The committee recognizes the efforts in evaluations work. A 

thoughtful design and approach were taken. There is an opportunity to still learn about the 

metrics and outcomes used to measure performance, which is a potential focus topic for a 

future meeting. Interested in the aspect of being able to use Gateway to track the long term 

involvement of participants in OSTEM activities. 

Recommendations

The committee recommends to keep up with the emphasis on the strategic plan, integrating 

and using this across NASA’s mission directorates. Administrator and the mission directorates, 

along with the Office of Procurement, should ensure that there are higher goals to build 

research capabilities at Minority Serving Institutions (MSIs). NASA should support and create 

infrastructure to sustain MSIs, to enable them to be competitive and be successful in 

contributing to NASA work. This helps build a strong K-12 pipeline of interest and engagement. 

NASA, other FC-STEM agencies, and other current and potential partners should collaborate to 

support the STEM education community in addressing the disruption of schools and other 

identified challenges resulting from the ongoing pandemic. NASA alone cannot solve this.