Transcript_Technical Webinar Recording Session_MES Dataset_Final Page 1 of 6

MES Dataset Technical Recording Session Transcript

Shane Dermanjian:

Hello, everybody. We're here today to talk about the AmeriCorps member exit survey, sort of the ins 

and outs of it, why we do it, who takes it, what can be gained from it, who can use it, and so on. So to 

start, my name is Shane Dermanjian. I'm a research analyst with the Office of Research Evaluation, ORE 

for short, with AmeriCorps. So just a quick outline of this presentation. We'll first take a look at what is 

an MES, right? Why it's conducted, when it started, some ins and outs and technical points, as well as 

the number of survey takers. Then we'll move to a theoretical background, sort of explaining why we ask 

the questions we ask. Then we'll take a look at some analyses and just how people can actually use this 

data.

So what is the AmeriCorps member exit survey? Or we'll just call it the MES. I'll just read this real quick. 

So the purpose of the member exit survey is to collect information about AmeriCorps members' 

experiences for the purpose of monitoring progress towards the agency's strategic goals around 

member development. So when you think of AmeriCorps, there's always the part where members serve 

a community or they serve an organization, and they make those places and those organizations better 

after serving. But we can also take a look at what serving does to the members themselves, and that's 

really what this survey gets down to. So the current iteration of the survey began in 2015. There were 

some surveys prior to that, but they were revised, they were edited, and the version that's in use today 

began in 2015. In the survey, there's 31 numbered questions, and of those questions, there's many subquestions, so there's 108 total questions.

We do give the survey to all members that do serve, and some members may start serving before they 

turn 18, but the data we look at for any analyses we'll conduct is only inclusive of those members 18 and 

up. So members are offered the MES for as many service terms as they can participate in. So say 

somebody did a term of service in 2016, they would take the survey. Say two years later they decided to 

do another term of service, so they would take the survey then as well. There's no compensation for 

taking the survey, just an opportunity for a member to share their thoughts and feelings on many things, 

which we'll get into. And the MES is offered on a rolling basis, so truly just whenever somebody leaves 

service, they take the survey, or rather, they're given the opportunity to take the survey.

So exiting service though, that's both actually finishing your service term. So say somebody serves a full 

term, 1,700 hours, they take the survey after they finish their full term. Or say a member were to leave 

after just a month. They have to leave for a compelling or non-compelling reason. Regardless, they 

would still be offered the survey to take. And they have about 180 days total to take the survey once it 

appears in their My AmeriCorps portal, although most members will complete the survey just within a 

couple of months, which really, most complete it within just one month.

So this table shows some completion rates according to program, and we can see that after 2015, the 

completion rate just across the board is anywhere from 75 to about 80% each year, which is a pretty 

great response rate. And it varies year to year, varies program to program, but overall, we have a lot of 

responses. So going to the fiscal year '22 data, we have up to 360,000 responses, so this is a very large 

survey that we learn a lot about our members from. And let's see, just to note, for the 2015 data, we 

only have the data from April 2015 through September 2015. We say September, because we collect it 

on a fiscal year basis.

So for the MES completion rates, we could also take a look at those who finished service versus those 

who did not finish service. And looking from 2020 on, the completion rate for those who did not finish 

service, so that bottom right of this table here, it's improving by a good amount. So it used to be if you 

did not finish service, about under half would just take the survey. But those numbers are going way 
Transcript_Technical Webinar Recording Session_MES Dataset_Final Page 2 of 6

higher, which is a really great trend to see. Whereas those who did finish service, 85-plus would be 

completing the survey.

So we kind of know now some ins and outs, sort of who takes it, right? But now we can take a look at 

the questions that are within the survey and sort of the theoretical background of those questions. A lot 

of the theoretical backing comes from the AmeriCorps member theory of change. And I sort of spoke to 

it earlier, but AmeriCorps sees that service creates empowered and prepared citizens who are more 

civically engaged, that will work to strengthen their community, and that are dedicated to improving the 

lives of those in their community. And we measure all of that along four pathways that we feel are 

essential to develop that empowerment and preparation to essentially become an active community 

member. You may be one before you join AmeriCorps, but you certainly will be one after you leave. 

Those four pathways are here, cultural competency, civic engagement, life and career skills, and getting 

things done.

So to go through each of those, let's start with cultural competency. So that is basically the member 

ability to work with and within diverse groups and to achieve a common goal with folks different from 

you. And a lot of that is rooted in contact hypothesis where, just to quickly read that, during 

interpersonal contact between majority and minority groups, given certain conditions are met, 

members can effectively overcome prejudices, biases, and preconceived notions of the minority group. 

And what makes that work is when there's sort of an equal footing amongst those groups and having a 

cooperative goal in mind, which is AmeriCorps to a T. Working on teams to achieve a common goal is 

the bread and butter of AmeriCorps in a lot of ways. So that's what cultural competency refers to. Then 

civic engagement is sort of baked into AmeriCorps just by default, right? Members who decide to 

dedicate their time to service is an act of civic engagement, but it goes a step further than that as well.

So we ask questions about, do you feel you'll vote more frequently? Do you feel you'll stay more 

attuned to the news and socio-political issues? Do you feel like you'll go to more meetings and things 

like that? So we define civic engagement as the constellation of activities in which individuals engage to 

make a difference in their communities. So AmeriCorps is providing that opportunity for just regular 

ordinary citizens to become these change agents, or as we call them, force multipliers, who get to work 

with local stakeholders to solve issues. So AmeriCorps does have an opportunity to really shape longterm civic attitudes and behaviors, whereas it may not be as prevalent to just somebody who didn't do 

AmeriCorps.

[inaudible 00:08:09] is life and career skills. So members have the opportunity to enhance their hard 

skills and soft skills through service, and work on a variety of different jobs, different activities, engage 

with different members of the community, that can really help prime them to become public servants, 

where other positions may not offer as wide variety of a role. Also in line with that is the educational 

attainment pathway, which is the fact that we give an educational award helps enhance members' 

ability to obtain the higher education, have more training and so on. So AmeriCorps does kind of want to 

create a more educated populace to tackle community issues. In theory, a more educated alumni base 

will be able to think a bit more creatively, more strategically, have the skills needed to actually make 

positive change.

Following that is getting things done. So getting things done is essentially having the grit and 

determination to get through adversity. It's creating what we call a change in action orientation to the 

world, which is just having a sense of self-efficacy, right? It's being able to take problems thrown your 

way, having that hands-on experience to be able to get through things that are in the "real world". And 

for a lot of members, service occurs during a critical period of life known as emerging adulthood. So the 

AmeriCorps member service, it offers exposure to new ideas and experiences when sort of that notion 

of self-identity and figuring out what you actually believe in is kind of prevalent. But AmeriCorps kind of 
Transcript_Technical Webinar Recording Session_MES Dataset_Final Page 3 of 6

helps answer a lot of those questions and instills that motivation to solve problems. And it promotes the 

ability to take initiative in things like planning, preparing, executing, and assessing an endeavor.

So there's additionally these five domains, which are just other questions that we'll ask that don't fit into 

those four pathways. And these are a bit more just asking questions to get to the bottom of what 

they're going to be doing with all of that, the member development. So why did you join, motivation to 

join? Can you describe your experience as far as did you do the following things during your service? Do 

you feel satisfied with your experience? Would you recommend it to a friend or family member? Was it

a defining experience for you? Training, do you feel as if the training you went through helped you 

prepare to take on all the responsibilities of your service here? And once you're done with AmeriCorps, 

what do you plan to do?

So we can take a look now at some of the analyses that we've completed, some analyses we plan to 

complete, as well as what you can do with these analyses. So ORE, as I said before, will take a look at 

data by fiscal year, and in our case that runs October 1st through September 30th. So members, we'll 

place them into a specific fiscal year according to their service end date. So here are a couple examples, 

but essentially, we want to take a look at end date just being it's an exit survey, right? So we want to 

take a look at when members are leaving at the same time to account for any kind of seasonality or any 

current events happening, such as a presidential election, right? We can see for fiscal year '20 and '21, 

there were some spikes in some civic behavior when the presidential election was happening, right?

And we'll disaggregate the data any which way you want to cut it, pretty much. But we'll typically take a 

look at demographics. So that's things like race, ethnicity, gender, age group, educational attainment. 

We'll take a look at things programmatically, such as which AmeriCorps program you're in. We can take 

a look at different focus areas, right? We could say members in the who serve in an environmental 

program feel this way versus those who serve in an educational program feel that way. We take a look 

at term type or even service completion. So if you really wanted to cut it down, it could be full-term 

service members who finished service, who were AmeriCorps state and national members, who served 

in education. Well, we can cut it down many, many ways to be super specific. And furthermore, we can 

kind of use some of the questions within the MES itself to categorize the members.

Specifically, we have one analysis, which we'll talk about in a little bit, where we take a look at if

members were serving, if they felt they were serving on a team more often than not. And we have 

another analysis coming up where we take a look at what a member's post-service plans are. If they 

come into it wanting to work in the private sector, does that affect how they feel about things, versus 

somebody who wants to maybe re-enlist for another service term, or somebody who wants to work in 

the public sector. There's a lot of ways we can cut this data. And our analyses, they come in a variety of 

ways, so full reports, one-pagers FAQ documents. You've got your graphics, your graphs, your 

appendices. They can be top level findings, they could be super drilled down specific findings. We can 

really kind of cover all of our bases depending on what we want to be looking at.

And something also that we'll do is incorporate other data sources into our analyses. Specifically, we 

take a look at national voting data that we got from Tufts University and that we compare to just 

AmeriCorps member voting behavior, which I'll show in a second. And then we also will take a look at 

the U.S. Census current population survey, as well as the civic engagement volunteering supplement, to 

sort of see how national civic behavior looks at AmeriCorps member civic behavior. So here are a couple 

completed analyses that will be on the evidence exchange. There's the Bridging Differences Analysis, 

which we just call it Bridging Differences, but it's the cultural competency pathway. So we took a look at 

data from 2016 through 2020, and we really drilled down and disaggregated by demographics, by 

program, and going back to that notion of did a members feel that they served on a team more often or 

not.
Transcript_Technical Webinar Recording Session_MES Dataset_Final Page 4 of 6

Then there's the civic engagement pathway that we took a look at between 2017 through 2021, and that 

was just disaggregated by program. So we took a look at just different civic behaviors, and more on that 

on a second. And again, these are on the evidence exchange. So in the Bridging Differences memo, this 

was a visual that was created from it. And essentially, all of our members, or at least nine out of tenplus, agree that they like people different from themselves, which is great to hear. There's respecting 

the values of people from different cultures and backgrounds. There's enjoying how the differences exist 

between each other. There's enjoying interacting with people from different cultures, just feeling 

confident when interacting with different cultures. And these are high numbers, right? These are across 

all members. So say there's 50,000-plus members in a year, 93 to 97% of them will exhibit that cultural 

competency that we like to see.

And here's just an example of some tables, or excuse me, some graphs that are in the report. So 

specifically, we wanted to look at whether or not a member that felt that they served on a team versus a 

member who felt that they didn't really serve on a team, if that explored any differences, right? It's sort 

of in line with the contact hypothesis where if you're serving on a team with folks different from you,

you will be able to exemplify a bit more cultural competency. And we saw that come through. It's still 

pretty decent numbers for those who didn't feel that they worked on a team all that often, but markedly 

different from those who did feel that they worked on a team. And so that's an example of the Bridging 

Differences memos of just, we have a visual, we have our team, our graphs, and so on.

And moving on to civic engagement, here are examples of some top level findings. So in this memo on 

civic engagement from 2017 through 2021, we have that there's increased community engagement. 

Members will report that they want to stay engaged in their community after serving in AmeriCorps. 

There's an increased frequency of discussing socio-political issues. Members feel that they'll talk about 

things a bit more, right? There's high levels of attachment and obligation to their community. Members 

feel more attached after having served than not. There is high levels of member confidence to 

adequately address community concerns. So members report, according to the survey, that they feel 

they can create plans, get others to care about the problem, organize meetings, express one's views in 

front of a group, and contact elected officials. This is a level of confidence that is built by survey, right? 

And furthermore, we can bring in sort of that outside data where we can take a look at how our 

AmeriCorps members stack up against national populations.

So taking a look at national youth voting rate, in this case, youth voting rate equals 18 to 29-year-olds, 

for 2016, the voting rate was 39%. For the 2020 election, it was about 50%. And you can see on the 

graph that the 2016 election is in 2017, the 2020 election is in 2021, and that's just because of the fiscal 

year, because our fiscal year starts October therein, the election is in that fiscal year and so on. Point is 

that no matter which way you cut it, our members are all voting in presidential elections at a higher rate 

than the national youth voting rate is. There's a bit decline you can see right there, and that's just an age 

related thing, where somebody who's 18 in 2019 couldn't have voted in the 2016 election and so on. So 

that's why you see the number go back up each election year and so on, just because AmeriCorps 

members kind of stay the same age.

And we have some forthcoming analyses that hopefully will be completed soon. And one of them is a 

life and career skills memo from 2017 through 2022, where we're dividing up members according to 

what they say their post-service plans are, right? So do people who want to pursue public sector jobs 

feel differently about how AmeriCorps prepared them for things versus those who want to do a private 

sector job and so on. We're going to take a look at responses to the survey according to if members 

completed their service or not. In some preliminary analyses we've done thinking like that, we can see 

that there is in fact a difference between those who finished service and those who did not finish 

service, in the favor of those who did complete service. But those who didn't complete service are still 

reporting fairly high outcomes on the survey, which is a good thing to see.
Transcript_Technical Webinar Recording Session_MES Dataset_Final Page 5 of 6

We're going to take a look at motivation to serve, right? Does that affect how you feel about things? 

Why you joined AmeriCorps? Was it because you wanted to do something to get you into grad school? 

Was it something because you just wanted to give back to your community? Do these things sort of 

affect what you get out of serving? And then there's always the opportunity to analyze the AmeriCorps 

member exit survey according to state by state.

And moreover, there's the open data platform, which MES data will be placed onto, rather, some top 

level findings and summary tables will be provided, as well as a data dictionary, FAQs, other resources to 

help anybody download the data and interpret it. So for an example, this is a table for question seven, 

which is the cultural competency question. And you can see that there's different years on the table. 

There's the questions, there's the responses. And in this case, just as an example, question seven, please 

indicate how much you agree or disagree with the following statements, 7A, I enjoy exploring 

differences between coworkers and/or friends from different cultures or backgrounds than me. And we 

add together the people who say I strongly agree with this statement or agree with this statement, and 

that's where you get those percentages, or proportions, rather, that we could show year by year, 

program by program, individual programs, all the programs together on that top line and so on.

And to get to the open data platform, that's through data.americorps.gov, which will bring you to this 

site here, and you click on the data catalog. So the MES data is forthcoming on that and will be there 

soon. So who can use the AmeriCorps member exit survey data? Frankly, everybody, but AmeriCorps 

staff in the agency, they can use the data. State service commissions, people can use that data, as well 

as policymakers who just want to know about the effect of AmeriCorps service on its members. And this 

survey is beneficial to knowing, really, just to getting to the bottom of how members feel about their 

service. Are things going right? Are things not going right? Can members take what they've learned and 

what they've gathered from having served and take that with them as they move on to just the rest of 

their lives?

So to conclude, AmeriCorps member exit survey is a wealth of information about AmeriCorps members. 

So we have over 350,000 responses that can be disaggregated by demographics, by program, over time, 

and even more, based on questions specifically and so on. So it really gets to the bottom of how 

members feel over time. And if we go back far enough, 2016, 2017, that's before certain world events, 

where we can kind of see how members felt about things. It's almost in some ways splitting that 

generational divide for a lot of our members between Gen Z versus millennials, so on. We can really 

track differences over time, which is beneficial to just analyzing how AmeriCorps is doing when it comes 

to member development. So it really does follow that theory of change to a T, where it's just getting 

down to do members feel that they were made better themselves as they made their communities 

better?

So furthermore, just the MES, it can be used to improve the service experience also from a more 

programmatic top level view. So stakeholders can use the MES findings to inform future policies and 

programming. If, for example, we were to see that all of the members, not all of the members, but a 

large portion of the members feel that they are not satisfied with a certain facet of the service that they 

all wrote on their survey, then that's something of concern that should be addressed and it will be 

addressed. And furthermore, it's just sort of an opportunity for members to be heard and to make 

things better for the next group of members, which is thoroughly in line with the AmeriCorps ethos of 

making places better, as well as just furthering this opportunity for those that come after them.

And to note, there is a new version of this survey coming down the pike, and it's not even necessarily a 

new version of the survey rather than it is a new survey itself altogether. We're looking to do a pre-post 

participant survey, where we'll be able to ask questions before members really get serving and can then 

follow up with the exit survey that we normally do, where we can actually test in each member if 
Transcript_Technical Webinar Recording Session_MES Dataset_Final Page 6 of 6

AmeriCorps is doing what it says it's doing, right? We know that right now, we just have the post, but 

having the pre, we'll be able to really test some of these hypotheses. And furthermore, it will be 

inclusive of AmeriCorps seniors volunteers, whereas the MES is just for AmeriCorps members.

So for more information, you can contact the AmeriCorps Office of Research and Evaluation at 

AmeriCorpsMES@CNS.gov, where you can ask us any questions about any of the questions that we ask, 

any of our data, and so on. And we look forward to you using our open data platform and accessing this 

treasure trove of data that we have on our members. Thanks.