Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

1 

ANDREA ROBLES: Well, thanks so much for joining us 

for our research evidence webinar. My name is Andrea 

Robles, and I work for the Office of Research and 

Evaluation. Our office objectives are to build 

knowledge on civic engagement, volunteering, and 

national service by funding research and supporting 

our programs and their grantees as they build 

evidence through evaluations. We strive to share and 

use our research findings and do so in several ways, 

including posting studies and reports on our webpage, 

circulating a quarterly newsletter, and offering a So 

What series, essentially a type of workshop where we 

translate research for CNCS staff. 

And for the last two years, we have held a research 

summit in December that is attended by academics, 

researchers, CNCS staff, and other federal agency 

staff, foundations, and practitioners. We are hoping 

to have the research summit in February or March of 

2018. This webinar series is another way for — to 

share our cutting-edge research that we are 

conducting both internally and externally. We want 

our research to spark ideas and assist in developing 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

2 

and improving programs that can ultimately improve 

lives and communities. 

We understand that research needs to be understood, 

and — in order for it to be used. In these webinars, 

we hope to be able to appeal to various users, 

including scholars and practitioners. Although for 

us, it's exciting to have a variety of audiences 

interested in this research, we understand that there 

is a — different styles of communication between the 

two groups. So we are asking our speaker to present 

somewhere in the middle, which is a difficult task. 

But we have both a paper that is accompanying this 

webinar, if you'd like some more details, and you 

could also just reach out to us if there's other 

questions that you may have. 

So today, we are pleased to welcome Professor Pamela 

Paxton and Kristopher Velasco from University of 

Texas Austin Population Research Center. Pamela 

Paxton is a professor of sociology and public affairs 

and the Christine and Stanley E. Adams Jr. Centennial 

Professor in the Liberal Arts at the University of 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

3 

Texas at Austin. She is the author of articles and 

books on social capital, women in politics, and 

quantitative methodology. Her research has appeared 

in a variety of academic journals, and most recently, 

she is co-author of the 2016 book Women, Politics, 

and Power: The Global Perspective. She has also 

consulted for the US Agency for International 

Development and the National Academies. 

Kristopher Velasco is a graduate student who has just 

completed his masters at the University of Texas. It 

considers how international, non-governmental 

organization influenced the adoption of LGBT policies 

around the world, so I will now turn it to Pamela and 

Kristopher. Thanks for doing this. 

PAMELA PAXTON: And thank you for giving us the 

opportunity to reach a wider audience with this 

research. So I'm Pam Paxton at the University of 

Texas at Austin, and I want to speak today about our 

research investigating whether national service 

programs improve community well-being. Next slide. So 

we are interested in trying to understand whether 

national service programs, like AmeriCorps, have an 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

4 

impact, and the issue is that we know some things 

about the impact of national service programs but not 

everything.

So at present, we have evidence that AmeriCorps

members help nonprofits at an organizational level, 

and they do this through service provision, so we 

have lots of evidence, for example, that AmeriCorps

members are serving children through food programs. 

They are operating reading camps for children, so 

nonprofits are recording the service that AmeriCorps

members are doing and demonstrating how these members

help nonprofits themselves, right, at the 

organizational level. We also have evidence that 

AmeriCorps members benefit themselves over their life 

course as a result of their service. 

So there are studies showing, for example, that 

AmeriCorps members show increased levels of civic 

engagement later in life, such as their rates of 

voting or volunteering. So we currently know that 

nonprofits help — I'm sorry, that AmeriCorps members 

help the nonprofits in which they're located, and 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

5 

that AmeriCorps members help themselves through their 

participation in AmeriCorps. Next slide. 

 

What we don't know at present is whether national 

service programs have an impact beyond the individual 

or beyond the organization and also affect the 

community, and this is important because one of the 

purposes of the 1993 legislation was to have 

AmeriCorps influence communities. So here, on this 

slide, we produce some of the legislation. It says, 

"It is the purpose of this act to renew the ethic of 

civic responsibility and the spirit of community to 

provide structured service opportunities with visible 

benefits to the participants and community and to 

provide tangible benefits to the communities in which 

national service is performed." 

So this was one of the goals of the legislation and 

of AmeriCorps, but we don't, at present — or we 

didn't before this research have evidence that 

AmeriCorps does have an impact on the community. So 

our research question was do national service 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

6 

programs improve community well-being? That's what 

we're trying to address in this paper. Next slide. 

So we are interested, again, in understanding whether 

national service programs influence community wellbeing, so what I'm gonna do now is walk through our 

research design to give you a sense of how can we 

answer this question. So there's a number of things 

we need to do to answer the question. First, we need 

to define community, so in this research we define 

community as county, and specifically we're looking 

at 1,347 counties. Why do we consider county? Well, 

we felt, theoretically, that the effective of 

national service program may not be limited to a 

specific neighborhood or zip code. You can imagine 

individuals may travel across a zip code or a city, 

right? To receive services from an organization. 

So to go down to a lower level than the county, I 

think, would not really capture what we mean by 

community in this context. We also wanted a level 

that is comparable, level of analysis that is 

comparable across diverse communities, including 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

7

those in both rural and urban areas, so we felt 

county best exemplified that. And, of course, data 

availability played a role as well in that we have 

better data available to us at the county level. So 

for us, community is county going forward. Next 

slide.

So the next thing we need to consider when we're 

trying to answer this question, how do — or do 

national service programs impact the community, is 

how to measure national service, so we're looking at 

AmeriCorps programs, and we are looking at the 

presence of AmeriCorps programs. We have other 

measures, such as the number of programs in a 

community or the number of members in a community, 

and our results are similar, but the simplest measure 

here would be just the presence of AmeriCorps

programs in a community over time, and we split, 

also, AmeriCorps into some sub-programs. So we 

consider, overall, AmeriCorps programming, and we 

also consider sub-programs, AmeriCorps State, 

AmeriCorps National, AmeriCorps VISTA, and some other 

AmeriCorps programs that are — tend to be small with 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

8 

unique goals, so we don't — we split them out, but we 

don't talk about them much. So before going on, I 

just wanna note State and National are somewhat 

similar programs in how they use — how non-profits 

use members. 

VISTA is a little different in that it focuses 

specifically on capacity-building and community 

empowerment in high-poverty areas. So VISTA has a 

unique mission, and it is definitely worth splitting 

that out from State and National so we can understand 

whether these sub-programs have different effects on 

communities compared to, say, overall AmeriCorps

programming. Like I said, we have these auxiliary 

analysis as well that consider the number of programs 

and the number of members. Next slide. 

The next thing we need to do when we're asking how 

can — you know, when we're doing our research design, 

how can we answer this question, is to define 

community well-being, right? So we need some measure 

of community well-being. So we use the concept of 

subjective well-being. There's increasing interest in 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

9 

the concept of subjective well-being. For example, 

it's highlighted in some OACD documents, in the 

Healthy People Initiative from the CDC, and you may 

have heard of the idea of gross domestic happiness, 

which some are promoting as an analog to gross 

domestic product to think about how individual 

citizens in nations, how much subject well-being they 

have. 

So subjective well-being is typically measured via 

surveys where people are asked questions about their 

happiness, their life satisfaction, a range of 

questions, and to get something about a community or 

nation's subjective well-being, those survey 

questions are then aggregated to that level. Next 

slide.

So we do something different. Our data source for 

subjective well-being is Twitter, so this actually 

makes this a novel measure, and so what's good about 

this — there's several nice things about Twitter. 

First, we can capture multiple dimensions of 

subjective well-being. So by looking at tweeted words 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

10

in a community, we can assess that community's 

engagement, disengagement, positive relations, 

negative relations, positive emotions, and negative 

emotions. 

So what we do is there are existing dictionaries, 

psychological dictionaries of words that are 

associated with these dimensions of subjective wellbeing, so words, for example, that indicate 

engagement, words that indicate disengagement. So we 

can take disengagement words, for example, that are 

being tweeted in a community and divide that by the 

total words tweeted in a county to get a proportion 

of disengagement words over the total words tweeted. 

So that is how we're measuring, for example, 

disengagement. The Twitter and other social media 

outlets are increasingly used in a range of fields to 

give information about what's called ambient 

geospatial information, so it gives a sense of what's 

happening in communities.

Subjective well-beings in prior research have been 

related to self-reported life satisfaction from 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

11

surveys. Remember, that's the traditional way that 

subjective well-being is measured, so research has 

shown that that traditional measure is related to the 

subjective well-being tweets in communities. Twitter 

data has predicted election outcomes. It predicts the 

path of fires and tornados, so there's a lot of 

information we can get about communities and this 

idea of ambient geospatial information from Twitter. 

So we're using our measure, like I said, of one of 

these dictionaries, for example, disengagement over 

total words tweeted. Next slide. 

So to give you a sense of what we mean by this, this 

slide shows the most common words within two of the 

subjective well-being dictionaries. So for the rest 

of this talk, the paper encompasses positive and 

negative relations and positive and negative 

emotions, but for the rest of this talk we're going 

to focus just on engagement and disengagement because 

those are of particular interest to civic engagement 

outcomes and communities. So this slide gives you the 

most common words within two of the subjective well-
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

12

being dictionaries, engagement and disengagement, and 

you can see that it has face validity, right? 

So the words that are often tweeted that are used to 

indicate — or that indicate disengagement are coded 

as — ah, sorry, coded as engagement are learning and 

interesting and alert and creative and involved, 

right? So these really indicate engagement. There's 

good face validity there. Disengagement, also, nice 

face validity. Tired, bored, blah, and meh, right? 

These are wonderful — I love that we can use meh, 

right? So disengagement, again, has this kind of nice 

face validity that the words people are tweeting that 

are being coded as disengagement makes sense. All 

right? So this gives you some sense of our measure of 

subjective well-being then in communities. Next 

slide.

So let's go back, right? We're trying to connect 

AmeriCorps programming to subjective well-being. We 

are trying to think about whether national service 

programming affects community well-being, so for us, 

that's AmeriCorps programming and subjective well-
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

13

being and communities. So this slide shows that basic 

association that we're trying to assess, so the arrow 

indicates direction of influence. So — and I should 

say that AmeriCorps programming, we're looking at 

this over time to start, from 2005 to 2010, so a 

community can have zero years of AmeriCorps or six 

years, up to six years.

So what we first do is a series of linear 

regressions. What linear regressions allow us to do 

is just show association between AmeriCorps and 

subjective well-being in communities, so just 

association. It allows for a very broad overview of 

these many types of AmeriCorps programming, and 

remember, we're going to be thinking about AmeriCorps

and State and National and VISTA separately, as well

as the several dependent variables that we're looking 

at for subjective well-being, all these different 

types of subjective well-being, like engagement, 

disengagement, and positive emotions, so we're — we 

can do a lot of linear regressions to look for these 

associations.
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

14

So when — I'm going to get into these results a 

little bit later, but we did find significant 

effects. We did significant effects suggesting that 

AmeriCorps is associated with subjective well-being. 

So what we need to do next, however, is defend the 

idea that there's an association here against 

arguments that might undermine our conclusion. Next 

slide.

So a way of thinking about this, okay, in terms of 

research design is asking the question, "What could 

be wrong about this result?" Right? So we found a 

significant association between AmeriCorps

programming and community subjective well-being. 

Well, what could be wrong with those results? So 

there's a couple things to think about. The first is, 

well, what if counties with AmeriCorps programs are 

different than those that do not receive such 

programs? In that case, the — what we're finding 

here, what we see in terms of association could be 

due to differences in — across the counties that are 

receiving or not receiving AmeriCorps programming 

rather than actually an association. 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

15

The second issue that we need to think about, or the 

challenge with the results, is what if it's the 

opposite? What if counties that have greater levels 

of subjective well-being are better able to attract 

and sustain AmeriCorps programming? And this is 

absolutely a reasonable question. If you think 

especially about what we're gonna be focusing on here 

between — where we're looking at an engagement and 

disengagement, it may be that more engaged 

communities are just better able to attract 

AmeriCorps and sustain AmeriCorps, as I said, or 

disengaged communities have difficulty attracting 

AmeriCorps.

So if that were true then that association that we 

see from the regression analysis may not be that 

AmeriCorps is affecting its objective well-being, but 

in fact the opposite. So we need to address both of 

these threats to our results. Next slide. 

So how do we address these concerns? Well, the first 

thing that we do is to consider a range of control 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

16

variables, so looking at other features of counties 

that might make a difference both to their ability to 

get AmeriCorps programming and to their subjective 

well-being. So here that we show that we're including 

income and measures of racial and ethnic diversity, 

education level, rurality, age, and also state-fixed 

effects, which allow us to consider any unmeasured 

features of states that would affect the counties and 

AmeriCorps programming in those states. 

The next thing that we do to address these problems 

is to be — to move to longitudinal models, meaning we 

add in a second wave of data. So we have two waves of 

data, one where we're thinking — where we look 20 —

sorry, 2006 to 2009, and a second wave that's 2010 to 

2013, and this allows us to do — having that second 

wave of data is really important. It allows us to do 

several things. The first thing that we do is 

consider AmeriCorps as an intervention or a 

treatment, if you will, into a county. So in 2009, 

there were 250 counties that did not have any 

AmeriCorps programming, and over time 72 counties 

received at least 1 AmeriCorps program.
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

17

So we can look at — we can estimate the change in 

subjective well-being in counties that got an 

AmeriCorps program over the time period, and those 

that don't get — that did not get an AmeriCorps

program over the time period, and that allows us to 

think about AmeriCorps as, like, a treatment, as I 

said, or an intervention into that county and see if 

they have an effect, it has an effect on subjective 

well-being. The second thing that the second wave of 

data allows us to do is to consider these reverse 

effects that I mentioned, these reciprocal effects. 

So the picture here on the slide indicates what I'm 

talking about, so what we have here now are two 

waves, so we have AmeriCorps and time one, so we 

subscript it with one, and we have AmeriCorps and 

time two, subscript it with two. 

Similarly, we have our measure of subjective wellbeing in time one and time two. So the arrows 

indicating direction of influence, we see the center 

there, the middle two, are what we call cross-lagged 

paths. So AmeriCorps and time one is affecting 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

18

subjective well-being in time two, but subjective 

well-being in time one is also affecting AmeriCorps

in time two, so we're acknowledging that reciprocal 

effect that I talked about a minute ago that 

disengaged communities, for example, may just not 

have the personal — the personnel or financial 

capacity to obtain and sustain AmeriCorps

programming. 

So we're gonna estimate that reverse effect. There 

are also what we call stability effects in here as 

well. We acknowledge that community subject wellbeing at time one is likely related to their 

subjective well-being at time two, similarly with 

AmeriCorps. We would expect some stability effects 

there. So that's how we can use longitudinal 

modeling, modeling over time, to try to get at these 

threats that I spoke about earlier to a conclusion 

that AmeriCorps affects subjective well-being. Next 

slide.

All right, let's turn to the results now. So these 

are our results from the linear regression, so each 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

19

one of these cells indicates a result from a linear 

regression. So the first thing to notice here are 

those red negative signs indicating that AmeriCorps

generally seems to operate by buffering negative 

aspects of subjective well-being, and here are — like 

I said, we're only looking at disengagement, but what 

this is saying — just think about overall 

programming. So more AmeriCorps programs in a 

community reduces disengagement, right? It's reducing 

disengagement. So we see that as buffering, like I 

said, buffering or mitigating the negative aspects of 

subjective well-being, all right? 

And we see that effect across all the categories, 

sub-categories of AmeriCorps, National, State, VISTA, 

and these other kind of small and specialized 

programs. We do see two positive coefficients, 

positive for engagement, so we do see overall 

programs effecting engagement positively and looking 

at the subtypes we see that that's largely due to the 

effect of AmeriCorps VISTA, so VISTA is having a 

positive effect on engagement. However, we do need to 

acknowledge something about this finding, so if VISTA
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

20

members coming into a nonprofit — remember, they're 

doing capacity building for nonprofits. Well, what if 

capacity building involves, as one aspect, increasing 

a social media presence for the nonprofit? 

And what if increasing that social media presence 

includes starting more tweets? So it is not possible 

for us to tell whether this positive effect of VISTA

on engagement is due to an effect of VISTA on 

engagement or is due to some — just that the VISTA

members are actually starting to tweet, and if they 

were tweeting it would be kind of engagement words, 

right? Like involve or interesting. Now let me point 

out that this issue does not occur — is not a threat 

to the disengagement negative effects, right? Because 

that's a buffering. It's a decrease in words tweeted 

rather than an increase in words tweeted. Next slide. 

So now turning to the models where we look at things 

over time, I wanna point out just — the first thing 

to say is that the intervention, thinking about 

AmeriCorps as an intervention is significant. So 

remember, we're thinking about counties that received 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

21

AmeriCorps programming that had not had AmeriCorps

programming before, right? And so counties that 

received AmeriCorps programs as a treatment on 

average had 105 fewer disengagement words tweeted. 

Okay? And 61 more engagement words tweeted compared 

to counties that continue to have no presence of 

AmeriCorps. So let me say that again about the 

disengagement, so if compared to counties that 

continue to have no AmeriCorps programming, those 

that did receive AmeriCorps on average had 105 fewer 

disengagement words tweeted. 

Is that a large or small effect? Well, I can tell you 

that the median number of disengagement words tweeted 

across counties is 247. Also when we consider the 

cross-lagged panel model that is pictured here, we've 

bolded those cross-lagged arrows to indicate that we 

do see both these effects, right? So when we account 

for the possibility of a reciprocal effect from 

subjective well-being to AmeriCorps programming, the 

effect we're looking for, this AmeriCorps affecting 

subjecting well-being, is still significant, but we 

do see the reserve effect as well, so we do see a 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

22

significant effect of a county's pre-existing levels 

of disengagement influencing the level of AmeriCorps

programming it will receive. 

So what we see in these cross-lagged panels, to 

reiterate, is both effects, AmeriCorps affecting 

subjective well-being and the reverse, subjective 

well-being affecting AmeriCorps. Next slide.

So what are the takeaways from this research? I want 

to remind you that one of the purposes of the act 

Initiating AmeriCorps, was to provide visible 

benefits to communities, tangible benefits to 

communities. So this research, one takeaway is that 

AmeriCorps programs do help improve community 

subjective well-being. They do so largely through 

buffering against negative aspects of subjective 

well-being. So one conclusion to take away from this 

research is that as AmeriCorps is fulfilling its 

stated goals, it should be funded into the future. 

Next slide.
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

23

A couple of other takeaways, we did find some 

differential effects across AmeriCorps programming, 

slightly stronger effects for National programs, for 

example, compared to State programs, and this 

interesting VISTA positive effect that we cannot, at 

this time, attribute entirely to VISTA programming. 

We find that the results that we see hold up to 

various threats against the inference. So moving to 

those longitudinal models allowed us to attempt to 

address threats to our conclusion. So what we find is 

— the final takeaway is that the reverse effect that 

we find. 

That subjective well-being of communities, 

specifically disengagement, can influence whether 

communities are getting, acquiring, and sustaining 

AmeriCorps programming, suggests that when 

considering applications, CNCS should pay particular 

attention to quiet or atypical communities, right? 

And we see this in a few places. The Texas One Star 

Foundation had a program to try to help communities 

that had not had AmeriCorps programming obtain 

AmeriCorps programming, so there's ways that CNCS can 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

24

pay more attention to quiet or atypical communities 

knowing that these are first effects take place. 

Well, let me just end by thanking the Corporation for 

National and Community Service, NIH, and the Center 

for Open Science, which allowed us to find and work 

with the University of Pennsylvania, which developed 

the Twitter data in the first place. 

ANDREA ROBLES: So Pam and Kris, thank you so much. 

This is fascinating work, and as you know, there's a 

lot of interest in this, and you are putting out a 

paper with this — these findings. Can you tell us 

where we might be able to find that, or other 

information that might be in the paper? 

PAMELA PAXTON: Well, so we have the paper. It's 

currently under review, so if people are interested 

in the paper, they are welcome to contact myself at 

the University of Texas or Kris Velasco at the 

University of Texas and we can send them the paper. 

I've also given the paper to research and evaluation 

at CNCS, so that's another way to get it, and of 
Can National Service Programs Improve Community Well-Being?

UT Austin Webinar Transcript 

25

course we are pursuing publication at this time, so 

it will hopefully become publicly available through 

publication outlets soon. 

ANDREA ROBLES: Okay. Well, thank you so much for 

your time and for all your work. It's great. 

PAMELA PAXTON: Thank you, my pleasure. 

 

END OF FILE