Assessing the

Volunteer Infrastructure Program 

using the Volunteer Capacity 

Assessment

November 12, 2018

Submitted to:

Research & Evaluation Manager | Community Programs

Napa County Office of Education Community Programs 

5789 State Farm Dr., Suite 230, Rohnert Park, CA 94928

Submitted by:

JBS International, Inc.

555 Airport Blvd, Suite 400

Burlingame, CA 94010-2002
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

i

Table of Contents

Contents 

Table of Contents...........................................................................................................................................i

Introduction...................................................................................................................................................1

Methods ........................................................................................................................................................1

Sample ......................................................................................................................................................2

Statistical Approach..................................................................................................................................2

Results...........................................................................................................................................................2

Figure 1: Overall Growth by Scale .......................................................................................................3

Table 1: Overall Growth (means) .........................................................................................................3

Table 2: Change Scores by Scale..........................................................................................................4

Figure 2: Change Scores by Scale.........................................................................................................4

Conclusions...................................................................................................................................................4

Appendix A...................................................................................................................................................6
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

1

Introduction 

The Napa County Office of Education’s (NCOE) CalSERVES AmeriCorps Volunteer 

Infrastructure Programs (VIP and VIP-SE) support AmeriCorps members who work with 140 

community-based organizations in building volunteer infrastructure and capacity. AmeriCorps 

members work as teams to provide infrastructure support for volunteer programs in service 

organizations across California. The aim is to build sustainable volunteer management systems at 

partner sites.1 The program began in 2010 and has supported service organizations for varying 

numbers of years, some have been enrolled for one year and others for multiple years. 

In this report, JBS International provides a summary of analyses conducted using the 

CalSERVES AmeriCorps Volunteer Capacity Assessment (VCA) and its three scales.2 The data 

that NCOE collected from partner sites using the VCA, both pre and post participation in VIP,

were utilized to assess the following research question: 

How did the rate of growth in organizational capacity growth differ comparing sites that have had one 

year of participation to those that have had more?

Methods 

CalSERVES staff collected data used for this analysis with Version 3 of the VCA; only 

organizations using Version 3 for pre- and post-VIP assessments were included in this analysis. 

Administration of the VCA began when the community-based organizations (program sites)

were enrolled in VIP or VIP-SE (2010-2018) and at a single follow-up point in Summer 2018. 

Requests for follow-up data were sent to all past and current VIP participants. The information 

used for the analyses included:

 Site identification

 Site status (current or past participant)

 Date of program entry

 Date of program exit

 VCA preservice and post participation scores for all three scales and overall

Data were collected on three scales of the VCA: Volunteer Plan Development (16 individual 

items), Volunteer Plan Implementation (16 items), and Volunteer Plan Sustainability (15 items). 

The VCA asked program sites to self-report their progress, using the same VCA instrument, at 

two periods in time, pre- VIP participation and during a 2018 follow up. Program sites indicated 

their achievement on each item by responding ‘No’, ‘Somewhat’, or ‘Yes’.

1 Source: https://serviceyear.org/calservesamericorps/

2

JBS tested the validity and reliability of the survey in 2016. The results showed that the VCA has good test-retest 

reliability, internal consistency, and external validity.
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

2

Sample 

Of the programs served by VIP and which had a pre-test VCA (220), 55 responded to and

completed the 2018 post assessment. The sample of 55 programs comprised 37 programs sites 

that had completed only one year of VIP participation (one-year programs), including both 

current participants and participants that completed VIP during any single year since 2015. The 

sample also included 18 programs sites that have participated in VIP for more than one year 

(multi-year programs). Those that participated for more than one year were mostly two-year 

participants. In fact, the sample contained 16 two-year programs and only two programs which 

had participated for three or more years.3

Statistical Approach 

Using data from 55 programs, JBS conducted a descriptive analysis and assessed to what extent 

program sites increased achievement levels after enrollment in VIP or VIP-SE. JBS also 

examined how much programs enrolled in VIP for more than one year improved in comparison 

to those that had completed only one year in the program. Due to the small sample size, it was 

not possible to impute any missing data, therefore, researchers only included programs with 

complete follow-up data. 

JBS cleaned and analyzed the data using SPSS, a statistical software program (Version 22). The 

cleaning steps included converting missing responses to ‘No’, checking for duplicate cases, and 

finding any out-of-range responses. Researchers then divided program sites into one-year and 

multi-year variables to conduct the analysis. The analysis examined pre and post means for each 

item on the VCA and for each of the three scales as a whole. Finally, the two groups (one-year 

and multi-year) were compared using change scores, tabulations which measured the difference 

in the mean from baseline to final.

Results 

Results of the descriptive analysis show a statistically significant change (or, ‘growth’) from 

baseline VCA scores to the summer 2018 VCA follow up scores for program sites overall. The 

average mean for each of the three scales improved over time, indicating a higher average 

achievement score among all program sites after at least a year of participation in VIP. Program 

sites chose more ‘Somewhat’ or ‘Yes’ responses on a majority of the items in the VCA during 

the post-participation survey in Summer 2018. Generally, VIP participants made the most 

improvement on Scale 2, the Volunteer Plan Implementation, followed by Scale 3, Volunteer 

Plan Sustainability, and then Scale 1, Volunteer Plan Development. All of these changes showed 

statistical significance at the p<.00 level.

4 Figure 1 demonstrates the overall change in mean on 

each scale for the total sample (N=55).

 

3 One of the three-year programs enrolled during the 2014-2015 year, making it a four-year program, but since it 

participated in the VCA Version 3, it is included in this analysis.

4 This was assessed using dependent means t-tests, which can assess whether mean values on a continuous variable 

are statistically different from one another at two time points.
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

3

Figure 1: Overall Growth by Scale

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Pre Post

Overall Growth

Scale 1

Scale 2

Scale 3

Note: All VIP programs (N=55); Change in all scales was significant at .00 level, per dependent means t-tests. 

When divided into two groups, one-year and multi-year, both groups’ means independently 

exhibited statistically significant improvement from pre to post on all three scales. Table 1 

demonstrates the changes from pre to post VIP participation means for one-year and multi-year 

programs as well as the total for all programs combined. 

Table 1: Overall Growth (means)

Years Enrolled

Scale 1: Volunteer Plan 

Development

Scale 2: Volunteer Plan 

Implementation

Scale 3 Volunteer Plan 

Sustainability

Pre Post Pre Post Pre Post

One Year

N=37

.95 1.64 .57 1.44 .51 1.30

Multiple Years

N=18

1.10 1.80 .69 1.62 .60 1.34

Total

N=55

1.00 1.69 .61 1.50 .54 1.32

After assessing the level of change in each group independently, researchers examined whether

the mean scale change significantly differed for one-year programs and multi-year programs. 
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

4

The analysis revealed that the improvement in achievement scores from pre to post appeared 

quite similar for both groups across all three scales. Researchers tabulated the change score, or 

average change in means by group, and the difference between the average change scores did not 

exhibit statistical significance. Of the scales, Scale 2 displayed the largest difference between the 

groups, with the multi-year group demonstrating a higher degree of improvement than the oneyear group. Table 2 and Figure 2 show the average change in scale scores by group. 

Table 2: Change Scores by Scale

Years Enrolled

Scale 1: Volunteer 

Plan Development

Scale 2: Volunteer 

Plan Implementation

Scale 3: Volunteer 

Plan Sustainability

One year 

N=37

.69 .87 .79

Multiple Years 

N=18

.70 .93 .75

Total

N=55

.69 .89 .78

Figure 2: Change Scores by Scale

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Scale 1 Scale 2 Scale 3

Change Scores by Scale

One year N=37 Multiple Years N=18

Conclusions 

Overall, pre to post VCA results show that program sites that participate in VIP for one year or 

more show substantial improvement in volunteer plan development, implementation, and 

sustainability. See Appendix A for the mean change for each indicator on the VCA. 
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

5

Though the analysis results did not reach statistical significance (i.e., the findings were ‘null’), 

the observed differences in ‘growth’ means was concordant with researchers’ expectations. First, 

Scale 1 means are perhaps less likely to rise more with additional years of participation, since 

volunteer plan development mostly takes place during the first year of enrollment. Results 

indicated that ‘growth’ means for Scale 2 were greater than either of the other scales, which 

could be due to the additional time to execute implementation activities. 

Scale 3, which is focused on sustainability, showed a slightly higher gain for one-year 

participation versus multi-year participation. Though this result was unexpected, the lack of 

programs in the sample with over two years of participation in VIP could explain the stall in 

multi-year programs in achieving additional volunteer plan sustainability. Only 18 programs sites 

had more than one year of participation, and 89 percent of those had just two-years of 

participation. Most two-year programs have not had time for the long-term planning that 

sustainability requires. Further, some of the one-year participants (N=37) may have had some of 

the sustainability infrastructure in place already or were in a position to implement these 

measures more easily during the first year of participation. A larger sample of participants with 

three years or more of implementation is needed to draw conclusions about improvements in 

volunteer plan sustainability. 

The promising improvement in overall ratings among programs sites demonstrates positive gains 

for VIP participants, which is consistent with the past quasi-experimental study of the program 

impact.5 The small group sample sizes and low response rate for the current study limits 

generalizability and potential reliability of results that could show differences in one-year versus 

multi-year programs. The upward trend resulting from VIP involvement could be further 

validated with higher VCA participation rates, especially among long-running VIP enrollees. 

This could be achieved through more active follow-up with non-respondents and emphasis on 

the benefits of the program evaluation for future VIP programs.

 

5 See https://www.nationalservice.gov/impact-our-nation/evidence-exchange/AmeriCorps-VIP for the 2012 

AmeriCorps VIP 2012 Volunteer Capacity Study.
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

6

Appendix A 

VCA indicators and means, baseline to final. 

Scale 1: Volunteer Plan Development 

Indicator

Mean: 

Baseline

Mean: 

Final

1a Benefits and challenges related to volunteer involvement inside the 

organization have been identified.

1.35 1.89

1b

Organizational leadership has defined volunteer engagement as a priority.

1.47 1.93

1c There is a plan in place to dedicate appropriate resources (e.g., financial, 

space, training, supervision, etc.) to the organization’s volunteer program.

1.25 1.76

1d A recruitment plan for volunteers has been created, including marketing 

to community organizations (e.g., outside organizations, businesses, 

service clubs) and diverse populations.

.67 1.65

1e Potential community partnerships for recruitment have been identified 

(e.g., outside organizations, businesses, service clubs).

.91 1.80

1f Human resources and risk management policies that relate specifically to 

volunteers have been developed.

1.13 1.67

1g A volunteer waiver form that addresses potential risks is signed by all 

volunteers.

1.20 1.67

1h

Needs for volunteer roles throughout the organization have been 

identified, including opportunities for skilled volunteers or volunteer 

leaders.

1.04 1.82

1i Volunteer position descriptions that include performance goals or 

expectations have been developed.

.69 1.62

1j

A volunteer application form and application process is in place (e.g., 

system for distributing, collecting, and reviewing applications and 

conducting interviews as appropriate). 

1.16 1.76

1k An interviewing and screening process, including appropriate criminal 

and/or motor vehicle background checks when applicable, has been 

developed and implemented.

1.18 1.60

1l Volunteer orientation trainings and materials have been created and 

implemented (e.g., handbook, emergency procedures plan, etc.)

.80 1.53

1m A supervisor has been identified for every volunteer role. 1.20 1.76
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

7

1n A volunteer retention and recognition plan that includes formal (e.g., 

celebration, gifts) and informal (e.g., organizational culture of 

appreciation) recognition has been developed.

.53 1.55

1o

A system to receive volunteer feedback has been developed.

.45 1.29

1p Volunteer tracking mechanisms (e.g., tracking of volunteer hours, tasks 

accomplished, etc.) are in place.

.93 1.76

Scale 2: Volunteer Plan Implementation

Indicator

Mean: 

Baseline

Mean: 

Final

2a A purpose statement has been developed for the organization’s volunteer 

program.

.45 1.33

2b Resources (financial, space, training, supervision, etc.) have been 

allocated for a volunteer management program.

1.04 1.58

2c

A plan has been created to incorporate community organizations (e.g., 

outside organizations, businesses, service clubs) in regular volunteer 

activities.

.56 1.51

2d

Strategies to develop potential partnerships with community 

organizations (e.g., outside organizations, businesses, service clubs) have 

been identified.

.67 1.65

2e

A volunteer recruitment plan has been implemented.

.60 1.71

2f

All volunteers and staff have been trained in appropriate human 

resources and risk management policies that relate specifically to 

volunteers.

.44 1.18

2g Diverse volunteer roles that include a wide range of abilities, ages, and 

interests have been established.

.85 1.65

2h Volunteer roles have been documented and communicated to 

appropriate staff.

.82 1.67

2i A system to provide feedback and evaluation on the work of individual 

volunteers has been developed.

.38 1.18

2j A responsive communications process is in place so volunteers are 

contacted within a reasonable time after they apply.

.95 1.80
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

8

2k An ongoing volunteer training plan and materials have been developed. .56 1.35

2l

Volunteer supervision responsibilities are included in staff and higherlevel volunteer position descriptions (e.g., providing support, 

opportunities for communication, accountability).

.67 1.42

2m Strategies (e.g., events) are in place to build relationships between staff 

and volunteers.

.53 1.55

2n

A system is implemented for volunteers to receive formal (e.g., 

celebration, gifts) and informal (e.g., organizational culture of 

appreciation) recognition of their contributions.

.49 1.47

2o

A system to receive volunteer feedback has been implemented

.27 1.27

2p Volunteer tracking mechanisms are maintained and checked for accuracy 

on a regular basis.

.49 1.67

Scale 3: Volunteer Plan Sustainability

Indicator

Mean: 

Baseline

Mean: 

Final

3a A plan for integrating volunteers into the organizational structure at 

multiple levels has been developed.

.47 1.36

3b The organization’s annual budget reflects detailed expenses for 

volunteers (e.g., staff time, recruitment, training, supplies, space, 

recognition, etc.)

.67 1.25

3c At least two community organizations (e.g., outside organizations, 

businesses, service clubs) have been approached in efforts to build 

partnerships for volunteer roles in the organization.

1.15 1.87

3d Recruitment efforts are regularly evaluated for their effectiveness and 

adjustments are made to strategies accordingly.

.44 1.47

3e

The organization has a plan to ensure that volunteers represent the 

diversity within the community, including a system for recruiting and 

meeting needs of diverse groups.

.53 1.22

3f Volunteer supervisors are held accountable for ensuring that their 

volunteers are complying with applicable human resources and risk 

management policies.

.73 1.51
Assessing the Volunteer Infrastructure Program Using the Volunteer Capacity Assessment

9

3g The organization conducts an annual review and update of all volunteer 

position descriptions.

.31 1.22

3h A system to provide feedback and evaluation on the work of individual 

volunteers has been implemented.

.29 1.07

3i Volunteers are matched with appropriate positions based on their 

abilities, interests, and level of commitment.

1.13 1.78

3j Ongoing training plan and materials have been implemented. .45 1.11

3k

Staff and volunteer leaders are trained in volunteer management best 

practices that include varied approaches appropriate for diverse 

individuals and groups.

.49 1.16

3l Conflicts between staff and volunteers have been anticipated and 

addressed by organization administration.

.49 1.29

3m The organization’s volunteer retention rate (e.g., number of volunteers 

that completed their original commitment to the organization) is tracked, 

recorded, and analyzed.

.40 1.22

3n There is a process in place to analyze volunteer feedback and to make 

adjustments to the program.

.27 1.24

3o An evaluation plan has been developed to utilize the data obtained using 

the volunteer tracking mechanisms.

.27 .96