The Efficacy of AmeriCorps Weed Treatments in Montana 

State Parks 

2019 Impact Evaluation 

Montana State Parks AmeriCorps 

Prepared October 2019 by:

Joe Naiman-Sessions & Seth Shteir

Montana State Parks

1420 East Sixth Avenue

Helena, MT 59620-0701

(406) 444-5275

Montana Fish, Wildlife & Parks - Montana State Parks AmeriCorps 
2 

Executive Summary

A critical threat to all Montana State Parks is the presence of noxious weeds. Noxious weeds are defined

by the 2008 Montana Fish, Wildlife & Parks Weed Management Plan (2008) as, “any exotic plant species 

established or that may be introduced into the state which may render land unsuitable for agriculture, 

forestry, livestock, wildlife, or other beneficial uses.” Noxious weeds infest 20% of the land base of 

Montana State Parks (1,291 of 6,255 acres) and are present in most of the state’s 55 park units. They

adversely impact both cultural/historical sites and recreational/natural parks. 

One of the main program goals of Montana State Parks AmeriCorps (MSPA) is enhancing park land, 

which includes noxious weed control. AmeriCorps members implement a variety of weed treatments: 

organizing volunteers for hand pulls, chemical applications, and biological control collections and 

releases (Montana Fish, Wildlife & Parks 2018). AmeriCorps members also educate the public about the

threats posed by noxious weeds to Montana’s economy, culture and recreational opportunities. 

As a grantee of the Corporation for National and Community Service (CNCS), MSPA is required to 

complete a program evaluation once every three-year grant cycle. Because MSPA receives less than 

$500,000 from CNCS annually, it is permitted to conduct an internal evaluation. In 2016, MSPA 

developed our Evaluation Plan to examine the efficacy of AmeriCorps members’ land improvement 

projects. In striving to meet the qualifications for the highest CNCS evidence tiers, the evaluation was 

further refined to examine the efficacy of AmeriCorps members’ 2018 weed treatment projects. 

This final evaluation was designed to assess changes in a targeted noxious weed species (spotted 

knapweed), total weed canopy cover, and beneficial plant canopy cover utilizing a Before-After ControlImpact (BACI) statistical analysis. The evaluation examines the before and after treatment conditions of 

noxious weed infested plots of state park land for the duration of one growing season. Three plots were 

examined at three state parks utilizing three line-intercept transects per plot for a total of 27 analyzable 

units. Due to a weather event, nine units were excluded from this evaluation. Thus, 18 units were

ultimately examined using an analysis of variance (ANOVA). 

The ANOVA indicates that AmeriCorps members’ treatment efforts achieved the goals of decreasing 

spotted knapweed and total noxious weed cover and increasing beneficial plant coverage. Spotted 

knapweed coverage increased by 19 percent at the control sites and decreased by 6 percent as a result 

of treatments. Total noxious weed cover increased by 9 percent at control sites and was reduced by 25 

percent when treated by AmeriCorps members. Finally, beneficial plant cover decreased in control sites 

by 8 percent and increased in treatment sites by 13 percent. Our data demonstrates that Montana State 

Parks AmeriCorps members’ treatment of noxious weeds was successful and statistically significant. 

Treatment Results Statistical Analysis

Metric

Control 

Sites

Treatment 

Sites

pvalue Statistically significant?Effect 

size

Spotted Knapweed Cover

19% 

increase 6% decrease >0.001 yes 47%

Total noxious weed cover 9% increase 25% decrease 0.020 yes 57%

Total Beneficial plant cover

8% 

decrease 13% increase 0.049 yes 49%

Table 1: BACI Analysis Results

Further areas for future evaluation exploration are also discussed. 
3 

Table of Contents 

Page #

Introduction..................................................................................................4

Goals and Objectives...................................................................................4 

Methods........................................................................................................5 

Results...........................................................................................................7

Discussion...................................................................................................10

References..................................................................................................13 

Appendix A: Line-Point Intercept Data Sheet........................................14 

List of Tables 

BACI ANOVA Results....................................................................................2

List of Figures 

Figure 1. Mean difference as percent of knapweed cover at control 

and treatment sites, pre/posttest.............................................................8 

Figure 2. Mean difference as percent of total noxious weed cover at 

control and treatment sites, pre/posttest................................................8

Figure 3. Mean difference as percent of total beneficial plant cover at

control and treatment sites, pre/posttest................................................9 

Figure 4. Mean difference as percent of total no plant cover at 

control and treatment sites, pre/posttest..............................................10
4 

Introduction

The Montana State Parks AmeriCorps Program is an environmental service program of Montana Fish, 

Wildlife & Parks that promotes healthy, active, and environmentally aware communities by enhancing 

park land, enriching educational opportunities, increasing volunteerism, and improving community 

outreach in state parks. Montana State Parks manages 41,780 acres of land and 15 percent of that

(2,740 acres) have been reported to be infested with noxious weeds (Montana Fish, Wildlife & Parks 

2018). Montana State Parks system receives 2.5 million visits per year and is often utilized as the 

public’s place of entry into outdoor recreation and environmental education (Montana Fish, Wildlife & 

Parks 2019). Montana State Parks AmeriCorps members are well poised to educate and engage our 

visitors about the threat of noxious weeds to our parks, public lands, and the state as a whole. 

To comply with AmeriCorps grant requirements and review its restoration practices, the Montana State 

Parks AmeriCorps program has opted to complete an internal impact evaluation. This evaluation

examines the efforts of Montana State Parks AmeriCorps members’ treatments of noxious weeds as 

defined by the State of Montana Noxious Weed Management Plan (2017). The intended results are a

decrease in spotted knapweed (targeted noxious weed for treatment) and total noxious weed cover, 

while improving the coverage of beneficial plant species in Montana State Parks. 

This internal evaluation has been completed in consultation with the Montana Fish, Wildlife, and Parks 

Weed-Habitat Improvement Program (WHIP) and utilized the U.S. Department of Agriculture’s 

Monitoring Manual for Grassland, Shrubland and Savanna Ecosystems (Herrick et al. 2017) and Bureau 

of Land Management’s Measuring and Monitoring Plant Populations (Elzinga et al. 2015) for best 

practices.

The evaluation assesses the outcomes of weed treatments in two State Parks after one year of 

treatment. Outcomes evaluated include changes in knapweed cover, invasive cover, and beneficial 

plant cover. 

Goals and Objectives 

Efficacy of noxious weed treatment is commonly measured against project goals and objectives over a 

period of years. It is often years before sustainable change can be measured due to the length of time it 

takes for native vegetation to grow; biological controls to become established; seed banks to decrease; 

and perennial and biennial weeds to die off. Five to ten years are typical monitoring periods for stream, 

wetland, and buffer mitigation sites. (Elzinga 2015).

With that context, it is important to keep in mind that this evaluation took place over one growing 

season and therefore focused on noxious weed treatments to reduce noxious weed cover (i.e. amount 

of area that weed canopy encompasses). An expected result of reducing noxious weed canopy cover is 

that beneficial plants are able to expand their canopy coverage due to the absence of weeds. However, 

a recognized limitation of this study is that it can take years for native vegetation to become established 

and fully recover.
5 

As part of this evaluation, AmeriCorps members applied treatments to known areas of spotted 

knapweed infestations in Montana State Parks. Knapweed was selected as a target species because: 

1. Knapweed utilizes allelopathy to inhibit neighboring plant communities.

2. Knapweed can be treated by a variety of methods.

3. Knapweed is widespread in both Montana’s State Parks and the state as a whole.

The goals and objectives of Montana State Parks AmeriCorps weed treatments is to reduce the negative

impact of noxious and invasive species on park ecosystems, while restoring beneficial plant 

communities. It should be noted that our definition of beneficial plant communities includes native 

species, as well as other species that have been traditionally utilized in restoration practices to reduce 

erosion consistent with the Montana Department of Fish, Wildlife & Parks Noxious Weed Management 

Plan. Specifically, our program’s goals and objectives are the following: 

Goals:

1. Improve park ecosystem functions within designated park plots by:

a. Reducing the presence of spotted knapweed

b. Reducing the presence of noxious weeds

c. Increasing the presence of the native plant population

Treatment Objectives: 

1. AmeriCorps members improve habitat functions within selected plots of Montana

State Parks by:

a. Reducing the presence of spotted knapweed by 10%

b. Reducing the total presence of invasive plants by 10%

c. Increasing the beneficial plant population by 10%

Methods

This evaluation utilizes the Before-After Control-Impact (BACI) analysis in order to meet a quasiexperimental design standard. The BACI method is the preferred method in environmental monitoring 

and experimental design when treatment sites cannot be randomly selected (Conner, et al., 2016). In 

the case of this treatment program, sites cannot be randomly selected within a park unit because the 

presence of weeds are not evenly distributed and therefore would not be useful covariates. 

Additionally, covariates must meet a rigorous selection criterion to ensure similarity (Stewart-Oaten 

2001). In order to meet this criterion geographically proximate sites were selected that were also similar 

in aspect, elevation, and moisture levels. Finally, to qualify for inclusion into this experiment, park units 

must have AmeriCorps members stationed at the park to perform the treatments. 

Sampling Sites

Three known noxious weed infestation sites were selected at three separate state park units where 

AmeriCorps members were stationed. This created a total of nine sampling sites that ranged in size from 

one to three acres. Control sites were carefully considered based on their similarity to the 

intervention/impact sites (size, aspect, elevation, drainages, etc.). 
6 

Sampling Design

The sampling design included an assessment of pre-treatment site conditions collected in the spring and 

post-treatment site conditions collected in the fall of 2019. Pre and post conditions were collected at 

both the control and intervention sites. At each site (control and intervention) three line-point intercept 

transects were performed for a total of twenty-seven transects. 

Data Collection

Three representative plots were located at three state parks. Each of these plots were assigned to be 

either a control plot or a treatment plot. Three line-point intercept transects were performed at each 

plot for a total of twenty-seven transects. Each transect spanned one hundred feet with a point 

recorded at every one-foot interval. The monitor recorded all vegetative growth, or lack thereof, that 

intercepted the given point column for thirty inches above the point (30 inches is the average height of

spotted knapweed and all other herbaceous Montana noxious weeds). 

The data collected at each point was: no vegetation, beneficial grass, beneficial herbaceous plant, 

beneficial woody shrub, beneficial tree, spotted knapweed, other noxious weed species. These data 

points were then coded into knapweed cover, noxious weed cover (including knapweed), beneficial 

plant cover, and no plant cover. The presence of spotted knapweed or other noxious weeds superseded 

the presence of beneficial plants in the coding to what was present within a given line-point intercept 

column. This coding was then tallied for an entire transect to include total knapweed cover, total 

noxious weed cover, total beneficial cover, and total no plant cover. 

Photo sites were also established looking along the transect at point one and one-hundred. A photo site 

was also taken at point 50 perpendicular to the transect at a distance of ten feet. 

Pre-treatment (BEFORE)

The twenty-seven transects were conducted prior to treatments being applied to the noxious weeds at 

the sites in June (Montana’s late spring). The transects were marked at either end using landscaping 

whiskers and a latitude and longitude were recorded for future reference in order to ensure the 

pre/post transects were the same.

Post-treatment

The transects were revisited in late September as the growing season was in decline and after 

treatments were applied. The same sampling method as the pre-treatment monitoring was employed to 

gather the point data for the exact locations as the pre-treatment. 

Statistical Analysis

Analysis was conducted on the transect level. This was chosen over the plot level and individual data 

point level as it was determined to be most useful at looking at the ecosystem as a whole while retaining 

enough sample size to be useful. 

Data collected on beneficial and invasive plant cover was analyzed using ANOVA test. The analysis tested 

the following null hypothesizes:

• Treatments did not affect knapweed cover.
7 

• Treatments did not affect noxious weed cover.

• Treatments did not affect beneficial plant cover.

An alpha level of .05 was utilized to determine statistical significance. The power to detect a 25 percent 

difference in cover was .94, given a standard deviation of 25 percent. 

Quality Control

The results were evaluated for completeness, accuracy, and other external influencing factors that 

would exclude data from inclusion in the analysis. Data from one park (9 total transects) was excluded 

due to a record early snow storm that shifted vegetation coverage out of its normal standing location 

that was measured during the pre-treatment monitoring. This park’s data was omitted as it would not 

be consistent with the other two parks whose post treatment monitoring was conducted before the 

storm. Sixty-six percent of sites met the criteria for inclusion in this analysis. 

Results 

Summary of Treatment Actions

During the summer of 2019, Montana State Parks AmeriCorps members performed two types of 

treatment for this evaluation. One plot at Giant Spring State Park received chemical treatments. One 

plot at Lewis and Clark Caverns State Park received chemical treatment and a second plot received 

manual control by the means of hand pulling applicable weeds. Members utilized the Montana Noxious 

Weeds (Pokorny 2018) field guide to help identify weeds and determine appropriate treatment actions.

This guide details which weeds are eligible for hand pulling as rhizomatous weeds should not be hand

pulled because defoliation promotes further sprouting of new plants from any remaining root 

fragments. Therefore, only tap rooted plants were hand pulled (i.e. spotted knapweed, houndstongue, 

hoary alyssum, etc.), and not all noxious weeds were treated in hand pull sites (rhizomatous plants such 

as dalmatian toadflax, leafy spurge, etc.).

Descriptive Statistic Analysis

The weed infestation transects selected for inclusion of this evaluation prior to treatment had a mean 

noxious weed presence of fifty-four percent at control plots and fifty-three at treatment sites; the 

beneficial plant presence was thirty-three percent at control sites and thirty-two percent at treatment 

sites; and spotted knapweed cover was twenty percent at control sites and eight percent at treatment 

sites. 

Change in Cover

Change in Spotted Knapweed Coverage

The cover of spotted knapweed, the target weed for treatment, decreased by 6 percent at 

treatment sites compared to control sites which saw an increase of 19 percent. The difference 

between control and reference sites was found to be very significant (p < .001). Levene’s Test of 

Equality rejected the null hypothesis, and the effect size of that can be attributed to members 

treatments was 47 percent as a result of a partial eta squared test. 
8 

Figure 1: Mean difference as percent of knapweed cover at control and treatment sites, pre/post test

Change in Total Weed Coverage

On average, the total noxious weed coverage on our control sites saw increase of 9 percent, 

whereas sites that had treatments performed by AmeriCorps members saw a noxious weed

decrease of 28 percent. The decrease in noxious weed coverage at treatment sites was found to 

be significant than at control sites (p=.002). Levene’s Test of Equality rejected the null 

hypothesis, and the effect size of that can be attributed to members treatments was 57 percent

as a result of a partial eta squared test.

Figure 2: Mean difference as percent of total noxious weed cover at control and treatment sites, 

pre/post test

0

10

20

30

40

50

60

70

Pre Knapweed Coverage Post Knapweed Coverage

Control Sites

Treatment Sites
9 

Change in Beneficial Plant Coverage

An increase in beneficial plant coverage was found to be significant compared to the alpha level 

of 5% (p=.049). Beneficial plant coverage saw a 13 percent increase at treatment sites as 

compared to a decrease of 8 percent at the control sites. Levene’s Test of Equality rejected the 

null hypothesis, and the effect size of that can be attributed to members treatments was 49 

percent as a result of a partial eta squared test.

Figure 3: Mean difference as percent of total beneficial plant cover at control and treatment sites, 

pre/post test

Change in No Plant Cover

The change in mean no plant cover increased as a result of treatments (Pre = 15%; Post = 27%) 

but stayed consistent in the control sites (Pre = 13%; Post = 12%). This change was found to be 

significant (p=.013) but is not a useful analysis because the why the change of no cover 

increased cannot be ascertained. An increase in no plant cover could be attributed to a point 

previously having a weed that treatment removed, or if treatments resulted in a point that 

previously held with a beneficial plant no longer did after treatment due to damage of that plant 

(i.e. trampling, herbicide wind drift, herbicide over concentration, etc.). A pre/post anaylsis at 

the individual point level may help determine why an increase in no cover was realized but was 

outside the scope of this evaluation due to budgetary and time limitations. 
10

Figure 4: Mean difference as percent of total no plant cover at control and treatment sites,

pre/post test

Discussion 

This evaluation provides strong evidence that AmeriCorps member’s weed treatments were successful. 

They resulted in a reduction of spotted knapweed and total noxious weed coverage and demonstrate 

promising results regarding increasing beneficial plant coverage. During 2019, Montana State Parks 

AmeriCorps utilized noxious weed intervention strategies that are consistent with proven weed 

management strategies practiced throughout the region. 

The program’s weed treatments proved to be successful as statistically significant changes were found in 

all the objective fields. There was a decrease in knapweed, decrease in total noxious weeds, and an 

increase in the beneficial plant community. 

Evidence of program success include the following:

• Knapweed coverage decreased six percent with an end total coverage of two percent in the 

treatment sites which represented a statistically significant change in comparison to the control 

group which saw a 19 percent increase.

• Three of the nine treatment sites had no detected knapweed on the post test, and another four 

sites had two or fewer plants present. This extremely low occurrence of knapweed should 

contribute to a future reduction of knapweed in the seed bank and a reduction of sprouting 

knapweed in subsequent seasons. 

• The mean total noxious weed coverage decreased by 28 percent, exceeding our program’s 10

percent objective. This significant decrease was realized through two main methods: hand 

pulling and chemical treatments. 
11

• On the whole the AmeriCorps’ weed treatments were quite successful nearly cutting the mean

noxious weed cover in half with the exception of one treatment site which saw an increase in

total weed presence while another site stayed the same before and after treatment.

• The mean beneficial plant coverage increased by 13 percent, meeting the 10% percent

objective. While this finding was statistically significant (p=.049), it only just surpassed the .05

alpha level. It should be noted that two of the nine treatment transects saw a decrease in

beneficial plant coverage.

Our program data was collected per our evaluation plan, with the exception being the exclusion of data 

from Spring Meadow Lake State Park due to a weather event that precluded an accurate post-test. Had 

this data been available, two more hand pull plots and one more control plot would have been available 

for analysis. Having this additional data would have permitted a more thorough analysis of treatment 

types (chemical vs hand pull) against each other and without it there was insufficient sample size with 

which to evaluate this research question. 

It should be noted that while the program found statistically significant results in terms of a decrease in 

knapweed, decrease in total noxious weeds, and an increase in the beneficial plant community, there 

were varying degrees of success in meeting some of the program’s articulated objectives. For example, 

the mean total knapweed coverage did not decline at treatment sites at the objective level of 10 

percent, because the mean knapweed coverage at the time of treatment was 8 percent and therefore 

could not be reduced by 10 percent.

Additionally, it is important to understand that both the hand pulling and chemical treatments that were 

applied to knapweed are also applicable to treating other noxious weed species (hand pulling: hound’s 

tongue, ox eye daisy; chemical: hounds’ tongue, Canadian thistle, dalmatian toadflax). Other types of 

treatment can be employed and evaluated in the future (biological controls, cultural controls i.e. plant 

species competition).

Finally, the treatment of our study plots did not involve an integrated weed management strategy, 

which combines treatment types that would have included seeding these plots with native plant seed in 

addition to other treatments. It is reasonable to assume that this would significantly increase the 

beneficial plant coverage, but is also harder to control for existing plant stock reseeding/spreading itself. 

Implementing an integrated weed management strategy is an area of expressed interest on behalf of 

the park managers and warrants future study.

Opportunities for Further Investigation 

Future evaluations would benefit from a longitudinal study design that would capture plot canopy cover 

over multiple years. Additionally, we believe it would also be valuable to include additional control and 

treatment plots in other parks to better analyze data and also to see the results from our treatments.

Our data demonstrates that AmeriCorps members’ treatments did have significant impact on noxious 

weeds and native plant communities during a single growing season, we recognize that changes in 

canopy cover often take years to be realized. This is due to existing weed seed bank in the soil continue 

to germinate for two to ten years post production (Schultz 2011) and the time for reseeding of native 

plants to happen because Montana native seeds often will not germinate during the summer when this 

evaluation took place (Morrison 2003). 
12

Whenever possible, Montana State Parks prefers to employ an Integrated Strategies Management (ISM) 

of noxious weeds for select infested sites. ISM is the utilization of two or more weed treatment 

strategies on a given site and has shown greater impact to noxious weed populations than a single 

treatment type alone (Miller 2016). These treatment types vary from mechanical (mowing, tilling, 

pulling, etc.), chemical (herbicidal), biological (grazing or introduction of predatory insects, fungi, or 

molds), cultural (reseeding & replanting competitive species). In this evaluation, AmeriCorps members 

only employed a single treatment type for a variety of reasons (i.e. budgetary constraints, time, etc.) but 

future evaluations could employ multiple treatment types on a single site. A limitation of evaluating ISM 

is it requires a much larger sample size due to the compounding treatments can confound analysis. Due 

to time and budgetary requirements of a larger sample size, this may not be pursuable. 

A finding of this evaluation was the increased beneficial plant coverage objective (p=.049) being slightly 

statistically significant at the .05 alpha level. To strengthen beneficial plant regrowth, an ISM strategy of

reseeding or transplanting desirable plants at the treatment sites could be utilized. 

Another avenue for future research would be the examination of beneficial plant diversity as a result of 

treatments. This would be done through the further delineation of native plants from non-native 

beneficial plant species, examination of plant guilds (i.e. native bunch grass, native rhizomatous grass, 

native forbes, etc.) or examining the makeup of the plant community down to the plant type. Because 

this would greatly increase the knowledge needed by the monitor and increase monitoring time, this 

evaluation was not selected for the current analysis. This level of monitoring would be able to be 

completed through the use of an external environmental monitoring company or university partnership.

Another limitation that could be examined in future studies would be the differences in weed 

treatments across the Continental Divide that runs through Western Montana. This evaluation only 

examined parks east of the divide, which contains two-thirds of the states’ parks. There are significant 

differences that would make this inquiry valuable in that parks west of the divide tend to occur at higher 

elevations with significantly more rainfall than east of the divide.

Lastly, a final future evaluation consideration would be to examine the efficacy of various treatments 

employed by AmeriCorps members. One future research question might evaluate the efficacy of 

spraying versus hand pulling, but that could not be examined in this evaluation due to lack of sample 

size. This is still an area of great interest to the program to be able to direct its members in the most 

effective treatment type. 

Summary

In conclusion, noxious weed treatments conducted by MSPA members during the summer of 2019 

successfully met the objectives of reducing total noxious weed and spotted knapweed cover as a result 

of members’ treatments. It also showed that members treatments increased the presence of beneficial 

plants. While the treatment was statistically significant on beneficial plant communities, it is desired in 

future efforts to see a stronger statistical significance to members treatments increasing beneficial plant 

communities through the implementation of integrated species management. 
13

References 

Conner, M., Saunders W.C., Bouwes, N., Jordan, Chris (2016) Evaluating impacts using a BACI 

design, ratios, and a Bayesian approach with a focus on restoration. Environmental Monitoring 

& Assessment 188: 555. 

Herrick, J, Van Zee, J, McCord, S, Courtright, E, Karl, J, Burkett L (2017) Monitoring Manual for Grassland, 

Shrubland and Savanna Ecosystems (USDA-ARS Jornada Experimental Range) NMSU Las Cruces, New 

Mexico

Elzinga, C, Salzer, D, Willoughby J (2015). Measuring and Monitoring Plant Populations (BLM Technical 

Reference 1730-1) Washington, DC: U.S. Government Printing Office. 

Montana Fish, Wildlife & Parks (2008). Statewide Integrated Noxious Weed Management Plan Helena, 

Montana

Montana Fish, Wildlife & Parks (2018). Noxious Weed Management Fiscal Year 2018 Helena, Montana

Montana Fish, Wildlife & Parks (2019). State of Montana State Parks – 2019 Annual Report Helena, 

Montana

Morrison, S, (2003) The Magic of Montana Native Plants: A Gardener’s Guide to Growing Over 150 

Species From Seed. Montana Native Plant Press. Lincoln, Montana

Pokorny, M, Mangold, J (2018) Montana’s Noxious Weeds. Montana State University Extension, EB0159. 

Bozeman, Montana

Schultz, B, (2011) The Noxious Weed Seedbank: Out of Sight – Out of Mind and Eventually Out of Control. 

University of Nevada Cooperative Extension. Fact Sheet 12-01. Reno, Nevada

State of Montana (2017) Montana Noxious Weed Management Plan Helena, Montana

Stewart-Oaten A, and Bence JR. (2001). Temporal and spatial variation in environmental impact 

assessment. Ecological Monographs, 71: 305–339. 
14
15