# Silviculture Best Management Practices Implementation Monitoring

A Framework for State Forestry Agencies



# Southern Group of State Foresters

Water Resources Committee

June 2007

# Silviculture Best Management Practices Implementation Monitoring

A Framework for State Forestry Agencies

Southern Group of State Foresters

Water Resources Committee

June 2007Foreword

The Federal Water Pollution Control Ac t Amendments recognized nonpoint source pollution, and called on states to develop and implement water quality management plans. Since then, state forestry and stat e water quality agencies have been working closely with the Environmental Protection Ag ency (EPA) to minimize silviculture-related sources of nonpoint source pollution.

To address silviculture related water pollution in the southern states, a traditional regulatory approach was initially propose d. However, after further analysis and consultation with the forestry community, EPA and the states generally agreed that a nonregulatory approach was more effective. This approach was based primarily on education and field demonstration, wi th the following basic components:

1. Identification of Best Management Pract ices (BMPs) to protect water quality during forestry operations; and

2. Widespread education/training of forestry practitioners and forest landowners to facilitate the implementation of BMPs; and

3. Routine monitoring of forestry operations to determine the level of BMP implementation.

To date, all southern states have developed silviculture BMPs, which have been approved by EPA. Most of these states have recen tly revised or updated their BMPs to keep current with changing information and technol ogy. Likewise, all southern states have developed and conducted education and trai ning sessions for forestry practitioners, landowners, managers and loggers, which in clude the distributi on of materials and emphasize BMP implementation.

However, not all of these states have deve loped routine BMP monitoring procedures to measure actual implementation levels. In addition, no model pro cedure for conducting such monitoring exists. Thus, states with monitoring programs have measured and reported BMP implementation using significa ntly different methods. Consequently, monitoring results have been met with vary ing degrees of accepta nce by the public and by regulatory agencies. Inconsistency among st ates with respect to statistical design, reproducibility, and general objectivity have been c ited as areas of concern.

In order to improve and maximize the integr ity of BMP implementation monitoring in the South, the Southern Group of State Forest ers appointed a Task Force to develop

recommendations for a more consistent a pproach to BMP monitoring in the region. Specifically, the Task Force was charged with developing a framework to provide southwide guidance for monitoring BMP implement ation that would be statistically sound, objective, and technically defensible. Th is framework would achieve analytical consistency and results would be ge nerally comparable among states.The Task Force, composed of hydrologists and water quality specialists from state forestry agencies, U.S. Forest Service, and forestry industry, in consultation with EPA Region IV, met during 1996 – 1998 and completed the initial document. On March 25- 26, 2002, a subcommittee of that Task Force reconvened and completed this revision.

Task Force Members

Jeff Vowell – Chairman, Florida Division of Forestry* Frank Green, Georgia Forestry Commission* Tim Adams, South Caroli na Forestry Commission Darryl Jones, South Carolina Forestry Commission* Robin Bible, Tennessee Division of Forestry Sam Austin, Virginia Department of Forestry Matt Poirot, Virginia Department of Forestry* Gary Cole, Alabama Forestry Commission* Burl Carraway, Texas Forest Service* Mike Sampson, Mississippi Forestry Commission* John Greis, U.S. Forest Service* David Hoge, U.S. Forest Service* Bruce Prud’homme, U.S. Forest Service* Rob Olszewski, Plum Creek Timber Company Jim Shepard, National Council of the Paper Industry for Air and Stream Improvement

* Subcommittee members, 2002

# Table of Contents

Introduction 5

Implementation Monitoring Framework Protocol Monitoring Frequency 5

Site Selection 5

Practices to be Evaluated 6 Basis for Practice Evaluation and Reporting 7 Scoring Methodology 7

Risk Assessment 8 Follow-up Actions 9

References 10

Acknowledgements 10

Glossary 10

Appendix Statistical Guide for BMP Implementation Monitoring 135 Introduction

This document is presented as an Implemen tation Monitoring Framework within which state forestry agencies can build or revise their current monitoring programs. Widespread

utilization of this document within the region is expected to improve consistency among states in the specific aspects of BMP m onitoring listed below. In addition, the recommendations for each specific aspect ar e envisioned to be core elements of a credible evaluation and reporting process.

Monitoring Frequency

Issue : How frequently should BMP implementation monitoring be conducted and reported?

Alternatives Considered : Annual, biennial, every three years and continual monitoring.

Recommendation : Statewide implementation monitoring should be conducted and reported at a minimum of every three years.

Rationale : Due to the large number of fore stry operations conducted annually, the number of sites necessary to achieve statisti cal reliability, and the logistics of locating, visiting and evaluating them, annual monitori ng and reporting is of ten not practical. Further, there are no significant advantages of annual monitoring and reporting that justify the additional burdens.

Monitoring and reporting on at le ast a three year basis is more logistically achievable, and is consistent with typical 319 funding cycl es for states receiving federal grants. In addition, monitoring at this frequency is considered often enough to allow visual observations of on-site problems and take timely corrective action.

Site Selection

Issue : What characteristics should a forestry s ite/operation exhibit in order to qualify as a BMP implementation monitoring site?

Alternatives Considered: 1. Minimum/no minimum area (acres)

2. Presence/absence of surface water on site 3. Time since treatment (years) 4. Site selection methodology (to eliminate bias) 5. Sample size (statistically valid confidence interval)

Recommendations: 1. No minimum area, but a site must be pa rt of a normal, ongoing silvicultural operation, i.e., not in the process of c onversion to another land use.6 Rationale: Since forestry operations occur on tracts of all sizes and BMPs apply regardless of acres involved, all forestry ope rations should be elig ible for monitoring. However, operations that include timber harvesti ng as part of a change in use, should be disqualified regardless of the size of the ope ration. Such activities would not accurately reflect normal silvicultural operations.

2. The presence of surface water features is not necessary for a site to be eligible for BMP implementation monitoring.

Rationale : BMP implementation in most states is not contingent upon the presence of surface water on-site. However, those states th at have proximity restrictions associated with BMP implementation should select monito ring sites using the appropriate criteria.

3. The most recent silviculture activity(s) on a site to which BMPs apply must not have been completed more than 2 years prior to implementation monitoring.

Rationale : Forestry operations more than 2 years prior are increasingly difficult to evaluate because of rapid regrowth of vege tation and more difficult access. Likewise,

evidence of erosion and sedimentation beco me less visible over time, as does the opportunity to correct such problems without "re-dist urbing" sensitive areas.

4. Sites for implementation monitoring may be located using aerial reconnaissance, severance tax records, notific ation logs, or other available sources of information. However, it is essential to achieve random, stratified random or randomized cluster statistical design to obtain an unbiased sample.

Rationale : Several data sources can provide the information necessary to select a random sample of forestry operations sites. However, it is importa nt that the sample population accurately reflect actual conditi ons in a given state. For example, portions of a state in which forestry operations are concentrated should be sampled accordingly, as should those with fewer operations.

5. The sample size should be sufficient to ach ieve an estimate of implementation that is

+ 5% within the 95% confidence interval.

Rationale: To maximize the validity and credibility of the monitoring results, the number of sites evaluated for BMP implementation should be calculated to provide minimum error (+ 5%) and high confidence (95%). Desi gning a statistically valid sampling procedure for implementation monitoring and an alyzing the results should be consistent

with " Sampling and Estimating Compliance with BMPs " (1) and/or the Statistical Guide for BMP Implementation Monitoring found in the Appendix.

Practices to be Evaluated

Issue: Which categories of practices shoul d be evaluated for BMP implementation monitoring?7 Alternatives Considered : Harvesting; Site Preparation (mechanical, chemical, burning); Forest Roads; Stream Crossings, Streamside Management Zones, Firebreaks, Forest Chemical Application (fer tilization, he rbicides).

Recommendation : All of the above altern atives should be evaluated.

Rationale: These BMP categories contain all practices that are generally associated with operational silviculture in the South.

Basis for Practice Evaluation and Reporting

Issue : On what basis should BMPs be evaluated and reported?

Alternatives Considered : Individual practices, Categorie s of practices, Overall site.

Recommendation : Evaluation and reporting should in clude all three levels of BMPs

listed above.

Rationale: Evaluation of BMPs at the practice le vel provides the ba sic measure of onsite BMP implementation. This level of information also allows for comparison of a specific practice among all monitoring sites and against any other site variables. Such comparisons are useful for identifying thos e variables most often associated with nonimplementation.

In addition, by evaluating cat egories of practices, mon itoring can provide broader conclusions about BMP implementation for stre am crossings, roads, etc. Also, this information can identify training needs for forestry agency personnel, and education needs for forestry practitioners.

It is likewise useful to water quality agenci es, other interested parties and particularly forest landowners to know the overall or cumulative level of BMP implementation for individual forestry operations. This is a primary and traditiona l measure of program success, and indicates the efficacy of the non-regulatory approach to controlling

silvicultural related nonpoint source pollution.

Scoring Methodology

Issue : How should BMP implementation monitoring be scored?

Alternatives Considered : Pass/Fail; Graduated Scale; Pe rcent Correct Implementation;

Yes/No

Recommendation : An individual practice should be scored as “Yes” when applied as specified in the state's BMP Manual. If a particul ar practice is not applicable, this should be noted as well. Any significant deviation fr om practice specificati ons should result in a8 “No” answer for BMP implementation. Categories and overall scores should be expressed as a simple percentage of all applicable practices. For example, if 100 practices were applicable but only 90 were actually implemented correctly, then the score would be 90% for that category or site, as the case may be.

Rationale: Evaluating whether or not BMPs have been properly implemented, and their applicability to specif ic site conditions yields the mo st objective and reproducible method of implementation monitoring. While some judgment will always be necessary in questionable situations, objectivity can be maximized by training. In addition, subjectivity and confusion are minimized by avoiding practice evaluations based on graduated scales for partial implementation, or arbitrary "Pass/Fail" declarations. Simple “Yes/No” scoring of BMPs al so facilitates the calculat ion, summarization and reporting of category and overall implementati on levels on a percentage basis.

Risk Assessment

Issue: How should the risk to water quality resu lting from failure to implement BMPs be

evaluated and documented?

Alternatives Considered : No evaluation of risk; Risk evaluated and significant risk

noted.

Recommendation : Risk to water quality should be evaluated and significant risk documented. Significant risk may be attrib uted to non-implementation for a specific BMP, category of BMPs or the overall operation . The field evaluation of significant risk

should be based on existing on-the-ground conditions resultin g from failure to correctly implement BMPs, that if left unmitigated will likely result in an adverse change in the chemical, physical or biological condition of a waterbody. Such change may or may not violate water quality standards.

Significant risk should also be considered as a situation or set of conditions that can be remedied or otherwise mitigated (2). In a ddition, failure to implement BMPs that results in risks to site productivity, road usability or other site valu es should not be considered a significant risk in the context of implemen tation monitoring. Significant risk should be directly and exclusively relate d to water quality impairment.

Key site conditions often associated with si gnificant risk include, but are not limited to: steep topography and highly erodible soils. Forestry operations conducted under one or more of these conditions without proper im plementation of certain BMPs may have a high potential to result in si gnificant risk to water quality. Some examples of forestry activities where significant risks have been identified are equipment operation in close proximity to surface waters, stream crossi ngs, logging slash disposal and intensive mechanical site preparation. A comprehensive list of on-site indica tors of significant risks to water quality is shown below:9 On-Site Indicators of Signif icant Risk to Water Quality

 Temporary stream crossings remain in channel following operation

 Stream crossings and approaches not stabilized

 Logging debris in waterbody a ffecting or obstructing flow

 Evidence of excessive sediment entering waterbody from adjacent treated area

 Canopy completely or almost completely removed from SMZ on perennial waterbody

 Evidence of heavy equipment operation in stream channel

 Waterbody banks compromised by e quipment or skidding activities

 Water diversion devices absent or severe ly compromised on roads or skid trails where runoff is likely to enter waterbody

 Ruts or other excessive physical damage to soils and cover within the SMZ

 Fill material in stream crossing without adequate means for conveyance of flow

 Un-stabilized fireline tied directly into waterbody

 Oil, chemicals, batteries or other hazar dous materials leaking or remaining on site following operation

 Road or skid trail too stee p or so poorly located that stabilization is improbable

 Excessive defoliation of riparian vege tation caused by herbicide application

Rationale : Documenting the occurrence of significan t risk serves a number of useful and practical purposes. First, risk assessment lends much credibility and integrity to the BMP monitoring process by recognizing that high risk conditions can occur, and that prevention and/or restoration is a high prio rity for state forestry agencies. Second, routine documentation of significant risk will determine whether such instances are the exception rather than the rule, and that lack of BMPs during a silv iculture operation may not necessarily equate to or result in a wa ter quality problem - th is is particularly important as it relates to BMP effectivene ss monitoring (3). Finally, providing forest landowners with an objective ri sk assessment is a valuable public service that not only protects the environment, but can also prot ect the landowner and/or operator from what might otherwise result in enforcement pr oceedings or other personal liability.

Follow-up Actions

Issue : What specific actions should states take following BMP implementation monitoring?

Alternatives Considered: No follow-up; Courtesy copies of monitoring results; Personal visit; Referral (where nece ssary) to regulatory agency.

Recommendation: Landowners who have participat ed in the implementation monitoring should be provided a copy and explanation of the monitoring results. In addition, participating landowners should receive r ecommendations for any remedial actions deemed necessary by the field observer. In cases where a significant risk has been identified, state forestry personnel should atte mpt to schedule a follow-up site visit with10 the landowner, to insure that recomme ndations were understood and implemented satisfactorily.

Rationale: Follow-up activities with landowners and/or loggers serves as a useful educational opportunity, as well as a demons tration of cooperation and courtesy. The BMP monitoring data provides an excellent fo cal point for reviewi ng the performance of an operator and the responsibilities of the la ndowner, in terms of water quality and site protection. Remedial or other actions can al so be recommended at this time, as can commendation for a job well done.

Where a significant risk has been identified in the monitoring process, an on-site followup can be vital to insuring th at the landowner/operator is aw are of the seriousness of the

situation and advised of remedial actions. Potential consequences of inaction can be explained and discussed at that time also, and should include environmental impacts as well as possible enforcement actions or other li abilities. This effort can provide the basis for fulfilling the responsibilities of the stat e forestry agency, and provide the landowner

with the information from which to make an informed decision.11 References

(1) McNew, Ronald W. 1990. Sampling and Estimating Compliance with BMPs, in Workshop on Implementation of Forestry Best Management Practices. Southern Group of State Foresters and USDA Forest Service. Atlanta, GA. January 23-25, 1990. Edited by G. Dissmeyer

(2) Vowell, Jeffery L. and Roy Lima, 2002. Results of the 2001 Silviculture BMP Compliance Survey. Florida Departme nt of Agriculture and Consumer Services, Division of Forestry; Tallahassee, Florida.

(3) Dissmeyer, George E. 1994. Evaluating th e Effectiveness of Forestry Best Management Practices in Meeting Water Quality Goals or Standards. USDA Forest Service, Miscel laneous Publication 1520.

Acknowledgements

The Southern Group of State Foresters wish es to thank the Task Force members and acknowledge the assistance provided by the wa ter resource specialists from other state forestry agencies in the Southern Region. In addition, appreciation is expressed to the U.S. Forest Service Southern Region and the U.S. EPA Region IV for considerable expertise in this effort. Finally a specia l thanks is expressed to George Dissmeyer (retired, U.S. Forest Service) for his leader ship in helping organize and initiate this

process.

Glossary

Implementation Monitoring – The process used to determine the proper application of BMPs according to the specifications in individual state BMP Manuals.

Risk Assessment – The process and criteria used to identify a significant risk to the chemical physical or biologica l integrity of water quality.

Significant Risk – An existing on-the-ground co ndition resulting from failure to correctly implement BMPs, that if left unmitigated will likely result in an adverse change in the chemical, physical or biological condition of a waterbody. Such change may or may not violate water quality standards.

## APPENDIX

Statistical Guide for BMP Implementation Monitoring

# Statistical Guide for BMP Implementation Monitoring

by



## Hughes Simpson, Texas BMP Coordinator John Greis, USDAFS Region 8 Hydrologist Dr. Ron McNew, Professor, University of Arkansas

Southern Group of State Foresters

Water Resources Committee

October 2006

# Table of Contents

Statistical Guide for BMP Implementation Monitoring

Introduction 17

Survey Design Determining the Number of Sites to Monitor 18

Site Selection 19 Data Storage 19 Ensuring Randomness 19

Stratification of Field Sites 19

Data Analysis Margin of Error 20

Confidence Intervals 21 BMP Trend Analysis 22 Area Weighting Results 23 Reporting 2417 

### Introduction

Implementation monitoring is the pro cess used to determine the proper application of Best Management Practices (BMPs) according to the specifications in individual state BMP Manuals. In 1999, the Southern Group of State Foresters (SGSF) endorsed a monitoring framework designed to provide regional guidelines for monitoring BMP implementation so that consistency and relia bility of southern state efforts would be

maximized. The framework calls for evalua tions to be conducted on randomly selected forestry operations and to result in data that is statistically valid.

Field evaluations consist of answering “yes”, “no”, or “not applicable” to questions regarding proper implementation of specific BMPs. These are typically broken down into several activity categories (roads, tra ils, stream crossings, etc.). Each question represents a specific BMP (“yes” means th e BMP was implemented correctly and “no” means it was not). If a BMP listed on the evalua tion form was not applicable to that site, “not applicable” is recorded. Additionally, the presence of a significant risk to water quality is noted for each question if, due to a lack of or malfunction of a BMP, water quality has been impacted or is clearly thre atened. To determine the implementation rate, the total number of “yes” answers is summe d and then divided by the total number of applicable answers (yes / yes + no) to determine the total BMP Implementation rate, expressed as a percen t, for the site.

After combining all results, BMP implemen tation may be reported for the state, regions of the state, landowner types, forestry activities, ri ver basins or watersheds, and BMP groups or other categories of interest for reporting purposes. Strengths (BMPs along streams) and weaknesses (BMPs on roads) ar e generally identified from the results.

In 2004, a task force of the SGSF Water Resources committee was formed to develop this statistica l guidebook to assist the southern state forestry agencies with BMP implementation monitoring design and reporti ng. Included with this guidebook is an Excel spreadsheet created to help states dete rmine how many sites ar e needed to conduct a statistically reliable survey, calculate th e margin of error for each BMP evaluated and reported, and analyze st atistical trends in BMP implementation.

Major elements in the design of a statistical ly valid BMP implementation survey include:

- sampling intensity (total number of sites needed for the survey)

- methodology of choosing sites

- how to ensure randomness of the samples

- stratification of field sites (# of samples per county, landowner type, etc.) so that sound conclusi ons can be drawn from each.

Key calculations for the analysis of a BMP implementation survey will include:

- determining statistical significance of BMP trends

- confidence intervals and margin of error18 

### Survey Design

Determining the sample size, or number of sites to evaluate

Where n = the number of sites to evaluate

p = the estimated overall percen t implementation in the state

m = the margin of error (5%)



### Notes:

- p must be estimated because it is unknown (% implementation from the most recent round of monitoring may be used)

- The closer the estimated value of p is to 100, the lo wer the value of n will be.

- n is highest when p is estimated to be 50%.

- m is the margin of error associated with the estimate of p . There is .95 (95%) probability that the sample taken will produce an estimate which differs from p by

a value of m

Example:

Where p (overall BMP implementation) is estimated at 80%

n = 4 (80) * (100 – 80)

5 2

n = 6400

25



### n = 256

Using the spreadsheet:

The spreadsheet is set up so that all that must be entered is the estimated value of

p (Est. % BMP Impl). It will then automatica lly calculate the number of sites to evaluate based on an embedded formula and a margin of error equal to 5% (as recommended by the SGSF framework). 

### n = 4p (100 - p)

m 2 

## n = 4p (100 - p)

m 219 

### * These equations calculate the mini mum number of sites necessary to evaluate. Increasing the sample size will yield an even more accurate estimate of BMP implementation. A mi nimum of 100 sites is recommended.

Data Storage

BMP implementation monitoring data can be stored in a number of different

formats. The easiest is an Access databa se consisting of the individual state’s BMP monitoring form (checklist), data tables, queri es, and reports. Site evaluations can then be entered directly into the database in a user friendly format. Queries and filters can be created to display the “target” data (i.e. implementation scores for tracts in which a professional forester was i nvolved) for export to the Statistical Guide book Spreadsheet. Reports can provide a quick glance at the resu lts of the survey (i.e. % implementation by county). GIS programs can import data fo r geographical representation and further analysis. A sample database is available for stat es to customize to fit their needs.



### Site Selection

BMP field sites may be selected in a nu mber of ways: aerial reconnaissance, severance tax records, timber deeds, drive-bys etc. To avoid bias, it is important that personnel involved in the site selection process do not c ontact consulting foresters, industry foresters, or large landowners to prov ide a list of recent harvesting operations. This could bias samples to the “good” sites. Of equal importance is to avoid selecting sites thought to be either “good” or “bad”. The SGSF framework calls for sites to be no older than 2 years after the most recent treatment activity.



### Ensuring Randomness

Ensuring randomness is critic al in any type of sampling. One way to help achieve randomness is to identify twice as many site s as are needed for the survey, and use a random number generator to identify specific sites to monitor.



### Stratification of Field Sites by Own ership, Watershed, or Other Factors

Stratifying the monitoring sites based on important characteristics such as ownership type, watershed, or physiographic region, can add substantial value to the survey’s results. It is important that th e sample taken be reflective of the actual conditions. There are two ways to accomplish this:

 Take a truly random sample from the population (this will solve

the stratification but is extremely difficult).

 Intentionally select sample sites based on their stratum

Forest Inventory and Analysis (FIA) data may be used to estimate the number of sites undergoing forestry operations by landowne r type. This percentage can then be

used to estimate the number of monitori ng sites each landowner group should comprise.20 

### Data Analysis

Margin of Error

The margin of error expresses the ma ximum likely difference observed between the sample mean and the true population mean with 95% probability. It is an important

statistical calculation and can be performed for an indivi dual BMP eval uation question (i.e. SMZ width). The following formula is used to perform this calculation:



### Where m = margin of error for a single BMP

P = the percent implementation for a single BMP

n = the number of sites on which the BMP were evaluated

Notes :

- If the actual value of P is larger than the estimated value of P , then the actual margin of

error will be smaller than m

. - This equation is not valid for a subset of all possible sites (i.e. calculating margin of error from the % BMP implementation for NIPF landowners.) - For a BMP that is not applicable to all site s, the actual margin of error will be larger

than m

. - Estimating the average % BMP implementati on across all possible si tes for a group of BMPs and then using this number of site s will produce a margin of error that is

smaller than m

. - If the value of P is 100%, the margin of error is not zero. No calculation can be made.

Example:

Where P (% BMP impl. for adequate SMZ width) was evaluated to be 89% on 125 sites

m = 2 √ 89 (100-89)

125

m = 2 √ 979

125

m = 2 √ 7.832



### m = 5.59721 Using the spreadsheet:

The spreadsheet is designed to calculate th e margin of error for a single BMP. All that must be entered is the % implementa tion for a single BMP (% for single BMP) and the number of sites on which that BMP was evaluated (# of sites).



### 95% Confidence Interval

The 95% confidence interval is a tool that statisticians use to demonstrate their confidence in the measured mean of a sample . It provides a range for which they are 95% confident (i.e. 19 times out of 20) that the actual mean will be found within that range. To calculate the 95% confidence in terval, you must also calculate the mean, variance, standard deviation, standard error, and margin of error.

Example:

Let’s calculate the 95% confidence interval for the following sample:

95%, 80%, 88%, 100%, 77%

First calculate the mean.

95+80+88+100+77 = 440 = 88%

5 5

Then calculate the variance.

Step 1: USS = 95 2 + 80 2 + 88 2 + 100 2 + 77 2 = 39,098 Step 2: SUM = 95 + 80 + 88 + 100 + 77 = 440 Step 3: CF = 440 2 /5 = 193,600/5 = 38,720 Step 4: CSS = 39,098 – 38,720 = 378 Step 5: DF = 5 – 1 = 4 Step 6: Variance = 378 / 4 = 94.5

Next calculate the standard deviation

.

Std dev. = √ variance = √ 94.5 = 9.721

After that, calculate the standard error.

Std. error = (Std dev. / √ number of sites) = 9.721 / √ 5 = 4.347

Next, calculate the margin of error.

Margin of Error = 2(Std. error) = 2 (4.347) = 8.695

Finally, use the margin of error to cal culate the 95% confidence interval.

95% Confidence interval = Mean ± Margin of Error = (79.305, 96.695)22 Using the Spreadsheet

The spreadsheet is set up so that all that must be entered is the individual tract scores (Indiv. % Impl) and the total number of sites (# of sites). The spreadsheet automatically calculates the mean, variance, stan dard deviation, standard error, margin of error, and the 95% confidence in terval (low and high ends).



### BMP Trend Analysis

Analyzing trends or patterns in BMP impl ementation can be useful to target areas or ownership types for concentrated educati onal efforts (i.e. additional logger training workshops). Commonly reported trends incl ude higher BMP implementation rates when professional foresters are used, the landowner is familiar with BMPs, and the logger has attended BMP training.

In order to determine trends in BMP im plementation, several statistical analyses should be performed. First, a parametric tw o sample t-test is c onducted because of the large sample size. This percentage da ta must undergo an arcsine square root transformation prior to analysis . Percentage data must be transformed because they are not normally distributed, which invalidates the normality assumption of the parametric test. A non-parametric test (Wilcoxon) may al so be performed to add greater statistical validity.

To determine statistical significance, the resulting P value was compared to the level of significance. The P value is the probability of observing a value of the test statistic as contradictory (or more) to the null hypothesis as the comput ed value of the test statistic. In these tests, a 0.05 (5%) le vel of significance was used. For the two implementation ratings to be significantly different, the P value must be lower than the level of significance.

Using the spreadsheet:

The spreadsheet is set up so that all you have to do is enter the individual scores for the tracts that answered “yes” to the pa rticular trend question and likewise for those that answered “no” in the respective column. It will then perform the arcsine square root transformation and conduct a parametric two sample t-test on the new data, based on a level of significance of .05. This value will be used to determine whether the difference in implementation scores for that particular trend is statistically significant. This classification is noted by the answer “T RUE” found under the Stat. Diff column.

** 

### The arcsine square root transformation was conducted so that Microsoft Excel could perform the analysis. Mo re robust tests (non-parametric tests like the Wilcoxon) may be conducted to a dd greater statistical validity. These

tests are not included in basic Microso ft Excel programs but can be found in programs like JMP, SAS, or Statistica.23 

### Area Weighting BMP Im plementation Data

Results are typically reported giving equal weight to all sites (i.e. a 20 acre tract counts the same as a 450 acre tract when compili ng all data). Statistically, tracts could also be weighted based on their acreage, i.e. larger tracts would have a greater influence on the total % BMP implementation than the smaller tracts. This analysis can be performed to provide information on how the pr actices are impacting the total landscape. Both methods are useful in reporting BMP implementation rates, though the SGSF framework does not call for area-weighting. The following formula may be used to perform this calculation.



### AW % = Σ (((indiv A/Total A) *100)) * % Impl)))

Where AW % = area weighted BMP implementation %

A = area (acres) % Impl = individual tract % BMP implementation

Example:

For this example, let’s use 5 individual tract scores and their respective size:

95% - 100 acres, 80% - 35 acres, 88% - 70 acres, 100% - 275 acres, 77% – 20 acres

Equal weighted % BMP Implementation = Sum of scores divided by number of sites

95+80+88+100+77 = 440 = 

### 88%

5 5

Area weighted % BMP implementation = Sum of scores proportional to tract size

% BMP Impl Tract Size % of Total AW %

95 100 20 19

80 35 7 5.6

88 70 14 12.3

100 275 55 55

77 20 4 3.1

Total 500 100 95

= 

### 95%

% of Total = Tract Size / Total Size AW % = % of Total * % BMP Implem entation for each individual tract Area Weighted % BMP Implementa tion = Sum of individual AW %24 Using the spreadsheet:

The spreadsheet is set up so that all that must be entered are the individual percent BMP implementation rates and their respectiv e tract sizes in acres. It will then automatically weight the BMP implementati on scores based on the tract size.



### Reporting

Using the statistical procedures contained in this guide, BM P Implementation data can be reported in the following ways:

- Overall % BMP implementation for the state

- % BMP implementation by landowner group

- % BMP implementation by BMP category

- Area weighted % BMP implementation