1 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events 

## Pesticide Applicator Recertification: Verifying Attendance at Training Events

August 2009

The question, “What makes a good recertification offering?” is a many-facetted question. The answer in large part depends on the responder. T he primary goal of recertification offerings may differ for a trainer who provides informati on and education, a regulat or who grants approval and uses recertification as a tool to qualify per sons for a certification or license to apply pesticides and a consumer attending recertification.

While the criteria for proficiency of certified pes ticide applicators are clear ly specified in Title 40 of the Code of Federal Regulations sections 171. 4 & 171.5 (Pesticide Applicator Certification),

there are no criteria defined for the content and evaluation of pest icide applicator recertification programs. Effective training is ve ry important to a pesticide recert ification program, but what are the key elements to a sound and effective progr am? To assist those who provide and manage pesticide applicator recertif ication programs, CTAG devel oped the following guidance

documents:

• Pesticide Applicator Rece rtification: Verifying A ttendance at Training Events

• Pesticide Applicator Recertif ication: Content Criteria

• Pesticide Applicator Recertif ication: Online Training – C ourse Design and Structure

• Pesticide Applicator Rece rtification: Evaluation of Recertification Programs

These documents are intertwined and serve as t he beginning to addressing some critical parts of producing a good recertificat ion program. As CTAG develops more guidance documents, they will be added to this series.



### Introduction

The purpose of any certification pr ogram is to verify that a person has successfully completed a course of study and passed some gauge of compet ency that demonstrates sufficient proficiency to perform a particular task. In the case of pesticide certific ation, this usually involves successfully passing one or more exams. Over the years, CTAG has laid out a considerable body of fact-sheets, guides, and position paper s regarding how to best develop and administer pesticide certification exams. However, to date , CTAG has not addressed the process by which people maintain their certification status by means of recertificat ion workshops, recertification through continuing education unit (CEU) systems, or recertification by on-line web-based2 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events courses. Since this is a very broad area for di scussion, the issue of verification of attendance at recertification workshops and CEU events will be addr essed first. On-line web-based training will be addressed separately.



### Why is this important?

Pesticide certification is a means by which the general public can be given some assurance that the people who use and sell pes ticides have demonstrated a mi nimum level of competency before they distribute, appl y, or supervise the application of a pesticide. For regulators it is a process to demonstrate that a pesticide applicator has proven some competency in understanding the regulations, co mplex environmental and health risks, and many other issues and important factors a ssociated with conducting pesticide rela ted activities. For pesticide manufacturers, it may be a means of mitigat ing risk and ensuring product performance. F Finally, for the applicator or dealer, it is a means of minimizing personal risk, ensuring compliance with federal and state pesticide laws and potentially reducing legal liability (in the event of an incident).

Regardless of the reasons why a per son becomes certified, it should be evident that a pesticide certification must be legitimate and based on a sound and defensible process. A certification must not be granted or renewed if the applic ant has not successfully completed a valid certification exam and/or actively participated in an accredited training progra m. If the certifying agency cannot verify either or both, the credib ility of the entire certif ication process becomes compromised. In this case, the public is not assured that a pesticide applicator is competent, that a risk is adequately mitigated; for the applic ant, the certification serves no purpose other than to maintain a bureaucracy.

On the surface, verifying that someone has parti cipated in a training event seems likeit should be easy; when a person registers for an event, some form of identificati on is presented to the training organizer before admittanc e. Then, during the course of the training, the organizer of the event takes reasonable steps to ensure that the applicant fully participates in the training. In reality, this presents a series of challenges for the regulator y agency and/or the organizer of the event. Examples include but are not limited to:

• Sufficient staff resources and or time to verify identifications at admittance

• Sufficient staff resources to monitor training attendance

• Confidence by the regulatory authority that the (thi rd-party) training organizer is following through with verifications and monitoring attendance

• Resistance by the applicant to producing iden tification on religious or privacy grounds

• Resistance by the applicant for the purpose of avoiding or circumventing the training requirement

• Limitations of technology in the case of par ticipation in an on-line web based training3 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events 

### CTAG’s Goal With Respect To Ve rification of Training Attendance

In previous papers, CTAG has argued that verificati on of who is taking an exam is essential. So too is verification of attendance, but that veri fication should be balanc ed against matters of practicality. So what CTAG will attempt to do, in this discussion paper, is describe methods by which certifying agencies can take reasonable steps to verify atten dance at trainings.



### Is this a significant issue?

Most of the evidence for non-compliance with res pect to attendance at pesticide certification training events is difficult to quantify. Minnesot a has levied fines to people who have falsified attendance signature records. Oregon has purs ued a formal enforcement action against a supervisor in another state agency for falsif ying names of training attendees. Finally, anecdotally, North Dakota has, on multiple occasi ons, observed instances where participants of a training session left the event prematurely when t hey observed identificatio n being reviewed at the registration table. At the time of this writing, no high-profile incidents that have led to actual

harm have been identified.

In other industries or institutions, fraudulent partici pation in trainings have resulted in disbarment for lawyers, loss of a medical license for docto rs, demotions in the armed services, loss of certification status for computer technicians, and in a high-profile incident involving firefighters, lack of positive identification to demonstrate adequate training was thought to be a contributing

factor in the death of nine fire fighters in a June 2007 Charleston, SC furniture store fire.

CTAG is not indicting the existi ng initial pesticide certification system. It is believed that the certification system is a sound process. However, it is also believed that regulatory agencies may be underestimating abuse of recertification proc esses that sustain the initial certification purpose. Therefore, CTAG is advocating that a systematic and reasonable approach be employed to minimize fraudulent participation in mandatory recertification training events or

activities.



### Can you ask for identification prior to admittance to a training session?

Most states have in their statutes provisions that allow them to explicitly or implicitly ask for some form of personal identificat ion. In very few situations t he ability, or authority to ask for positive identification is an impediment. So, the next question is, if you ask to see identification, will a significant number of applic ants resist on privacy, religi ous, or political philosophical grounds?

In the case of pesticide certification it has long been argued that some, especially farmers, would object to being challenged for identification. However, in the 2008 training season, Minnesota, for the first time, requi red that all Private Applicators seeking recertification through attendance at University of Minnesota and Minnes ota Department of Ag riculture sponsored trainings provide proof of ident ification prior to admittance. After thousands of requests, no one lodged a complaint. Apparently, since the terrorist attacks of 9/11 and/or c oncerns about identity4 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events fraud, applicants have conceded that providing positive evidence of who they are to a government authority or the agent of a government authorit y is a reasonable request.



### How practical is it to ask for proof of identification?

It depends. In a small training, 10 to 15 people or less, even a lone trainer operating without the benefit of additional staff can easily check identifications prior to the start of an event. However, as the numbers grow to dozens, hundreds, or even thousands of people, it becomes difficult to check all participants without undue inconvenience to both the organizers and the participants. When dealing with these larger numbers, th e following methods have been employed by sponsoring agencies and/or other cooperating training sponsors:

• Applicators are notified ahead of time that positive proo f of identification will be requested. (This is a forewarning that identif ication is important and the expectation is that it will be checked.)

• Mandatory pre-registration fo llowed by a confirmation letter that is used to remind applicators of attendance expectations and laws pertaining to fraudulent claims.

• Collect the actual identification card and or ce rtification card at the door and return them

to the attendees at the conc lusion of the training.

• Require applicators to place t heir identification on the table at which they are seated so

that monitors can review credentia ls anytime during the training.

• Employ random identification checks before admittance. In th is situation, identification may be checked for one in three, one in fi ve, or one in ten applicators. While this introduces the possibility of slippage, if used in conjunction with pre-notification, it can provide sufficient deterr ence to reduce abuse.

• If asking for identification at the door is not feasible, ask a sampling of the attendees to provide proof of identification later in the day . This not only helps verify attendance, but also can be used to identify people who may leave early.

• A sampling of the signatures obtained at registration can be compared to signatures on file after the training is completed.

Regardless of the method used to verify attendanc e, those who are found to be misrepresenting themselves, should be heavily sanctioned in order to discourage non-compliance.



### Sampling, auditing, and deterrence

If asking every applicator to produce identificat ion is not feasible for efficiency reasons, having a state lead agency monitor every thir d-party sponsored training is not realistic due to resource limitations, or finally, if technol ogy limitations preclude verifica tion of on-line training, using a sampling and auditing approach can be a reasonable alternative.

First, is sampling a valid means of verifying a ttendance? This is a question that goes to the

heart of the science of statistica l analysis. The answer is yes. Essentially, if the sample is representative of the p opulation, then inferences and conclu sions made from the sample can be extended to the population as a w hole. The most difficult part of using sampling is how to determine what constitutes a representative sample.5 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events Second, sampling is valid provided it is r epresentative and there is a sufficient number of samples. The U.S. Internal Revenue Service (IRS ) reviews taxpayer returns by sampling only a small number of filed returns. This is not ju st a random sample, but an adjusted sampling based on income level and “red flag” actions that are reported in returns. (Through IRS analysis these “red flag” actions more often result in non-compli ance.) At a minimum, the IRS samples 0.8% of returns in low to middle income populations and then escalates the sample percentage to 1.66% in higher income groups. “Red flag” actions or people who have a hist ory of non-compliance have a much greater chance of being selected for an audit.

Third, if sampling is to be employed to verify attendance, then the sanctions need to be stiff, not only to punish the offending person, but to send a message to the rest of the population that while you may be able to “get-away-with-it” fo r a while, eventually you will be caught. Going back to the IRS example: for people who willfu lly misrepresent their tax liability, the IRS of course collects the unpaid tax, collects an interest on the unpaid tax, levies penalties of 75% of the underpaid tax, and then can, and does ask the courts to impl ement fines and even jail time.

Using a sampling and auditing approach with verifi cation of attendance for pesticide certification would have the following elements:

• Fair warning—the applicant community must know and understand t hat abuses of the system will not be tolerated. Failure to comply will result in sanctions.

• A sample number needs to be determined. Finding that number should take into

consideration:

o Resources of the regulatory agency

o Number of people in the population

o “Red flag” actions (i.e., pesticide co mplaint history) in the population

o Deferential sampling based perhaps on t he degree of risk associated with noncompliance

• Subjecting the sampled population to an audi t could include, but is not limited to:

o Asking for photo identification at a training event

o Reviewing signatures on training verifi cation forms after a training event

o Reviewing computer logs to determine if time stamp records are unrealistic

o Asking for comprehensive post training evaluations or surveys

• Sanctions for non-compliance for the violator could include:

o Public condemnation via publ ishing names in newsletters

o Invalidation of the actual training

o Limitations on recertification options

o Suspension or invalidat ion of a certification

o Fines or criminal liability

• Deterrence—whatever the sanction, the rest of the applicat or population needs to understand that people who circ umvent the system are caught and suffer legitimate

consequences.6 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events 

### Determining a sample size

There are a host of factors to consider when determining sample size. They are beyond the scope of this paper, but generally the following might be considered:

The first factor is level of confidence desired. Do you want to be 99% sure your sample is an accurate predictor of your population or ar e you willing to accept 95% or even 90%?

Researchers in the social scien ces usually aim for 95% or bette r, but people who do marketing or very rough estimating for sales purposes might accept 90% or better. The higher the number you have (level of confidence), the la rger the sample size you will need.

Another important factor is estimating what per centage of the population has the desired trait or circumstance that you are sampling for. This factor is based on what you know about your population. If you believe that 50% of your c lients falsify attendance records, you would not need a very large sample to discover this. On the other hand, if you estima te that only 2% of your applicators skip recertification trainings, t hen a much larger sample is needed to discover a problem.

Finally, determine the accuracy leve l of your sampling process. If you have a college work-study student doing the sampling who likes to listen to t heir iPod and does not pay attention to details, your error rate could be very high. In this case , you would need a significantly larger sample size to compensate for the high erro r rate. On the other hand, a c onscientious employee who is reviewing records may seldom ma ke errors so a much smalle r sample size could be used.

An example of how this would play out :

A. Define the size of the popul ation you want sampled. (An example would be all the people who participate in a recertification program in a particular year.) B. Determine what level of confidence you want to target. Under most circumstances, a 90 to 95% number should be acceptable since your goal is to verify attendance, not to publish the results in an academic journal. C. Estimate how many people falsify attendance. The best way to determine this is to visit with inspection staff, training staff, and people in the regulated community. The more estimates you have the better, but you need to begin somewhere. So ask people with good judgment and experience and thus you should be able to agree upon a number. D. Accurately predict what your error ra te would likely be when sampling. Generally a conscientious employee will make errors less t han 5% of the time if what they are looking for is clearly defined. E. Once these variables are put together cons ult a statistician to run the numbers and come up with a sample number. Most land grant universities have departments that will do consulting for a very modest fee. Other s ources of help are actual state statistics agencies or state agencies that have statistici ans on staff. Examples would be: state revenue agencies, insurance regulator s, and worker’s compensation bureaus. Additionally, private consult ants may be found in the yellow pages or business resource directories in larger communities and m any also advertise on the internet.7 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events While it may not be possible to determine ahead of time your population size for a given program, it is possible to determine the sa mple size needed for an estimated total annual attendance at training sessions. Base your c onfidence level on the estimated total annual population at training sessions to determine the sample size needed for the confidence level you desire. Then find what percent of the total populat ion the sample represent s. Example: If your total annual attendance at training sessions is 10, 000 and you want a 95% confidence level, find the size of the sample necessary to assure the 95% confidence level for the total population, 10,000. Then determine what percentage the samp le size needed is of the total 10,000. Use this percentage to determine the sample si ze needed at each individual training session.



### Relationship & Expectations Between the Certifying Agency & Third-Party Sponsors

Over 50% of all pesticide certifying agencies (CA) in the US use third-party sponsors (TPS) to conduct recertification trainings. In some sit uations the relationship and expectations between the CA and the TPS is tightly controlled. This is the case in Minnesota where the Department of Agriculture (MDA) and the University of Minnes ota (UM) has strict requirements for agenda development, speaker approval, event operat ion, attendance verification, and follow-up evaluation. In addition, even though a TPS may bear responsibility for organizing the training, the MDA/UM always have representat ives on-site at each training and they are usually part of the program. While this is not a foolproof system, it is perhaps one of the most tightly regulated recertification systems in the country, especially with respect to verifi cation of attendance.

Often this relationship is not as formal as in Minnesota. Instead, the CA basically implements a memorandum of agreement with t he TPS. Below is Michigan’s approach to this. They approve approximately 900 training events per year using this system:

Guidelines for Conducting Training Sessions for Pesticide Recertification Credits

Pesticide applicators in Michigan may obtain pesticide recertification credits to renew their applicator certification cr edential in lieu of retaking t he certification examinations. Currently the Michigan department of Agricu lture (MDA) approves one credit for every two hours of pesticide related training. T he MDA requires that t he training programs include updated educational information es sential to ensure the competence of certified/registered pest icide applicators. The recertificat ion credits are an extension of the public trust that applicator trainees ar e meeting requirement s set by U.S. EPA.

Hands-on activities are approved for one credit for each hour of traini ng, with two credits maximum. Hands-on training may include actual calibration of equipment, problem solving utilizing charts, specimen I.D., tool s, and scenario-type training. This action oriented training must be interactive between the trainer and applicators.

To support the recertification effort by se minar attendance, MDA reserve the right to monitor any seminar approved for credits. Th is ensures program integrity with EPA.8 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events Program Requestors/Hosts: You must check and initial the Request for Seminar Credits form. This will acknowledge that you t he requestor/host agree to the following stipulations:

o You agree to submit the Request for Seminar Credits form at least 30 days prior to

the date of t he seminar.

o You agree to monitor attendance and proc tor MDA recertification form.

o You agree to collect, edit and initial all completed recertification forms from attendees and mail them to the assigned MD A regional office within two weeks

after the seminar.

o You agree to return all unused forms to MDA unless permission is granted to keep

them for future seminars.

o You must agree to allow MDA staff to monitor any portion of a seminar approved for recertification credits free of charge. Refusal to do so will cause MDA to deny or revoke application for recertification credits.

Oregon has similar expectations for TPS trainings as Michigan. Nearly 89% of all recertification trainings in Oregon are conducted by third-party sponsors. Below is Oregon’s specific statement on verification of training attendance:

What are the sponsor’s responsibilities for the course?

Sponsors must make sure attendees who int end to sign the sign-up sheet are present for

the entire course. Attendees w ho are loitering out side the presentation room, reading newspapers, talking on cell phones or otherwis e not engaged in the c ourse, must not be allowed to sign the recertification sign-up sheets. Attendees leaving a session early should not be allowed to sign the sign-up sh eets. Each attendee must sign the sign-up

sheet to receive credit. No attendee should si gn for any other attendee. Course sponsors must return completed orig inal sign-up sheets to ODA within 15 days after course completion for timely posting of attendees’ credit hours. Sponsors should keep a copy of the sign-up sheets for their records.

Sponsors who do not adhere to these require ments, continually submit course information late, do not monitor attendees, or whose courses deviate substantially from the submitted agenda may have credits reduced and risk the denial of accreditation of

future courses.

Ohio established similar requirements and expectati ons with their TPS, but adds a couple of interesting highlights. One is that the sponsoring organization must become pre-approved and the second is transparency. Hi ghlights are listed below:

Guidelines for Program Spons ors of Ohio Applicators

.....Because recertification credits are in lieu of testing, the Ohio Department of Agriculture requires that the re certification credits are offer ed in a credible, educational program that updates the applicator in res ponsible use of pesticide products. The

recertification credits are an extension of the public trust that applicators are meeting requirements set by U.S. EPA for ri sk mitigation of pesticide products.9 of 9 Pesticide Applicator Recertification: Verifying Attendance at Training Events The Ohio Department of Agriculture will require potential vendors of recertification credits to register as a program sponsor. Once approved as a program sponsor, the vendor may submit programs for recertification credit.

In-house training for the purpose of this docum ent is training conduc ted for a place of business for example a pest control distributor or company, a WDI Inspection company, a lawn care distributor or company. All in-h ouse training must be conducted at a neutral site not at the place of business and mu st be open to the general public, including competitors. The speakers must all follow the recertification criteria.

In order to not only verify and monitor attendance on-site, but to also do follow up on quality control, the New York State Continuing Legal Educat ion Board requires sponsoring organizations to submit mandatory evaluations of t heir trainings. This approach could be used in pesticide applicator certific ation programs as well to validate attendance and assess comprehension of recertification training. Below is the Westc hester County Bar Association’s statement about this to lawyers participating in their events:

You will receive an Evaluation Form at the r egistration desk. If you don’t have a form handed to you, please pick one up at the registration table. It is your responsibility to fill out the form, and hand it back to the staff at the registration t able, at the conclusion of the program. Return of the Evaluation Form is a PREREQUISITE to your receipt of a Certificate of Attendance at the meeting. (WCBA is required, by NYS CLE rules, to obtain these evaluations from ALL parti cipants attending EVERY CLE program.)



### Summary

This paper has focused on the reasons why verifica tion of attendance primarily at face-to-face

recertification events is essential to maintain ing the integrity and cred ibility of the pesticide certification system. Verification of attendance is equally important for on- line trainings, but due to the nature of the technology, this issue will be addressed in a separate discussion paper.

Ultimately, it would be most desir able if a CA or a TPS checked identification fo r all proctored events, but in very few instances would this be practicable. In lieu of implementing 100% identification verification methods, the concept of sampling and auditing is presented as a reasonable substitute. Regardless of the exact methods employed to verify attendance, the agency responsible for certification of pesti cide applicators should develop a systematic approach to minimize abuse in the recertificat ion process. Without effective checks and balances in a program, confidence and the integrity of the entire system could be dramatically

undermined.