1 MEMORANDUM | 30 December 2008

TO Cho

- Yi Kwan and Timothy Roberts, U.S. EPA

FROM Cynthia Manson and Aaron Kamholtz, Industrial Economics, Inc.

SUBJECT Draft Methodology for Assessing Cost

- Effectiveness of Operator Training Options

WA 1

- 19, Task 4

The Environmental Protection Agency's (EPA's) Office of Underground Storage Tanks (OUST) is currently exploring options related to federal rulemaking action(s) associated with implementing provisions of Title XV, Subtitle B of the Energy Policy Act of 2005 (entitled the Underground Storage Tank Compliance Act of 2005). The Energy Policy Act expands eligible uses of the Leaking Underground Storage Tank ( LUST) Trust Fund and includes provisions regarding inspections, operator training, delivery prohibition, secondary containment and financial responsibility, and cleanup of releases that contain oxygenated fuel additives.
In addressing operator training, EP A in August 200 7 developed and published “Grant Guidelines to States for Implementing the Operator Training Provision of the Energy Policy Act of 2005.” These guidelines explain how states receiving Subt itle I funding must implement the operator training provisions in Section 9010 of the Solid Waste Disposal Act. The guidelines outline different levels of training required for three classes of operators , and provide information on topics such as retraining.
EPA is currently considering options for establi shing regulations related to training for operators of underground storage tanks (USTs)

. To assist EPA in this process

, Industrial Economics, Incorporated (IEc) has developed an initial comparative overview of the relative cost

- effectiveness of different operator training mechanisms. This initial overview considers a range of variables including the type of operator being trained, options for live and remote instruction, verification techniques, and program design and implementation costs.

The memorandum first generally discusses cost

- effectiveness in the context of operator training , and briefly discusses the link between an effective operator training scheme and changes in human health and ecological risk. We then frame the analysis with a brief overvi ew of the scope of EPA’s operator training requirements, and a summary of the variables that are most important in measuring program costs and effectiveness. BACKGROUND AND

INTRODUCTION2 The memorandum next provides an overview of training options and preliminary cost estimates, including both unit costs for operators to obtain training and a brief d iscussion of costs associated with curriculum design and implementation. The training options and associated costs are presented in graphic form for each class to provide initial insights into cost

- effectiveness and implementation issues. Finally, the memorandum presents initial qualitative conclusions about the relative cost

- effectiveness of different program

o ptions, with a brief discussion of limitations and areas for additional examination.
Although this memorandum contains preliminary information on both unit (per capita and per class) costs of obtaining training and also on course and test design costs, our assessment generally represents a qualitative comparison of different options, in part because we have been unable to identify state data that tracks the impacts of existing operator training options. As a result, the methodology and data presented he re should be considered a preliminary screen, and not a conclusive cost

- effectiveness assessment.
The first step in a complete assessment of operator training options is to define “cost

effectiveness” in the c ontext of operator training. Ideally, a cost

- effective operator training program would be designed to ensure the following operational improvements:
A more rapid response to emergencies limiting the severity of potential leaks and resulting in potentially less environmental damage.
An improved understanding of statutory and regulatory requirements helping to reduce the number of violations.
Improved long

- term compliance due to fewer human errors resulting in fewer avoidable compliance violations.

These ac hievements, in turn, improve environmental quality (i.e., reductions in human health and ecological risk) by decreasing the number and length of the non

- compliance incidents that are associated with a higher risk of spills, and by effectively responding to spills that occur. A cost

- effective operator training program is one that will ensure an optimal degree of “knowledge regarding operating and maintaining underground storage tank systems.” 1 Typically, cost

- effectiveness is measured in terms of achieveme nt per dollar. The option with the highest level of achievement (e.g., the greatest reduction in

non

- compliance incidents) at the lowest cost is the most cost

- effective option.

NOTE ON ENVIRONMENTA L IMPACTS ASSOCIATED WITH OPERATOR TRAINING Because operato r training programs are compliance tools designed to support existing regulations, they do not provide unique human health or ecological benefits that would not occur if all facilities maintained full compliance with existing regulations. As a result, the ir impacts are not incremental to the potential total benefits of the existing technical standards under RCRA Subtitle I. 2

1 “Grant Guidelines to States for Implementing the Operator Training Provision of the Energy Policy Act of 2005.” EPA, August
2007.

2 Regulatory impact analyses commonly assume full compliance in assessing the prospective benefits and costs of new or revised regulations. COST

EFFECTIVENESS AND

OPERATOR TRAINING3 However, to the extent that operator training increases actual compliance, training can provide measurable benefits related to reduced risk of releases and associated damage. It is important, however, to avoid double

- counting other benefits estimates in measuring operator training benefits. For this reason, at this time we constrain our consideration of effectiveness to changes in compliance behavior and do not attempt to estimate changes in human health or environmental risk. EPA’s operator training regulatory requirements are not yet final, but will be designed to ensure appropriate training for all employees who are involved in compliance and operations at facilities with USTs that are primarily in Tribal Lands and in states that do not comply with EPA’s operator training guidelines or chose to not accept Subtitle I funding. Consistent with the current guidelines, EPA ’s regulation will likely specify training for three different types of operators . At this time, however

, EPA has not yet decided who fits into each operator type :
Class A Operator

– at least one per company. For each organization that owns/operates USTs, EPA will require that at least one operator be trained to understand and implement all regulatory requirements, including, for example, permitting, financial assurance, facility design and equipment standards, and recordkeeping and reporting requirements.
Class B Operator

– at least one per facil ity. For each facility, EPA will require that at least one operator be trained to manage all activities associated with operating the UST, including technical standards, filling and dispensing procedures and equipment, alarms, and reporting and response p rotocols.

Class C Employee

– varies by facility. EPA will require at least that all employees whose work involves UST operations (e.g., cashiers at a gas station who monitor or run the pumps ) be trained in emergency response procedures and other basic ope rations

, such as filling and dispensing procedures

, as needed. 3

C onsistent with provisions of the 2005 Energy Policy Act , EPA’s Grant Guidelines anticipate that all facilities will provide initial training for employees by 2012. Following that date

, Class A and Class B operator s must be trained within 30 days of hire or within a reasonable time frame established by the implementing agency. C lass C employees must be trained before assuming their duties

. 4
EPA’s Grant Guidelines do not specify the specifi c content or delivery mechanism (i.e., in

- person or remote training).

3 Employees who have no role in the sale of gasoline (e.g., staff who work specifically at a co

- located deli ) do n ot typically require training.

4 The specific time frame for initial training has yet to be established for EPA’s operator t raining regularity requirements, but
we assume that it will be similar in structure to the content of the Grant Guidelines in this regard. Note that regardless of when Class A and Class B operators are trained, the owner/operator assumes liability for operations at the time of purchase, ensuring an unbroken chain of liability. ANALYTIC
FRAMEWORK: KEY

VARIABLES AND

ASSUMPTIONS4 With this general framework for the operator training requirements as a basis , t his section first provides a summary of the key variables that are likely to drive the effectiveness and costs of operator training programs . Second, it outlines the main challenges to address when ch o osing an optimal operator training program, and the assumptions we use to help reach a conclusion.

VARIABLES AFFECTING TRAINING EFFE CTIVENESS AND COST We fir st present a general description of the key variables that are likely to drive the effectiveness of training by ensuring that useful information is conveyed to training participants . The list of variables below has been constructe d from several interviews with professional trainers who provide training for UST facilities, and State and Tribal representatives responsible for implementing training at regu lated facilities. The section then documents the main variables driving the cos ts of training options, based on a review of the same sources.

Key Effectiveness Variables The program features of an effective operator training program should have many of the following characteristics:

Initial Transfer of Information (i.e., delivery of training course)

A clear , well

- focused curricul um . An effective operator t raining program will provide the appropriate material to the target operators and employees. EPA’s current guidelines consider a distinct three

- tiered structure, with: at least one Class A operator per company who must know regulatory and compliance requirements; at least one Class B operator per facility who must understand equipment function, operational compliance, and response procedures for any operational pr oblem; and any number of Class C employees who must understand emergency procedures and general operations. The appropriate curricula and training approaches will likely differ significantly for each of these classes of operator

. High quality instructors

. High quality instruction is central to the effectiveness of training options. In

- person training, which can be highly effective because of the opportunity to interact with trainees, is also dependent on the quality of the instruction. The extent to whi ch EPA instructors are licensed or certified may affect both the cost and effectiveness of training programs. In the absence of instructor certification, standardized courses such as those offered on

- line by the International Code Council could potentially represent a cost

- effective option because the quality of the instruction is consistent , though interaction is limited

. Opportunity to ask questions. Training for UST operators involves complex regulatory requirements and comprehensive understanding of facility operations. As a result, the ability of operators to ask questions of instructors will likely measurably affect the effectiveness of training. Live instruction provides a signifi cant advantage in this respect; to a les ser extent , some remote options may5 provide opportunities for question and answer. These options include a Webinar format or the availability of a helpline for employees and operators. A Webinar would allow for a certain degree of interaction between trainer and op erator during the online course.

Hands

- on training for site

- specific information. Since there are many different types of equipment in use across the country, when the training required is specifically related to the equipment used on site, operators are likely to better understand operational requirements if they are able to physically conduct the actions and repeat the steps they are learning about.

Verification of Learning
Ensuring Attendance and Testing

. An effective training program will include step s to verify that operators have attended and successfully learned the content of the course. A number of methods for verifying identity and testing is potentially effective, but formal verification is particularly important for remote training options whe re interaction with instructors is limited.

Retention of Knowledge

A ccessible “refresher” training opportunities

. The r etention of knowledge from training over time is imperfect, and in some cases new technologies and regulatory practices emerge. There fore, accessible and regular re

- training is likely to improve the long

- term knowledge and overall performance of operators.

Key Cost Variables The costs of a training program include both the costs to the operators to obtain the training, and program design and implementation costs (e.g., curriculum development, trainer certification). Below, we summarize the key variables that a re likely to have the biggest impact on the total cost of the operator training program:

Features that Affect Operat or Costs

In

- Person Instruction: Obtaining in

- person instruction, whether on

- site training provided by a certified trainer or a Class A operator, off

- site classroom

- based training, or “remote” training through a Webinar or similar interface, is typically

t he more expensive option for operators than accessing remote training options such as computer

- or on

- line training without live instruction . Costs for obtaining in

- person instruction include time away from work, the cost of instruction itself, and the co st and time associated with any necessary travel (either for the participants or for bringing a trainer on

- site). These costs increase for more remote facilities and facilities that are very small or sole proprietorships.6 Length of Train ing Course

. Training programs can range in length from less than an hour for Class C employees to several days for some comprehensive courses offered in recent years. Longer training curricula require more employee time away from work and in some cases, more travel costs. Again, these costs are likely higher for remote facilities and sole proprietorships

. Retraining requirements. EPA’s current guidelines require that operators be retrained only after a documented compliance issue at a facility . If ad ditional retraining or training updates are required (e.g., at fixed intervals), then total training costs will increase

.

Factors that Affect Program Development and Implementation Costs

Certification of Instructors: An effective system for certifying in structors, including EPA staff, professional “third

- party” instructors, and company representatives (including Class A operator s) who can train other employees in their organizations

, requires the development and implementation of clear requirements, metho ds for ensuring that certified instructors meet those requirements, and potentially the costs associated with maintaining a database of certified instructors.
Development of Remote Curricula: Development of high quality curricula for remote training options ( on

- line or computer) includes content development, programming, beta

- testing, material distribution and support, and periodic content and programming updates.
Development and Implementation of Verification Procedures: Particularly for remote t raining, development of a rigorous means of ensuring that operators have successfully completed training includes design of tests and verification procedures, distribution of materials and support, and, depending on test design, maintenance of database inf ormation on testing.

CHALLENGES TO ASSESSING COST

- EFFECTIVENESS T he variables that affect the costs and effectiveness of training programs are generally clear. However, a number of challenges prevent the development of a clear, general cost

effectiveness ranking of different program options. One difficulty lies in defining the specific options for curriculum design, instructor certification, and retraining and ensuring that both curricula and instructors are of high quality. Even assuming that a range of well

- focused and delivered training options are available to each facility

, however, other variables affect the specific training options that would be the most cost

effective. These include:

Location

- Specific Considerations Facility loca tion may drive up the cost of obtaining training if it is difficult for operators

to reach testing centers , and too costly to bring trainers out for on

- site instruction. Unfortunately, remote locations may also have limited access to high

- speed Internet7 which limits some lower

- cost options. In contrast, operators in urban areas may have a range of low

- cost options. In both cases, on

- site training by company representatives or regulators could provide an attractive option if it is available.

Operator

- Le vel Considerations Each of the three operator classes specified in EPA’s guidelines requires a different type of training, suggesting that different training approaches may be most effective for each. Specifically, Class A operator s required to understand regulatory and compliance requirements may benefit most from a classroom environment with opportunities to learn from peers. In contrast, Class B operator s required to understand operations would most likely benefit from on

- site, hands

- on instruction on the specific equipment that they will manage. Finally, Class C employees, though also likely to benefit from hands

- on instruction, are likely to benefit most from training by a fellow employee (e.g., a Class A or B operator ) who will also be available for follow

- up training and daily operations.

Initial Facility Training and New Operator Training The most cost

- effective options for the initial implementation of training requirements may differ from the options that a facility employs for new hires. For ex ample, it may be

cost

- effective to bring a professional trainer on

- site to provide initial training to all employees at a larger facility. This option is not likely to be cost effective for individual employees, particularly high

- turnover Class C employee s with limited training needs.

ASSUMPTIONS FOR THIS ANALYSIS While EPA has not yet finalized the parameters of its training requirements, we make the following general assumptions in order to frame the discussion of cost

- effectiveness.

Individual Training by Operator Class We assume that EPA will require separate, non

- overlapping training for the three classes of operator

. Class A operator s must understand regulatory compliance issues such as financial assurance, notification, and facility design and equip ment standards. Class B operator s (who at small facilities may also be Class A operator s) will be responsible for general operations, and Class C emplo yees are front

- line employees ( their role is limited to initial response to emergency situations )

.

Prese rvation of Training Options Consistent with the Grant Guidelines , o ur analysis assumes that operators will be able to choose from a range of training options; we therefore do not eliminate discussion of options that are not universally available (e.g., on

- line training). We also assume that state programs consistent with current guidance will not require significant revision to be compliant with final regulations.8 Geographic Range In general, we consider training options without specifical ly assessing geographic dynamics; specific options will be more or less attractive based on location. However, we do specifically consider issues related to the application of training options on Tribal lands.

First

- Party and EPA Trainer Options We assu me that all Class A operator s must receive training from a professional trainer (i.e., third

- party training). However, for Class B operators and Class C employees , we consider the option that a qualified employee (i.e., a first

- party trainer) could provid e on

site training

. For example, a first

- party trainer could be a Class A operator with extensive experience and appropriate Class B operator training

. We also consider the option that EPA inspectors provide training to all classes. We assume that the fu lly

- loaded cost of a GS11 ins pector would be $117,000 ($56 per hour , $448 per day ). 5
EPA’s operator training guidelines includes a range of training options for each operator class; each option has different features and costs. We briefly outline the different options for delivering operator training, and provide preliminary estimates of costs.

To inform our assessment of the cost

- effectiveness of existing operator training efforts, we reviewed a r ange of literature and relevant course information, and also spoke with several people involved with UST operator training, including professional trainers, state personnel involved with implementing and tracking training programs, and Tribal representativ es involved with training implementation. 6

TRAINING OPTIONS Exhibit 1 below outlines the range of possible training options c urrently available for all classes of operator ; some of these options are more appropriate for specific categorie s than others. We then describe each of these options in more detail, and briefly summarize its effectiveness in different contexts.

5 Costs calculated based on the 2007 Bureau of Labor Sta tistics salary for a GS

- 11 employee, with an applied overhead rate of 150% to reflect all overhead costs, including benefits, technology a nd support, and office space). Data available at: http://www.bls.gov/oco/cg/cgs041.htm#earnings

6 Mitch Scheel, OR, personal communication, June 26, 2008; Andrea Zajac, MI, personal communication, June 18, 2008; Ken McBride, personal communication, October 8, 2008; Chuck Tillman, personal communication, September 30, 2008; Robert Largent, personal communication, October 9, 2008; Ben Thomas, Ben Thomas Associates, Inc. Personal Communication, May 27, 2008; Klatt, Bruce, The Ultimate Training Workshop Handbook, 1999. McGraw

- Hill; and International Code Council, “2008 UST/AST Examination Information Bulletin,” Revised May 1 4, 2008. Accessible at: http://www.iccsafe.org/certification/ust

- ast/ OVERVIEW OF

TRAINING OPTION

FEATURES AND

COSTS9 EXHIBIT 1

. AVAILABLE TYPES OF O PERATOR TRAINING

TYPE OF TRAINING GENERAL DESCRIPTION

Hands

- On Training: On

- site (3 rd party) An independent trainer is paid to provide training services to the employees at the location on which they work. Hands

- On Training: On

- site (1 st party) Training provided at the facility by a trained/certified employee within the company for other employees.
Hands

- On Training: On

- site (EPA Trainer) Training is provided on

- site by certified EPA personnel. The trainer could be an EPA UST inspector with training certification.
On

- site Training: Posted notice combined with 1 st party training A Class A or B operator posts instructions for Class C operators in an easily visible location inside the facility reinforcing materials taught in training course.

Off

- site Training: Classroom Operators receive training at a location other than the facility at which they work, provided by a paid, independent instructor. Remote Training: Self

- taught (typically with a test) Operators independently use standard materials (hard copy or electronic) to learn requirements. Method may include a test providing certification to operate. Remote Training: On

- line (typically with a test) Employees login to a website and complete an o n

- line course. Method may include a test providing certification to operate.
Opt

- Out Test Experienced employees complete a comprehensive test on

- line to demonstrate that they do not require additional training.

To optimize training effectiveness, EPA could require a specific level (or levels) of certification for both third

- party and first

- party instructors, to ensure that they are familiar with the regulations and with the range of existing technologies on site at facilities. Both of the first

- party and third

- party training options have the potential to be highly effective given a capable trainer and the appropriate curriculum.

EFFECTIVENESS OF T R AINING OPTION S The appropriateness of training options varies wi th operator class, because each class has different operational requirements. Below we discuss the relative effectiveness of each type of training for specific operator classes:

On

- site (3 rd party): On

- site training provided by an independent trainer would be highly ef

fective for all operator classes . On

- site training is likely to include both standard materials and site

- specific, hands

- on instruction . The combination of live instructor and relevant hand s

- on training provides an opportunity for participan t s to ask questions and “practice” skills learning in the training

. The hands

- on aspect of this training may be less critical for Class A operator s than for other operator classes.

On

- site (1 st party): On

- site training provided by a 1st party trainer is most likely to be effective for Class C emplo yees, who require limited, site

- specific operating information t hat could be provided by a Class B operator. This is particularly reasonable because 1st part y trainers are best posit ioned to train new employees in high

- turnover position s . In limited cases (e.g., larger companies or Tribal organizations with multiple facilities )

, a 1st party trainer may be able to develop10 the expertise to provide on

- site tech nical training to Class A or Class B operator s as well as Class C employees . However, in many cases Class B and Class A operator s could benefit from a n external trainer with updated information on both regulations and on a range of technologies in use at UST facilities. 7

On

- site (EPA Trainer): An EPA trainer could be an UST inspector who has passed a required training certification. Additionally, it could be a staff person who does not have inspector experience, but rather has obtained EPA’s required

cert ification. This option is potentially highly efficient because EPA will be able to verify the trainer has the knowledge needed to provide effective training , and can coordinate training with scheduled inspections

. On

- site training

- Posted not ic es: While ineffective for Class A and Bs, a posted list of instructions, phone numbers, and other important information may be effective for Class C employees, when combined with 1st party training. A posted summary of required knowledge ensures that informat ion is easily accessible when needed, and reduces the need for retraining.

Off

- site training

- Classroom : Classroom training off

- site can be an effective training method for all operator classes , because it can be tailored to employees’ needs and learnin g styles by allow ing for questions and provid ing the opportunity for illustration of certain site

- specific issues. Hands

- on training can also be incorporated into classrooms, though on

- site training for technical issues is likely to be better targeted (e. g., to specific equipment used by participants)

. This method may be most effective for Class A operator s, who are likely to benefit from the ability to interact with both instructors and peers as they address complex issues. Cl assroom training may be mor e difficult to adapt for retraining, where operator s may need only targeted “refreshers.” 8

Remote: Self

- taught: This type of training involves an employee working independently through a curriculum ( either hard copy or electronic), and typically involves taking a test . This approach requires varying curricula and tests for each operator class. It is more likely to be effective for non

- site

- specific topics such as regulatory compliance, and less likely to be effective in providing techn ical inf ormation about site equipment, though a modular approach can target certain types of equipment based on information provided by the user. While this represents the most accessible and flexible training option, a key limitation of this training is the inability of a participant to ask questions or seek clarification on complex topics

.

7 Chuck Tillman, personal communication, September 30, 2008 . The Choctaw Nation in Oklahoma owns multiple travel plazas and Chuck Tillman’s role is in part to ensure that all employees at each facility understand the operating and response requirements.

8 One issue that does not appear to significantly affect the implementation of training is “c heating.” Particularly in “live”

on

- site or classroom training, state officials and Ben Thomas agree that students are typically motivated to make use of the training opportunity, assuming that the curriculum is appropriate. On

- line classes could potenti ally be subject to “substituting” one test

- taker for another, but verification procedures, follow

- on training, and the incentive to avoid non

compliance appear to limit the incidence of this type of behavior as well.11 Remote: On

- line : An on

- line class , generally with a final test

, could be effective for each operator class. However, similar to the self

- administered training, on

line l earning is more likely to be effective for non

- site specific topics such as regulatory compliance, and less likely to be effective in providing technical information about site equipment, where hands

- on training is particularly useful. Similarly, this option limit s the opportunities for students to interact with instructors and seek clarification on complex topics. While this option is flexible and generally accessible, it may also be impractical in areas without high

- speed Internet service (e.g. , some Tribal lands and rural areas). Retraining using this method could potentially be effective, however, beca use those with I nternet

access c ould easily login and retake a course and final exam.

On

- line training options can also include a wide spect rum of more “interactive” features, from simple e

- mail question forums to “live” instructions through Webinars with audio and even video support. Webinar and video

- conferencing options can improve the quality of the training itself by providing access to instructors for questions and interactive exercises and tests ; costs can also be higher

– even similar to live training . At this time “live” remote options are not readily available to UST facility operators, and are likely to face technology constraints similar to or more significant than those associated with on

- line training

– lack of service in remote areas and lack of appropriate software and hardware at smaller facilities. We therefore consider th ese options briefly but do not attempt to specify a specific “live” on

- line approach.
Opt

- Out Test: This training option is a limited application option where current employees can bypass training requirements if they demonstrate knowledge by passing a comprehensive test. The most important factor in ensuring effectiveness of this option is the design of an appropriate test.

In addition to the options laid out above, some “hybrid” options such as annual testing update requirements may be useful in improving the effectiveness of remote options. In the case of annual testing or compliance demonstration, retention is likely improved, and operators can also have access to updated materials as technologies evolve. California currently employs a system where ICC testing is supplemented by annual compliance reporting requirements.

COST ASSESSMENT

– UNIT COSTS TO AN OPERATOR FOR OBTAINING TRAINI NG Exhibit 2 below shows each of the training types and a preliminary estimate of their respective unit cost s to an operator. The values in this tabl e assume that operators make use of existing professional training services (e.g., ICC, Ben Thomas) . Please note

, however, that these costs are based only on a few sources and competing companies offering similar services may have differing cost structures

.12 EXHIBIT 2

. TRAINING METHOD COST S TO OPERATORS

TYPE OF TRAINING ESTIMATED COST

Hands

- On Training: On

- site (3 rd party) $1,000

- $3,000 per class a Per employee costs depend on class size; most efficient for large facilities or multiple local facilities
Hands

- On Training: On

- site (1 st party) Unknown, but generally less expensive than 3 rd party b

Hands

- On Training: On

- site (EPA Trainer) $448 per day, per trainer

On

- site Training: Posted notice combined with 1 st party training $0.50 for notice c ; costs for 1 st party training vary but likely 1

- 2 hours of staff time, and includes other general facility training
Off

- site Training: Classroom $200

- $400 per person, per class a
Remote Training: Self

- taught (typically with a test) $45

- $75 per employee b

Remote Training: On

- line (typically with a test) e $5

- $20 per employee a

Opt

- Out test $45

- $75 per employee b

Notes:

a. Ben Thomas, Ben Thomas Associates, Inc. Personal Communication, May 27, 2008. b. This cost is likely to vary significantly within each company and also may be confidential information.

c. The $0.50 cost is attributable to printing a notice and affixing it to a highly

- visible location. d. International Code Council, “2008 UST/AST Examination Information Bulletin,” Revised May 14, 2008. Accessible at: http://www.iccsafe.org/certification/ust

- ast/ e. Remote training can include live t rainer opti ons such as Webinars. P er

- employee costs associated with Webinar training can be similar to costs associated with classroom training because they require a similar amount of preparation time from two staff members (a trainer and a technical support person)

.

While unit cost information informs the comparison of specific training options, it is important to note that most of these costs are highly sensitive to location. Travel costs can have a significant impact on the relative costs of each of these options. It may in some cases be cost

- effective to bring instructors into a remote region for one or a series of

on

- site training sessions . In densely populated areas, classroom training may be a viable option. The cost of on

- line training or remote self

- instruction is typically lowest in all

c ases, but can be less effective (as noted above) . Hybrid options such as Webinar training can be more effective in providing access to instructors, but can also have cost structures similar to live training options

. 9 In addition, all on

- line training is limite d to areas with effective Internet access.

COST ASSESSMENT

– PROGRAM DEVELOPMENT AND IMPLEMENTATION C OSTS In addition to the costs to operators of accessing training options, EPA’s operator training regulations will likely include signifi cant “start

- up” and implementation costs. Note that costs in this section are “straw

- man” estimates based on our professional judgment and experience developing and conducting similar courses. Therefore, t hey are highly

9 IEc has information on Webinar

- base d training for EPA enforcement staff . T his training is similar in cost to live training, because it requires two staff members (a trainer and a technical support person) and roughly 30 to 60 minutes preparation per participant to ensure scheduling, electr onic connection, and distribution of materials. In addition, enrollment in the course is typically capped at 10 to ensure that participants have ample access to instruction. As a result, the per

- person cost of the Webinar option in this example is simila r to the attending an off

- site live training course.13 uncertain, and depend on the spec ific course content, delivery system, and existing material s that EPA chooses to employ. Significant program development and implementation costs include:
Development of a System for Certifying Trainers : A key aspect of implementing a training requirement is identifying appropriate trainers. A range of certification systems are possible . For example, certification can be “outsourced” to an external company

, and would

- be trainers would be required to obtain certification. This approach would li kely require a competitive contracting effort to identify the best certification company and effort to finalize a certification curriculum, but would not involve significant implementation costs after inception. Other options include providing certificati on through EPA courses, certifying EPA instructors, or adopting certification approaches used by one or more states. The costs associated with each of these options are highly variable; more research on current approaches in other agencies or states mig ht be useful in identifying costs, but it is also necessary to clarify the options EPA would consider for certification

. Development of Curricula for Instructor

- Led Training Options: EPA may opt to design standardized materials to support “live” training options, whether

on

- site or in classrooms, or may simply require that training performed by instructors address all regulatory requirements. The development of training materials is a one

- time cost that would be driven by the materials EPA already has in

- house, and on the extent to which EPA intends to require standa rdized training. Assuming high

- end costs reflect development of all

- new, highly transferrable materials, the cost for this exercise could be $25,000

- $40,000 for each course.
Development and Distribution of Remote Training Materials: If EPA decides to develop training curricula for on

- line or remote training options, the following cost elements are relevant. We use professional judgment based on similar scope projects to provide very rough e stimates of the scale of potential

costs for development of one course (e.g., Class B) that can be published to the Web and distributed remotely . Note that costs for developing a second course would be significantly lower because they would build on the i nitial programming.

o Curriculum Content Development: total cost depends on the extent to which existing training materials exist; $40,000

- $ 8 0,000 , assuming that significant new material must be drafted , reviewed, and tested

. Could be much lower if

formatted materials exist.

o Programming and Design to produce interactive modular training (e.g., training that targets specific technologies based on user input); roughly $ 5 0,000

- $9 0,000. This estimate i ncludes the purchase of site licen ses, programming and formatting of interactive training materials and database s , development of randomized testing materials and a14 certification process using self

- instruction (e.g., not on

- line) , and review and beta

- testing of approach. Estimate highly u ncertain

– could be lower or higher depending on t he software platform EPA uses; assumes limited need for tracking information about trainees. Also includes limited time for suppor t; “hotline” or other support system could vary in cost.

o Distribute Mat erials: $5,000

- $10,000, including web launch and production of CDs for distribution upon request. 10

o Update and Republish Curriculum: $5,000

- $10,000, including updates in programming and materials. Note that cost could be much higher for major revisions

. Development of Procedures to Verify and Track Training Results: Our cost estimates outlined above allow for development of a database to store information from operators accessing training on

- line, but do not assume that EPA will be responsible for tracking or verifying training and certification. Instead, participant s would receive and print out certification for their records once training had been successfully completed. If EPA determines that a more centralized approach to tracking data on operator training is required, this could increase start up and annual mana gement costs substantially.
Again, these costs are highly uncertain, and are designed only to provide a rough scoping estimate of the magnitude of costs associated with establishing a program. Actual design and implementation costs could be much lower if based on existing materials and if EPA expects a limited role in both delivering “live” training and in managing data on certification and training status of trainers and operators, respectively.
Using the cost estimates and qualitative descriptions of effectiveness outlined above, we provide an initial assessmen t of the potential cost

- effectiveness of different training options. Exhibit 3 provides a general summary of the cost

- effectiveness of each training option with n otes about its applicability to different operator classe s. Exhibits 4, 5, and 6 provide more detailed graphical descriptions of the costs and potential effectiveness associated with Operator Training for Class A operator s ( Exhibit 4 ), Class B operator s ( Exhibit 5 ) and Class C employees ( Exhibit 6 ).

10 Note that this cost estimate does not consider the implementation costs associated with conducting Webinars or other “live” options. These costs , if not borne by participants, can vary widely depend ing on course design . C ost elements
include the live instruct or and technical support efforts to ensure that users can connect to the training. COST

EFFECTIVENESS

SUMMARY15 EXHIBIT 3

. AVAI LABLE TYPES OF OPERA TOR TRAINING

TYPE OF

TRAINING COST EFFECTIVENESS COST

EFFECTIVENESS SPECIFIC NOTES GENERAL NOTES

Hands

- On Training: On

- site (3 rd party) $1,000

- $3,000

per class Very high,

particularly for Class B operators Potentially high

but ranges based on travel costs Trainer travel is very

expensive to Tribal facilities, but hands

- on approach is valued Most states will likely make the operators pay for training Operators may be required to retake a class if violations are found at their facilities In Kansas, training is required every 4 years Kansas pays for training through a state pollution fund In California, operators train themselves and then ta ke a exam (similar to the driving permit test) Oregon allows instructions for Class Cs to be posted at the facility when combined with limited 1 st party direction Hands

- On Training: On

site (1 st party) Varies

–

depends on need to certify and travel to locations High if instructor

is certified; small companies may not have in

- house capabilities Potentially high

but limited application “ C ost

- effective within

a company” (Thomas 2008)

Hands

- On Training: On

site (EPA Trainer) $448 per day,

per trainer High if instructor

is certified High Would require EPA to create a trainer certification process

On

- site Training: Posted notice combined with 1 st party training $0.50 in

addition to cost of 1 st party training (e.g., by Class B operator on site) Moderate

- high for

Class C only Potentially high,

likely most cost

effective because information is always available Allows for easy

employee access to important information

Off

- site Training: Classroom $200

- $400 per

person per class Moderate to high,

particularly for Class A Moderate to high Travel may be expensive for rural/Tribal facilities Remote Training: Self

- taught (typically with a test) $45

- 75 (ICC) Moderate

– could be well targeted but no instructor interaction. High Requires verification/testing to ensure application

Remote Training: On

- line (typically with a test) $5

- $20 per

Class C employee; other Class operators likely similar to self

- taught Lower; limited

site

- specific information, limited access to instructor Variable; low for

Class C employees (cost not necessary) “probably the least

cost

- effective option” for Class C employees (Thomas 2008); more cost

- effective for other operators if curriculum is high quality

Opt

- Out test $45

- 75 (ICC) Potentially high High Similar setup to driving permit computer tests; could be used for all classes.
Notes: 1. The information in this exhibit on instructor

- led training is from Ben Thomas, personal communication, May 23, 2008, and verified by communication with program representatives from Oregon and California (Mitch Scheel, OR, personal communication, June 2 6, 2008 and Andrea Zajac, MI, personal communication, June 18, 2008), information from the ICC, and Tribal representatives (Ken McBride, personal communication, October 8, 2008, Chuck Tillman, personal communication, September 30, 2008, and Robert Largent, personal communication, October 9, 2008). Furthermore, in addition to the sources noted above, data in this exhibit is also based on IEc professional judgment. Also note that California and Oregon’s programs predate EPA’s Ene rgy Policy Act Guidelines. 2. The states that have been identified and contacted that are included in this exhibit may not necessarily meet the requirements of EPA’s operator training guidelines.16

EXHIBIT 4

. DETAILED COSTS AND B ENEFITS STRUCTURE FO R CLASS A OPERATOR TRAINING17

EXHIBIT 5

. DETAILED COSTS AND B ENEFITS STRUCTURE FO R CLASS B OPERATOR T RAINING18

EXHIBIT 6

. DETAILED COSTS AND B ENEFITS STRUCTURE FO R CLASS C OPERATOR T RAINING19 As Exhibits 4 through 6 illustrate, more than one training option may be appropriate for operators, depending on the circumstances ( e.g., number of employees requiring training, access to Internet). In general, hands

- on training opt ions are likely to be more eff ective, particularly where site

- specific technical information is central (e.g., for Class B operators ). 11 In circumstances where 1 st party or EPA trainers are available

, the on

- site training option may even be a low

- cost opti on. For Class A operators
, it is possible that classroom training will provide the most effective combination of an in

- person trainer and interaction with colleagues.
In all cases, remote training (either on

- line or distributed electronic training) provid es a flexible, accessible option, but is associated with a greater need for formal verification of operator/ employee identity and successful training (i.e., through a test); in

- person training, in contrast, can include relatively informal procedures for testing participants.
A number of initial insights may be drawn from the information in Exhibits 3 through 6 :

Class B operators and C employees are likely to benefit most from site

- specific and/or on

- site training, because they will need a working knowled ge of facility equipment and operations. Class A operators , in contrast, may be most flexible and able to use options such as on

- line training or off

- site classroom training, at least for the portion of their training that does not require on

- site knowled ge.

Classroom and on

- site 3 rd party training are relatively high cost options, but with qualified instructors, these are likely to be among the most effective approaches, particularly in cases and for operator classes where facility

- specific features are

c entral to compliance, and may be more cost

- effective in situations where multiple employees can be trained on

- site.

The lowest

- cost approach for Class A and Class B operators , in locations where the technology supports it, is on

- line training. An advant age to this approach is the consistency of the curriculum; a drawback is the limited ability to tailor a curriculum to address the specific questions of trainees. The impact of this limitation on the effectiveness of training varies with the curriculum, and with the specific training needs of the owners/operators.

For new Class B operators , remote training options are likely to be the most cost

effective unless a 1 st party on

- site option is feasible within the 30

- day window for obtaining initial trainin g. Anecdotal evidence suggests that the low

- cost “posted notice” approach may be

most cost

- effective for Class C employees when combined with 1 st party on

- site training (or remote training by CD if 1 st party training is unavailable). The high turnover and limited training needs of Class C employees are difficult to address with classroom training or 3 rd party on

- site training, and the fees associated with currently available on

- line training are higher than other remote options.

11 This point is further evidenced by a survey conducted by the Washington State Department of Agriculture on recertification training. “Hands

- on Training Boosts Students’ Retention,” WSDA PesticideNotes, July 2002. Available at: http://agr.wa.gov/PestFert/d ocs/P10PestNotes2002.pdf

.20 First party training is appropriate for Class C employee train ing, and may, in some cases (e.g ., larger companies) be the most cost effective option for the technical training of Class B operators . However, ensuring that 1 st party trainers are certified or otherwise qualified would be an important factor in promoting effectiveness.
EPA trainers, if feasible, would provide the most appropriate and cost

- effective option for post

- violation re

- training, and could also provide a re liable option for fixed

- interval re

- training that coincides with inspections. This option is likely less feasible for training new hires, however.
At this time we cannot derive meaningful quantitative estimates of cost

- effectiveness for different train ing approaches, but Exhibits 3 through 6 provide general comparative information that provides a starting point for assessment of more specific regulatory options.
From the information outlined above, we can draw s everal preliminary conclusions. The first is that for Class C employees, EPA could select a training option that is not costly, since it appears that training options are similar in effectiveness. An approach that combines 1 st party training with a posted notice may be the most cost

- effective training for Class C employees. The second conclusion is that on

- site training is generally likely to be more effective for technical and general operational training than off

- site training. This is likely true for Cla ss B operators and potentially also Class A operators. The third conclusion is that live training in a classroom is potentially the most effective method for Class A operators and could also represent the a cost

- efficient method for Class B operators if th e instructor is capable, has a well

- designed curriculum, and travel costs are modest. The final conclusion is that retraining may be most cost

- effective if implemented by EPA inspectors at the time of inspection and/or after violations.

DATA AND METHODOLO GICAL LIMITATIONS It is important to note that the methodology outlined in this document has several key limitations:

Effectiveness Metrics: This memorandum does not include a measurement of the direct environmental benefits from increased operator traini ng. In addition, we do not present any direct indicator of the environmental benefits resulting from each type of training individually as options. The initial literature search conducted in support of this analysis did not identify any data or studies th at directly document changes in compliance related to training, and we have not yet been able to identify theoretical literature that provides quantitative or ranked comparison of the effectiveness of different training approaches.

Feasibil ity of Training Options: Our methodology for this document did not include collection of data from owners/operators to identify the training options that are most useful and feasible. As a result, our information may not reflect key limitations relevant t o specific circumstances (e.g., Tribal settings, remote facilities, cost

- constrained operators). Appendix A outlines the relevant CONCLUSIONS,

LIMITATIONS AND

NEXT STEPS21 information we were able to collect from the sources we did interview, but a detailed assessment of options should consider (perhaps through public comment) input from owner/operators.
An important limitation to establishing effectiveness of operator training is th e fact that most states’ current operator training programs are too new to have established, well

- kept data for analysis of long

- term program effectiveness. While we received anecdotal indications that training has improved compliance rates, and some exis ting programs may be running well and have increased compliance rates, we were unable to collect any specific data showing compliance trends.

NEXT STEPS Currently, our ability to measure cost

- effectiveness is limited in part by a small base of informatio n sources (a small number of states and training experts) and a lack of available data showing actual changes in compliance linked to training. In addition, with the number of variables, the range of costs and impacts associated with each variable (e.g., travel requirements that affect cost or instructor qualifications and curricula that affect the impact of the program), and the potential for combining more than one option (e.g., adding periodic documentation or re

- training to a specific operator training program), it is not yet possible to clearly rank the options most likely to be cost

- effective. To improve the assessment of cost

- effectiveness, we recommend the following additional analyses:

A concerted effort to assess recent educational research to id entify key limitations/advantages in different training approaches. Emerging research assessing the effectiveness of remote training approaches is likely to be available within several months, and focused effort to coordinate with the researchers to provi de a theory

- based ranking of the effectiveness of different training options may provide useful information. More detailed access to educational literature may also provide insights in how to design specific courses to be most effective. This effort is p articularly important if EPA anticipates selecting or prioritizing a single form of training in its regulations, though.
Development of a mapping tool to assess geographic issues and travel costs. Another useful analysis may be use of spatial data locat ing USTs on Tribal Lands to develop a breakeven cost analysis tool that can identify the impacts of travel costs on training options. This may help EPA compare the cost of sending trainers to facilities to the cost of having employees travel to central lo cations. Depending on data availability, the analysis could also be adapted to identify areas that do not have access to high

- speed Internet.
Expansion of effort to identify reasonable cost estimates for hybrid options and 1 st party training options. Ana lysis of costs associated with training options such as Webinars and 1 st party training could be expanded using input from EPA on the potential structure of the training options and information collected from identified 1 st party trainers.A

- 1 A PPENDIX A:

DATA COLLECTED FROM TELEPHONE INTERVIEWS

BEN THOMAS (BEN THOM AS ASSOCIATES, INC. , MAY 23, 2008 ) Ben Thomas is a private trainer who currently conducts live operator training courses. For more information on Mr. Thomas, see http://www.bentanks. com. Travel is the biggest obstacle for providing live training on Tribal lands.
Ben Thomas was unaware of any fully developed training programs. Most facilities had training for business reasons, not for SPCC or other regulatory requirements (California a nd Oregon are the exceptio ns since they require training).

Believes that on

- site training is the best method available, but that having 3 rd party training can be very expensive for some facilities.
In CA, operators train themselves and take an on

- line exa m. In combination with a monthly compliance report, this may be the most cost

- effective method of training. However, an on

- line exam without continual reporting duties is likely the least expensive and least effective method.
Oregon allows posting instruct ions on the facility walls however there’s no validation it was read.
He has not witnessed any incidents of cheating.
Limited access to high

- speed I nternet limits Tribal facilities’ ability to have on

line training.
Kansas has training every four years f or Class A and B operator s, however operator s have to retake classes if there are violations.
Most states require the operators to pay for the training.

OREGON (MITCH SCHEEL , JUNE 26, 2008 ) Mitch Scheel is the UST Policy Coordinator for the Oregon Department of Environmental Quality.
Have had a training program since 2004 for Class B operators and C employees. In 2009 Oregon will include Class A operator s in the training
program. Ben Thomas solely provides the training, offering courses every 3 mo nths.
Classroom training sessions last eight hours and they are not required to be held

on

- site. The 8

- hour class costs $300.
Employee s /operator s are required to have training one time.
Operators pay for all training costs.A

- 2 Web

- based training is an opt ion, including ICC’s national exam. PEI may eventually offer a similar training program.
The fire marshal requires a posting of the important telephone numbers at the facility.

MICHIGAN (ANDREA ZAJ AC , JUNE 18, 2008 ) Andrea Zajac is the Head of the Stora ge Tank Unit at the Michigan Department of Environmental Quality.
Created an Operator Training Task Force that selected ICC for conducting its testing on

- line

. ICC offers training on

- line and the state is working with various associations for additional i nput in creating its training program.
Michigan and ICC will create a new test based on the new UST regulations that will be required when EPA enacts rules for each new provision of the Energy Policy Act.
EPA grant money is their only funding; there is n o in

- house funding available.

CHUCK TILLMAN (CHOCT AW NATION, OKLAHOMA , SEPTEMBER 30, 2008 ) Unique situation whe re there is no official reservation land; the Tribal boundaries encompass about 10.5 counties.
There are 13 Tribally

- owned plazas and he has no jurisdiction over anything else.
Unless there is something easier, he does all the training.
Turnover can be a problem with class C operators (every 3

- 6 months).
When somebody new is hired, they are required to watch a video that covers alcohol and tobacco sales and it could also cover UST topics such as ATG, alarms, and phone numbers. The video could be administered by a manager or Chuck. Maintenance workers would also watch the video.
These 13 facilities have locked fill caps that the transport drivers ne ed to get unlocked. This requires the employee to be out there verifying the procedures and checking the sumps and gauges before locking the fill pipe at the end of the delivery.

Chuck does on

- site training, and there i s a monthly managers meeting during which new issues can be raised for discussion.
He suggests leaving training class C workers to the managers, who he thinks should have PowerPoint presentations ready for them to view. After viewing the presentations, he thinks they should have to answer a couple questions to show they retained some information.
Chuck estimates there are between 10 and 30 employees at each of the 13 plazas.
One aspect of training he does involves creating and responding to a mock spill

.A

- 3 Chuck is the only C lass A employee, and he attended one of Carl Jeffries multi

day training programs as well as NETI trainings.
Chuck works off EPA grant funding money.
He also provides one

- day training sessions for managers that include 20

- 30 questions for them to answer. Believes that they could institute a PEI internet course costing approximately $75 that would be available for 30 days to log into. Thinks that internet access in his area is not a problem.
Chuck thought that while there could be cheating with remote (onl ine) training options, cheating would likely not be a problem because there would be very little benefit from doing so.
He also stated that sole class A operators should be more accountable and be required to take a 3 or 4

- day training class every year.

C huck noted that he does n o t have a system of tracking th e effectiveness of his training; however

, when he gets calls from facilities, it shows they have a better understanding of their requirements than they previously had.

KEN MCBRIDE ( RED LAKE, GRAND PO RTAGE, GRA ND SIOUX, AND LOWER SIOUX , OCTOBER 8, 2008 ) Has done some Inter Tribal Council of Arizona ( ITCA ) training with EPA in the past. Considers himself a compliance assistant.
Onsite training has been the best.
Has done a couple of classroom classes.

F or 3 years he has offered compliance assistance.
Did a couple 3

- 4 day trainings with Carl Jefferies and ITCA that were geared towards training inspectors.
Ken calls facilities with violations and offers assistance. Not all accept his offer.

It takes a lot of effort for facilities not at Red Lakes to come into compliance.
He mainly tries to deal with the B operators.
Sole proprietorships can be difficult because it i s hard to track down a contact.

He thinks that some facilities have n o t accepted the penaltie s they ha v e been assessed and think there i s still no reason to come into compliance.
It would be difficult to get people to attend a large off

- site training session, mainly because they have no travel money.
Class B turnover is not an issue, Class C turno ver can be.
Thinks trainers have to go to the facilities because the operators are not “book

learners” and would no t get much out of a classroom setting.A

- 4 Site visits are better because each facility has different attributes that can no t all be taught in a larger classroom setting.
He has been putting together 3

- ring notebooks with different sections on facility attributes to make assistance easier for facilities to know what information they have to review.
Recordkeeping is a major problem for Tribal facili ties

- they just do n o t seem to care about keeping up with it.
He said they used to use contractors for training, but the Tribal employees could

no t rel ate to the operators so it did no t work well.

Does no t think many would choose to take online training be cause the y value the hands

- on experience; however

, it should still be an option to choose from.

There are some facilities that do n o t even know what kinds of systems they have, or what kinds of notification and recordkeeping are required.

ROBERT LARGENT (I NTER

- TRIBAL ENVIRONMENTAL COUNCIL , OCTOBER 9, 2008 ) Provides Inter

- Tribal Environmental Council ( ITEC ) training with one other person to 41 Tribes.
All trainings are conducted on

- site.
Used to provide trainings in big conference rooms ; however

, they were n ot efficient

- too much unnecessary information for operators.
The trainings are mostly for As and Bs ; however

, some Cs will sit in with the As and Bs.
Also provides training to maintenance workers onsite.
Unless on state lands, would follow EPA’s regulatio ns. Approximately 75 percent of all facilities have had at least one training.
Have started to make DVDs for class C workers targeting turnover problems (also includes material for As and Bs).
There are only a couple of individually

- owned facilities.

Train ing has been offered to every facility.
The biggest challenge with the classroom trainings was making sure you have their attention and also to only give them necessary information related to their facility.
For onsite trainings, in the morning they would go over paperwork requirements and record

- keeping, and then in the afternoon they would go out and look at the equipment.
Traveling can be difficult for the trainers, and some facilities do no t have a small room that they can even sit down in.

Has found th e hands

- on training to be very effective and more helpful to the operators.A

- 5 In the less

- structured on

- site environment, operators seem to ask more questions.
With the onsite trainings, they get more input from Tribes showing better recognition and awarenes s of the requirements and systems at their facilities.
He has completed some inspector trainings.
They have websites setup up providing useful information to operators.
Did an online training ; however , many did no t have ac cess to the internet so it did

no t work that well.

Did no t have a final test, no incentive for any cheating.
Believes online training is a viable option, provided all targeted operators have internet access.