Document Headings

                    Document headings vary by document type but may contain                    the following:                  
                -                       the agency or agencies that issued and signed a document                    
                -                       the number of the CFR title and the number of each part                      the document amends, proposes to amend, or is directly                      related to                    
                -                       the agency docket number / agency internal file number                    
                -                       the RIN which identifies each regulatory action listed in                      the Unified Agenda of Federal Regulatory and Deregulatory                      Actions                    

                    See the                                          Document Drafting Handbook                                        for more details.                  

###### Department of Energy

###### Federal Energy Regulatory Commission 

          - [Docket No. EL05-24-000] 

*Before Commissioners:*                         Pat Wood, III, Chairman; Nora Mead Brownell, Joseph T. Kelliher, and Suedeen G. Kelly.                    

                    1. In this order, pursuant to section 311 of the Federal Power Act (FPA),[1]       ( print page 344)                        the Commission directs specified control area operators and transmission providers,[2]                                          whether or not they are otherwise subject to the Commission's jurisdiction as a public utility, to complete a survey on their operator training practices.[3]                                          This order implements the findings and recommendations set forth in the U.S.-Canada Power System Outage Task Force's (Task Force) Final Report on the August 14, 2003 Blackout in the United States and Canada (Blackout Report) [4]                                          and benefits customers because better understanding of operator training practices will help to support improvements to overall grid reliability.                 

2. The Task Force found operator performance was one of the root causes of the August 14, 2003 blackout. According to the Blackout Report, deficiencies in operator performance that contributed to the blackout included lack of situational awareness, failure of personnel to declare an emergency, and failure to take appropriate action to ensure that the bulk electric system remained in a secure and reliable state. Participation in the operator training survey is required by this order because it will provide the Commission with valuable information regarding operator training problems that could prevent line outages or improve grid reliability so that we can report to Congress on actions that could be taken to reduce the potential of operator-caused problems.

3.The Commission strongly supports legislative reform to provide a clear federal framework for developing and enforcing mandatory reliability rules. The information collected from the reporting requirement herein will be reflected in a Commission report to Congress on legislation concerning the reliability of the nation's interstate bulk electric systems, consistent with section 311 of the FPA. 

## Background 

4. On August 14, 2003, an electric power blackout occurred over large portions of the Northeast and Midwest United States and Ontario, Canada. The blackout lasted up to two days in some areas of the United States and longer in some areas of Canada. It affected an area with over 50 million people and 61,800 megawatts of electric load. In the wake of the blackout, the Task Force was created to study the causes of the blackout and possible solutions to avoid such future blackouts. On April 5, 2004, the Task Force made publicly available the Blackout Report, which described the blackout investigation findings and identified the causes of the blackout and made recommendations to minimize the future occurrences of large-scale blackouts.

                    5. The Task Force identified FirstEnergy Corporation's (FirstEnergy) inadequate situational awareness, that is, its failure to recognize or understand the deteriorating condition of its system, as one of the four primary causes of the August 14, 2003 blackout.[5]                                          It explained that FirstEnergy's operations personnel were not adequately trained to maintain reliable operation under emergency conditions.[6]                                          In addition, the Task Force stated that significant training above the standards set by NERC is needed to perform system operation and management functions.[7] 

                    6. The Blackout Report also compared the August 2003 blackout with seven previous major outages and concluded that inadequate training of operating personnel was a common factor among major outages.[8]                                          The Task Force concluded that “operating procedures were necessary but not sufficient to deal with severe power system disturbances in several of the events [leading to the blackouts].” [9]                                          It also noted that investigation reports from previous major outages recommended enhanced procedures and training for operating personnel.[10] 

                    7. Responding to the blackout and the blackout investigation, on February 10, 2004, the NERC Board of Trustees approved recommendations to take steps to improve the reliability of the bulk electric system, including a recommendation to improve operator and reliability coordinator training.[11]                                          This recommendation directed that all reliability coordinators, control areas, and transmission operators provide at least five days per year of training and drills in system emergencies using realistic simulations. This training was to be completed by June 30, 2004. On October 3, 2004, NERC issued an update on status of emergency training across the NERC regions.[12]                                          NERC's assessment indicated that all operating entities in three regions met the requirements of the recommendation; all reliability coordinators, in all regions, met the required training; 70 percent of all control areas met the requirement; and 89 percent of all individual operators had completed the training requirements, based upon available data provided by seven of the ten regional reliability councils.                 

                    8. The Final Blackout Report's Recommendation No. 19 supported NERC's near-term training requirements. In addition, the Task Force made several recommendations to improve both near-term and long-term training requirements. An essential element to this recommendation includes commissioning an advisory report by an independent panel to address a wide range of issues concerning reliability training programs and certification requirements. The Task Force concluded that the report should be delivered by March 31, 2005 and that the Commission and Canadian authorities, in consultation with NERC and others, “should evaluate the report and consider its findings in setting minimum training and certification requirements for control areas and reliability.” [13] 

## Discussion 

                    9. The Blackout Report indicates that inadequate power system operator training was a major cause of the August 14, 2003 blackout. Further, the Task Force's analysis of seven other major outages identified operator training as a contributing factor to such outages. It is clear from these reports that a higher standard of training for those that operate the transmission grid is needed to minimize the risk of regional power outages and ensure the uninterrupted flow of electricity in the nation's                           ( print page 345)                        interconnected bulk electric systems. As noted above, NERC requested that all reliability coordinators, control areas, and transmission operators provide at least five days of training and drills in system emergencies using realistic simulations to be completed by June 30, 2004. Although this is a useful step to promote near-term reliability, the Task Force recommended that, in order to improve long-term training and certification requirements, an advisory report by an independent panel should address a wide range of issues concerning reliability training programs and certification requirements.                 

                    10. The Commission has hired a consultant to examine operator training practices. The consultant has prepared the attached survey, which will be submitted to power system operators, as a part of the Commission's effort to determine the breadth of training practices across the industry, identify best practices, and evaluate minimum requirements for an effective operator training program. The Commission will analyze the data and provide a timely report to Congress on the need for legislation to ensure the reliability of the U.S. bulk power system. Accordingly, pursuant to section 311 of the FPA, the Commission is requiring that specified control areas and transmission providers, as specified in Appendix A, (whether or not they are otherwise subject to the Commission's jurisdiction as public utilities) submit the information requested in the survey contained in Appendix B to this order.[14] 

11. Respondents must submit the report by January 31, 2005 to the Commission. 

## Document Availability 

                    12. In addition to publishing the full text of this document in the                     **Federal Register**                    , the Commission provides all interested persons an opportunity to view and/or print the contents of this document via the Internet through the Commission's Home Page (                    *http://www.ferc.gov*) and in the Commission's Public Reference Room during normal business hours (8:30 a.m. to 5 p.m. Eastern time) at 888 First Street, NE., Room 2A, Washington DC 20426.                 

                    13. From the Commission's Home Page on the Internet, this information is available using the                     *e*                    Library link. The full text of this document is available on                     *e*                    Library in PDF and Microsoft Word format for viewing, printing, and/or downloading. To access this document in                     *e*                    Library, type the docket number excluding the last three digits of this document in the docket number field.                 

                    14. User assistance is available for                     *e*                    Library and the Commission's Web site during normal business hours at                     *FERCOnlineSupport@ferc.gov*                     or by calling (866) 208-3676 or for TTY, contact (202) 502-8659.                 

*The Commission orders:*

(A) The specified control area operators and transmission providers, whether or not they are otherwise subject to the Commission's jurisdiction as public utilities, are directed to submit to the Commission, by January 31, 2005, a completed survey of their operator training practices, as discussed in the body of this order. 

                    (B) The Secretary shall promptly publish a copy of this order in the                     **Federal Register**                    .                 

By the Commission. 

Linda Mitry, 

Deputy Secretary. 

## Appendix A—Control Areas and Transmission Providers 

Alabama Electric Cooperative Corp. 

Alabama Power Company 

Allegheny Electric Cooperative, Inc. 

Allegheny Power 

American Electric Power Service Corp. 

American Municipal Electric Power-Ohio, Inc. 

American Transmission Company, LLC 

Aquila Inc. 

Arizona Electric Power Cooperative Corp. 

Arizona Power Authority 

Arkansas Electric Cooperative Corp. 

Associated Electric Cooperative, Inc. 

Austin Energy Company 

Baltimore Gas and Electric Company 

Big Rivers Electric Corp.

Bonneville Power Administration 

Brazos Electric Power Cooperative 

Brownesville Public Utilities Board 

Buckeye Power Inc. 

California Independent System Operator Corp. 

Calpine Corporation 

CenterPoint Energy, Inc. 

Central Iowa Power Coop 

Cincinnati Gas & Electric Company 

Cinergy Corp. 

City of Tallahassee, Florida 

City Utilities of Springfield, Illinois 

Clay Electric Cooperative 

Cleco Power LLC 

Cleveland Electric Illuminating Company (First Energy) 

Commonwealth Edison Company 

Conectiv Energy Supply Inc. 

ConEdison Energy, Inc. 

Connecticut Light and Power Company 

Corn Belt Power Coop 

Dairyland Power 

Detroit Edison Company 

Dominion Generation 

Dominion Virginia Power 

DTE Energy Trading, Inc. 

Duke Energy 

Duquesne Light Company 

East Kentucky Power Cooperative Inc. 

East Texas Electric Cooperative Inc. 

El Paso Electric Company 

Electric Energy, Inc. 

Electric Reliability Council of Texas, Inc. 

Entergy Services Inc. 

Excel Energy 

Exelon Corp. 

First Energy Corp. 

Florida Municipal Power Agency 

Florida Power & Light Company 

Fort Pierce Utilities Authority 

Gainesville Regional Utilities 

Garland Power & Light 

Georgia Power Company 

Georgia Transmission Company 

Grand River Dam Authority 

GridAmerica LLC 

Gulf Power Company 

Hoosier Energy Rural Electric Coop Inc. 

Idaho Power Company 

Imperial Irrigation District 

Indiana Municipal Power Agency 

Indianapolis Power & Light Company 

Indiantown Cogeneration, LP 

International Transmission Company 

ISO New England, Inc. 

Jacksonville Electric Authority 

Kansas City Power & Light Company 

Keys Energy Services 

Kissimmee Utility Authority 

KPL-Westar Energy 

L.A. Dept Water & Power 

Lansing Board of Water and Light 

LGE Energy 

Long Island Lighting Company 

Louisiana Energy & Power Authority 

Lower Colorado River Authority 

Michigan Electric Transmission Company LLC 

Mid-America Interconnected Network, Inc. 

Mid-American Energy Company 

Midwest Independent System Transmission Operator Corp 

MIECO Inc. 

Minnesota Power 

Mirant Americas Development, Inc. 

Mississippi Power Company 

Missouri River Energy Service 

Modesto Irrigation District 

Municipal Electric Authority of Georgia 

National Grid 

Nebraska Public Power District 

New England Electric Transmission Company 

New York Independent System Operator, Inc. 

New York Power Authority 

New York State Electric & Gas Corp. 

North Carolina Eastern Municipal Power Agency 

North Carolina Electric Membership Corp. 

Northeast Texas Electric Cooperative 

Northeast Utilities Service Company 

Northern California Power Agency 

Northern Indiana Public Service Company 

Northwest Power Pool 

Ocala Electric Utility 

Oglethorpe Power Corp. 

Ohio Edison Company (First Energy) 

Ohio Power Company (AEP) 

Ohio Valley Electric Corp 

Oklahoma Gas & Electric 

Oklahoma Municipal Power Authority 

Orange & Rockland 

Orlando Utility Commission 

Otter Tail Power Company 

                        Pacific Corp                               ( print page 346)    

Pacific Gas & Electric Company 

PECO Energy Company 

PJM Interconnection, Inc. 

PPL Utilities 

Progress Energy 

PSEG 

Public Service Company of Colorado (New Century Energies) 

Public Service Company of New Mexico 

Puget Sound Energy, Inc. 

Reedy Creek Energy Services 

Reliant Energy Services, Inc. 

Rochester Public Utilities 

Sacramento Municipal Utility District 

Salt River Project 

San Diego Gas & Electric Company 

Savannah Electric & Power Company 

Seminole Electric Cooperative 

Sierra Pacific Resources Transmission 

Silicon Valley Power—City of Santa Clara 

So. Mississippi Electric Power Assoc. 

South Carolina Electric & Gas 

South Carolina Public Service Authority 

Southeastern Power Administration 

Southern California Edison 

Southern Company Services, Inc. 

Southern Minnesota Municipal Power Agency 

Southwest Power Pool, Inc. 

Sunflower Electric Power Corp. 

Tampa Electric Company 

Tennessee Valley Authority 

Tucson Electric Power Company 

TXU Electric Delivery 

Vermont Electric Power Company 

Virginia Electric & Power Company 

Wabash Valley Power Association Inc. 

WE Energies 

Westar Energy 

Western Area Power Administration 

Western Farmers Electric Cooperative 

Wisconsin Public Service Corp. 

Yadkin, Inc. 

## Attachment B 

This FERC Operator Training Study survey is based upon the template used in DOE STANDARD DOE-STD-1070-94 (June 1994), titled Guidelines for Evaluation of Nuclear Facility Training Programs. 

## Part A—Management and Administration 

1. Which of the following best describes the type of organization in which you work? 

A. NERC Regional Council 

B. Regional Transmission Operator (RTO) 

C. Independent System Operator (ISO) 

D. Integrated Utility 

E. Control Area 

F. Transmission Company 

G. Energy Marketing Company 

H. NERC Reliability Coordinator 

I. Municipal/Cooperative Company 

2. Which of the following best describes your job position within the organization in which you work? 

A. Transmission Operator 

B. Generation Operator 

C. Interchange Operator 

D. Marketing Operations Authority 

E. System Reliability Authority 

F. Balancing Authority 

3. Which of the following best describes your company's policy regarding training? 

A. Training is not available 

B. A training program exists and completion of the program is required 

C. Training program exists but completion of the program is not required but is encouraged 

4. How many staff are dedicated to providing training at your location? 

A. 0 

B. 1 

C. 2 or more 

D. I do not know 

5. Do you have a training department? 

A. Yes 

B. No 

C. I do not know 

6. Do you have dedicated training program developers? 

A. Yes 

B. No 

C. I do not know 

7. Do you have dedicated training program instructors? 

A. Yes 

B. No 

C. I do not know 

8. Does your training program have written program goals? 

A. Yes 

B. No 

C. I do not know 

9. Does your training program have written objectives? 

A. Yes 

B. No 

C. I do not know 

10. Is operational philosophy documented and used at your site? 

A. Yes 

B. No 

C. I do not know 

11. If you are responsible for conducting training, do you perform a routine audit of the training program against the current tasks required to be performed, the administrative requirements concerning system operation, current regulatory requirements affecting the program, and the adequacy of the resources necessary to run the training program successfully? 

A. Yes 

B. No 

C. I do not know 

D. N/A 

12. If you are responsible for conducting training, do you deliver a report to management on the findings of your training program audit? 

A. Yes 

B. No 

C. I do not know 

D. N/A 

13. If you are responsible for conducting training, does management respond positively to the findings of your training program audit? 

A. Yes 

B. No 

C. I do not know 

D. N/A 

14. If you are responsible for conducting training, are you required by your training program to keep records of student performance at the objective level (task, knowledge, and performance level)? 

A. Yes 

B. No 

C. I do not know 

D. N/A 

15. If you are responsible for conducting training, are you required to report on student progress to management at regular or predefined intervals using data? 

A. Yes 

B. No 

C. I do not know 

D. N/A 

16. If you are responsible for conducting training, do you have a process in place that provides quality management of your training program? 

A. Yes 

B. No 

C. I do not know 

D. N/A 

17. If you are responsible for conducting training, does your training program have defined customers (classifications of System Operators or maintenance personnel) to be served by the program? 

A. Yes 

B. No 

C. I do not know 

D. N/A 

18. Does your training program have a published schedule of training program events? 

A. Yes 

B. No 

C. I do not know 

19. Does your training program have a person who is responsible for overseeing the program? 

A. Yes 

B. No 

C. I do not know 

20. Does your training program have a budget process that exists to fund training? 

A. Yes 

B. No 

C. I do not know 

21. Does your training program have a means to keep records for individuals participating in the program? 

A. Yes 

B. No 

C. I do not know 

22. Does your training program have a means to evaluate the effectiveness of the training program? 

A. Yes 

B. No 

C. I do not know 

23. Does your training program have an annual budget sufficient to meet the needs of the training program? 

A. Yes 

B. No 

C. I do not know 

24. Does your facility have a site specific validated Job and Task Analysis that was the basis for the design of your training program? 

A. Yes 

B. No 

C. I do not know 

                        25. Does this validated Job and Task Analysis have a specific Difficulty, Importance,                               ( print page 347)                            and Frequency score associated with it that is used to determine its criticality within the training program?                     

A. Yes 

B. No 

C. I do not know 

                        26. Does your training program have sufficient training aid capability (                        *i.e.*                         overhead projectors, PowerPoint projectors, computer terminals, white boards, mock-ups, etc.) to adequately present the current training material?                     

A. Yes 

B. No 

C. I do not know 

## Part B—Training Staff Qualification 

27. Does your training program have full time dedicated instructors with no other concurrent duties for administration of the training program at your location? 

A. Yes 

B. No 

C. I do not know 

28. How many full time dedicated instructors are assigned to the training program at your location? 

A. 0 

B. 1 

C. 2 or more 

D. I do not know 

29. Does your training program have an Instructor Qualification program at your location? 

A. Qualification is required and completed at our location 

B. Qualification is required but not followed through at our location 

C. Qualification is not required, but will be in the future at our location 

D. Qualification is not required; there is no plan in place for qualification at our location 

30. Does the Instructor Qualification program develop instructor skills required for effective presentation of training materials? 

A. Yes 

B. No 

C. I do not know 

D. No qualification program is in place 

31. Does the Instructor Qualification program ensure that the instructors possess adequate technical qualifications in the subjects they are assigned to teach? 

A. Yes 

B. No 

C. I do not know 

D. No qualification program is in place 

                        32. Does the Instructor Qualification program provide for qualification of each instructor for each of the student training settings in place at your location (                        *i.e.*                         classroom, simulator, on the job training and/or computer based training)?                     

A. Yes 

B. No 

C. I do not know 

D. No qualification program is in place 

33. If instructors who are not fully qualified under your Instructor Qualification program or subject matter experts are used for class presentations are they under supervision and provided guidance by a fully qualified instructor? 

A. Yes 

B. No 

C. I do not know 

D. No qualification program is in place 

34. Does your training program have training workshops for the trainers and evaluators in support of program goals? 

A. Yes 

B. No 

C. I do not know 

35. Does your training program have designated individuals trained to perform objective evaluations of the trainee's performance? 

A. Yes 

B. No 

C. I do not know 

36. In your training program, are the evaluators qualified to a specific standard before they are allowed to evaluate student performance? 

A. Yes 

B. No 

C. I do not know 

37. Does your training program monitor and evaluate instructor performance to determine need for improvement in instructor technical knowledge and/or instructional skills? 

A. Yes 

B. No 

C. I do not know 

38. Do you have a process in place for making changes to your training program? 

A. Yes 

B. No 

C. I do not know 

39. Who can make recommendations for improvements to training? (Select all that apply.) 

A. Trainees 

B. Managers 

C. Supervisors 

D. Trainers 

## Part C—Program Entrance Experience and Education Requirements 

40. Are the majority of candidates for entry into the training program employees from inside your company or new hires from outside your company? 

A. Employees from inside your company 

B. New hires from outside your company 

C. I do not know 

41. For candidates from inside your company, does the training program specify previous job requirements or job specialties for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

42. For candidates from inside your company, does the training program specify any educational requirements for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

43. For candidates from inside your company, is company or union seniority a determining factor for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

44. For candidates from inside your company, does company or union seniority override previous job requirements, job specialties or educational requirements for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

45. For candidates from outside your company, does the training program specify previous job requirements or job specialties for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

46. For candidates from outside your company, does the training program specify any educational requirements for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

47. For candidates from inside or outside your company, which of the following does the training program specify as prerequisites for entry into the training program? (Check all that are applicable to your location.) 

                        A. Physical requirements (                        *i.e.*                         ability to stand, walk or sit for long periods etc.)                     

B. Medical health screening 

C. Psychological screening 

D. Personality screening 

48. For candidates from inside or outside your company, does the training program specify any knowledge screening test requirements for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

49. Are requirements for entry into the training program reviewed and evaluated periodically to ensure trainees capable of completing the course are selected for entry into the training program? 

A. Yes 

B. No 

C. I do not know 

## Part D—Training Program Content Determination 

50. Does your organization have a detailed Job Task Analysis (JTA) for each position staffed at your location? 

A. Yes 

B. No 

C. I do not know 

D. A detailed JTA has not been developed for my location 

51. Does your organization use a detailed JTA to assure that all tasks required for safe and efficient job performance are addressed by a training program at your location? 

A. Yes 

B. No 

C. I do not know 

D. No training program exists at my location 
      ( print page 348)    
52. Is the JTA for your location reviewed periodically and updated as necessary to incorporate changes resulting from procedure changes, facility systems changes, facility equipment changes, changes in job scope, or technological chances that could affect job performance requirements at your location? 

A. Yes 

B. No 

C. I do not know 

D. A detailed JTA has not been developed for my location 

53. Does your training program meet the recommendations of NERC guidelines for recommended training requirements for all positions staffed at your location? 

A. Yes 

B. No 

C. I do not know 

D. No training program exists at my location 

54. Does your training program meet the recommendations of FERC guidelines for recommended training requirements for all positions staffed at your location? 

A. Yes 

B. No 

C. I do not know 

D. No training program exists at my location 

55. Is facility and/or industry experience used to identify training program requirements for your location? 

A. Yes 

B. No 

C. I do not know 

D. No training program exists at my location

56. Are department managers and/or supervisors trained to the same standard as operational personnel at your location? 

A. Yes 

B. No 

C. I do not know 

D. No training program exists at my location 

57. Are technical personnel such as computer maintenance personnel, IT technicians and instrumentation and control technicians trained to the same standard as operational personnel at your location? 

A. Yes 

B. No 

C. I do not know 

D. No training program exists at my location 

## Part E—Training Program Design and Development 

58. Select which of the following best describes Classroom Training at your location? 

A. Classroom Training is required and completed at our location 

B. Classroom Training is required but not followed through at our location 

C. Classroom Training is not required, but will be in the future at our location 

D. Classroom Training is not required; there is no plan in place for classroom training at our location 

59. If classroom training is provided at your location, how much time is allocated to classroom training per year? 

A. One week or less 

B. One to three weeks 

C. Three to six weeks 

D. More than six weeks 

E. N/A 

60. If classroom training is provided at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Classroom training is not provided 

61. If classroom training is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Classroom training is not provided 

62. Select which of the following best describes the Job Qualification program at your location? 

A. Qualification is required and completed at our location 

B. Qualification is required but not followed through at our location 

C. Qualification is not required, but will be in the future at our location 

D. Qualification is not required; there is no plan in place for job qualification at our location 

63. If a job qualification program is provided at your location, how much time is allocated to complete the program? 

A. Six weeks or less 

B. Six to twelve weeks 

C. Twelve to sixteen weeks 

D. More than sixteen weeks 

E. N/A 

64. If Job Qualification is required at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of the qualification program? 

A. Yes 

B. No 

C. I do not know 

D. Job qualification training is not provided 

65. If a Job Qualification program is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of the program objectives? 

A. Yes 

B. No 

C. I do not know 

D. Job Qualification training is not provided 

66. Select which of the following best describes a Job Certification program at your location? 

A. Certification is required and completed at our location 

B. Certification is required but not followed through at our location 

C. Certification is not required, but will be in the future at our location 

D. Certification is not required; there is no plan in place for job certification at our location 

67. If a job certification program is provided at your location, how much time is allocated to complete the program? 

A. Six weeks or less 

B. Six to twelve weeks 

C. Twelve to sixteen weeks 

D. More than sixteen weeks 

E. N/A 

68. If a Job Certification training is provided at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of the job certification program objectives? 

A. Yes 

B. No 

C. I do not know 

D. Job certification training not is provided 

69. If a Job Certification program is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of job certification program objectives? 

A. Yes 

B. No 

C. I do not know 

D. Job certification training is not provided 

70. Select which of the following best describes On the Job Training program at your location? 

A. On the Job Training is required and completed at our location 

B. On the Job Training is required but not followed through at our location 

C. On the Job Training is not required, but will be in the future at our location 

D. On the Job Training is not required; there is no plan in place for On the Job Training at our location 

71. If On the Job training is provided at your location, how much time is allocated to On the Job training? 

A. One week or less 

B. One to three weeks 

C. Three to six weeks 

D. More than six weeks 

E. N/A 

72. If On the Job training is provided at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. On the job training is not provided 
      ( print page 349)    
73. If On the Job training is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. On the job training is not provided 

74. Select which of the following best describes Computer Based Training program at your location? 

A. Computer Based Training is required and completed at our location 

B. Computer Based Training is required but not followed through at our location 

C. Computer Based Training is not required, but will be in the future at our location 

D. Computer Based Training is not required; there is no plan in place for computer based training at our location 

75. If Computer Based training is provided at your location, how much time is allocated to Computer Based training? 

A. One week or less 

B. One to three weeks 

C. Three to six weeks 

D. More than six weeks 

E. N/A 

76. If Computer Based training is provided at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Computer based training is not provided 

77. If Computer Based training is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Computer based training is not provided 

78. Select which of the following best describes Simulator Training at your location? 

A. Simulator Training is required and completed at our location 

B. Simulator Training is required but not followed through at our location 

C. Simulator Training is not required, but will be in the future at our location 

D. Simulator Training is not required; there is no plan in place for Simulator Training at our location 

79. If Simulator training is provided at your location, how much time is allocated to Simulator training? 

A. One week or less 

B. One to three weeks 

C. Three to six weeks 

D. More than six weeks 

E. N/A 

80. If Simulator training is provided at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not provided 

81. If Simulator training is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not provided 

82. Select which of the following best describes Self Study Video training at your location? 

A. Self Study Video Training is required and completed at our location 

B. Self Study Video Training is required but not followed through at our location 

C. Self Study Video Training is not required, but will be in the future at our location 

D. Self Study Video Training is not required; there is no plan in place for Self Study Video Training at our location 

83. If Self Study Video training is provided at your location, how much time is allocated to Simulator training? 

A. One week or less 

B. One to three weeks 

C. Three to six weeks 

D. More than six weeks 

E. N/A 

84. If Self Study Video training is provided at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Self Study Video training is not provided 

85. If Self Study Video training is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Self Study Video training is not provided 

86. Select which of the following best describes Self Study Reading training at your location? 

A. Self Study Reading Training is required and completed at our location 

B. Self Study Reading Training is required but not followed through at our location 

C. Self Study Reading Training is not required, but will be in the future at our location 

D. Self Study Reading Training is not required; there is no plan in place for Self Study Reading Training at our location 

87. If Self Study Reading training is provided at your location, how much time is allocated to Simulator training? 

A. One week or less 

B. One to three weeks 

C. Three to six weeks 

D. More than six weeks 

E. N/A 

88. If Self Study Reading training is provided at your location, are detailed lesson plans or training guides available and used to define the objectives, the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Self Study Reading training is not provided 

89. If Self Study Reading training is provided at your location, do the lesson plans or training guides objectives reflect the task performances, associated knowledge and skills and standards of performance required for successful completion of each lesson objective? 

A. Yes 

B. No 

C. I do not know 

D. Self Study Reading training is not provided 

90. Do you feel that lesson plans or training guides used at your location provide sufficient information and detail to ensure consistent and repeatable training each time they are used? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

91. Do you feel that lesson plans or training guides used at your location provide sufficient information and detail to ensure training meets the expectations of your company management? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

92. Do you feel that lesson plans or training guides used at your location provide adequate standards for evaluating trainee performance? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

93. Do you feel that the evaluation standards are fairly and consistently applied to all trainees? 

A. Yes 

B. No 

                        C. I do not know                               ( print page 350)    

D. Evaluation standards are not used at my location 

94. Do you feel that lesson plans or training guides used at your location provide sufficient information to guide the instructor and trainee in performing and accomplishing the required tasks? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

95. Are lesson plans or training guides used at your location reviewed periodically by subject matter experts to ensure the material contained in the document is both accurate and up to date? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

96. Are lesson plans or training guides used at your location reviewed periodically by training department management to ensure the material contained in the document is both accurate and up to date? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

97. Are lesson plans or training guides used at your location subject to a document control program to ensure that only the latest approved revisions are used for training? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

98. Are lesson plans or training guides used at your location approved by company designated management prior to use in the training program? 

A. Yes 

B. No 

C. I do not know 

D. Lesson plans or training guides are not used at my location 

99. Is a continuing training program in place at your location to maintain and improve the knowledge and skills of workers? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at regular intervals at my location 

100. If continuing training is provided at your location, how much time is allocated to continuing training on an annual basis? 

A. One week or less 

B. One to three weeks 

C. Three to six weeks 

D. More than six weeks 

E. N/A 

101. Is a continuing training program in place at your location to provide refresher training on overtrain tasks at regular intervals? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at regular intervals at my location 

102. Is a continuing training program in place at your location to provide timely training on facility and industry events? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at regular intervals at my location 

103. Is a continuing training program in place at your location to provide timely training on facility and procedure modifications? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at regular intervals at my location 

104. Is a continuing training program in place at your location to provide timely retraining to address performance deficiencies identified on tasks performed at your location? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at regular intervals at my location 

105. Is a continuing training program in place at your location to provide timely training on infrequently performed tasks prior to expected performance of these tasks? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at regular intervals at my location 

## Part F—Conduct of Training Program 

### Initial Job Training 

                        106. At the start of a training program, are you provided with an overview of your training program describing the different phases or methods of training (                        *i.e.,*                         classroom, OJT, simulator, computer based training, laboratory, etc.) that will be required for successful completion of the course?                     

A. Yes 

B. No 

C. I do not know 

107. At the start of a training program, are you provided with an overview of your training program describing the different subjects to be presented and the order of presentation of subjects that will be required for successful completion of the course? 

A. Yes 

B. No 

C. I do not know 

108. At the start of a training program, are you provided with an overview of the learning objectives to be accomplished by the training program? 

A. Yes 

B. No 

C. I do not know 

109. Do you feel that training at your location is presented in a proper sequence to provide completion and understanding of basic or necessary prerequisite knowledge prior to receiving training on more advanced knowledge subjects?

A. Yes 

B. No 

C. I do not know 

110. Do you feel that training at your location is presented in a proper sequence to provide completion and proficiency at basic or necessary prerequisite skills prior to receiving training on more advanced skill level tasks?

A. Yes 

B. No 

C. I do not know 

111. Do you feel that the training materials (training manuals, system descriptions, operating and/or maintenance procedures, operating and/or maintenance manuals, administrative guidelines, etc) you were provided were sufficient to achieve the learning objectives required for successful completion of the training program? 

A. Yes 

B. No 

C. I don't know 

112. Do you feel that the instructor(s) used the reference material provided in the most effective manner during class presentations? 

A. Yes 

B. No 

C. I don't know 

113. Do you feel that the instructor/student ratio during classroom training was adequate to support effective learning? 

A. Yes 

B. No 

C. I don't know 

114. How many students are normally present during classroom instruction at your location? 

A. 1 to 5 

B. 5 to 10 

C. 10 to 20 

D. 20 or more 

115. During classroom training at your location, did the instructor solicit student participation by encouraging student questions during the presentations? 

A. Yes 

B. No 

116. If individualized instructional methods such as computer based training was used was sufficient guidance provided to ensure effective knowledge transfer to the student? 

A. Yes 

B. No 

C. Individualized instructional methods were not used 

117. If individualized instructional methods such as computer based training was used was the information presented by this media generic in nature or specific to your location training? 

A. Generic 

B. Specific to your location training 

C. Individualized instructional methods were not used 

                        118. If individualized instructional methods                               ( print page 351)                            such as computer based training was used was testing or evaluation of your performance on the material part of the program?                     

A. Yes 

B. No 

C. Individualized instructional methods were not used 

119. If testing or evaluation of your performance was part of the computer based training were the results discussed with you by an instructor? 

A. Yes 

B. No 

C. Computer based methods of testing or evaluation were not used 

120. If testing or evaluation of your performance was part of the computer based training and your results were unsatisfactory did an instructor discuss remedial actions that should be taken by you to correct the deficiency? 

A. Yes 

B. No 

C. Computer based methods of testing or evaluation were not used

121. Do you feel that computer based training is a viable option to instructor led classroom training? 

A. Yes 

B. No 

C. I do not know 

D. Computer based training is not used at my location

122. If On the Job training (OJT) is performed is it conducted by personnel trained in the instructional methods of performing OJT? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed at my location

123. If On the Job training (OJT) is performed are the standards and requirements for successful completion of the program explained to you? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed at my location

124. During OJT does the trainee manipulate controls and demonstrate task performance on actual equipment to the extent possible based on operational considerations? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed at my location

125. If manipulation of controls and demonstration of tasks is not possible due to operational considerations during OJT, is a simulated performance or walk-through performed which demonstrates the student's knowledge of the conditions necessary for performance of the task? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed at my location

126. If manipulation of controls and demonstration of tasks is not possible due to operational considerations during OJT, is a simulated performance or walk-through performed which demonstrates the student's knowledge of reference materials, tools and equipment necessary for performance of the task? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed at my location

127. Is student performance on OJT tasks evaluated immediately following the exercise to reinforce the student's performance and offer correction to any problems noted during performance of the task? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed at my location

128. Do you feel that the OJT program training is effective in enhancing the knowledge and performance skills associated with your job? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed at my location

129. If simulator training is included as part of your initial training program, is it conducted at your location or is travel to another location for simulator training required? 

A. Performed at my location 

B. Travel to another facility is required within my company 

C. Travel is required to a facility outside my company 

D. Simulator training is not performed

130. If simulator training is performed is the simulator an exact replica of the actual control stations you are training to use? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

131. If the simulator is not an exact replica of your control station, is sufficient explanation of the differences provided to you to allow correlation of the training back to your actual control stations? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

132. If the simulator is not an exact replica of your control station do you feel that simulator training would be more effective if the simulator was an exact replica? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

133. If simulator training is performed is it conducted by personnel trained in the instructional methods of performing simulator training? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed 

134. If simulator training is performed are the standards and requirements for successful completion of the program explained to you? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

135. Is Simulator training used to teach the trainee to recognize and control normal operations and conditions on your system? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

136. Is Simulator training used to teach the trainee to recognize and control abnormal operations and conditions on your system? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

137. Is Simulator training used to teach the trainee to recognize and control emergency operations and conditions on your system? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

138. Is Simulator training used to teach the trainee to recognize and respond to failures of computer system(s) associated with the process or system function on your actual control stations? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

139. Is Simulator training used to teach the trainee to recognize, interpret, and respond to alarms specific to your system operations conditions? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

140. Is simulator training used to teach the trainee the proper use of normal, abnormal and emergency operating procedures? 

A. Yes 

B. No 

C. Training on the proper use of operating procedures is not performed 

D. Simulator training is not performed

141. Is roll playing used during simulator training to teach the trainee proper interaction with other people, groups or entities? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

142. Is team training used during simulator training to teach trainees how to coordinate their activities with the activities of team members? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

                        143. Is team training used during simulator training to teach trainees the importance of proper and complete communication                               ( print page 352)                            of system conditions and changes to system conditions to other team members?                     

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

144. Is student simulator training performance evaluated immediately following the exercise to reinforce the student's performance and offer correction to any problems noted during performance of the training? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

145. Do you feel that the Simulator training program is effective in enhancing the knowledge and performance skills associated with your job? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

### Continuing Training 

146. Do you feel that continuing training at your location is presented in a manner that is effective in enhancing your knowledge on subjects important to your job? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at my location

147. Do you feel that continuing training at your location is presented in a manner that is effective in enhancing your skills that are important to performance of your job? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at my location

148. Do you feel that the continuing training program is effective in familiarizing you in a timely manner with changes in documents such as procedural changes and system modifications associated with your area of responsibility? 

A. Yes 

B. No 

C. Continuing training is not performed at my location

149. Do you feel that the instructor(s) are fully versed on changes to facility documents and able to communicate the importance and impact of these changes to personnel during training? 

A. Yes 

B. No 

150. Do you feel that the instructor/student ratio during continuing training is adequate to support effective learning? 

A. Yes 

B. No 

C. I do not know 

D. Continuing training is not performed at my location

151. How many students are normally present during continuing training instruction at your location

A. 1 to 5 

B. 5 to 10 

C. 10 to 20 

D. 20 or more 

E. Continuing training is not performed at my location

152. If individualized instructional methods such as computer based training are used for continuing training, is sufficient guidance provided to ensure effective knowledge transfer to the student? 

A. Yes 

B. No 

C. Individualized instructional methods are not used 

153. If individualized instructional methods such as computer based training is used for continuing training is testing or evaluation of your performance on the material part of the program? 

A. Yes 

B. No 

C. Individualized instructional methods are not used 

154. If testing or evaluation of your performance is part of the computer based training used for continuing training are the results discussed with you by an instructor? 

A. Yes 

B. No 

C. Computer based methods of testing or evaluation are not used 

155. If testing or evaluation of your performance is part of the computer based training used for continuing training and your results were unsatisfactory did an instructor discuss remedial actions that should be taken by you to correct the deficiency? 

A. Yes 

B. No 

C. Computer based methods of testing or evaluation are not used 

156. Do you feel that computer based training is a viable option to instructor led continuing training? 

A. Yes 

B. No 

C. I do not know 

D. Computer based training is not used for continuing training 

157. If On the Job training (OJT) is performed as part of your continuing training is it conducted by personnel trained in the instructional methods of performing OJT? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed for continuing training 

158. If On the Job training (OJT) is performed are the standards and requirements for successful completion of the continuing training program explained to you? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed for continuing training 

159. During OJT does the trainee manipulate controls and demonstrate task performance on actual equipment to the extent possible based on operational considerations during continuing training? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed for continuing training 

160. If manipulation of controls and demonstration of tasks is not possible due to operational considerations during OJT, is a simulated performance or walk-through performed as part of the continuing training program which demonstrates the student's knowledge of the conditions necessary for performance of the task? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed for continuing training 

161. If manipulation of controls and demonstration of tasks is not possible due to operational considerations during OJT, is a simulated performance or walk-through performed as part of the continuing training program which demonstrates the student's knowledge of reference materials, tools and equipment necessary for performance of the task? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed for continuing training

162. Is student performance on OJT tasks evaluated immediately following the exercise to reinforce the student's performance and offer correction to any problems noted during performance of the task for continuing training? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed for continuing training 

163. Do you feel that the OJT program as part of the continuing training is effective in enhancing the knowledge and performance skills associated with your job? 

A. Yes 

B. No 

C. I do not know 

D. OJT is not performed for continuing training  

164. If simulator training is included as part of your continuing training program, is it conducted at your location or is travel to another location for simulator training required? 

A. Performed at my location 

B. Travel to another facility is required within my company 

C. Travel is required to a facility outside my company 

D. Simulator training is not performed

165. If simulator training is performed as part of a continuing training program is the simulator an exact replica of the actual control stations you are training to use? 

A. Yes 

B. No 

                        C. I do not know                               ( print page 353)    

D. Simulator training is not performed

166. If the simulator used for continuing training is not an exact replica of your control station, is sufficient explanation of the differences provided to you to allow correlation of the training back to your actual control stations? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

167. If the simulator used for continuing training is not an exact replica of your control station do you feel that simulator training would be more effective if the simulator was an exact replica? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

168. If simulator training is performed as part of a continuing training program is it conducted by personnel trained in the instructional methods of performing simulator training? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

169. If simulator training is performed as part of a continuing training program are the standards and requirements for successful completion of the program explained to you? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

170. Is Simulator training used to teach the trainee to recognize and control normal operations and conditions on your system during continuing training?

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

171. Is Simulator training used to teach the trainee to recognize and control abnormal operations and conditions on your system during continuing training?

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

172. Is Simulator training used to teach the trainee to recognize and control emergency operations and conditions on your system during continuing training?

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

173. Is Simulator training used to teach the trainee to recognize and respond to failures of computer system(s) associated with the process or system function on your actual control stations during continuing training? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

174. Is Simulator training used to teach the trainee to recognize, interpret, and respond to alarms specific to your system operations conditions during continuing training? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

175. Is Simulator training used to teach the trainee the proper use of normal, abnormal and emergency operating procedures during continuing training? 

A. Yes 

B. No 

C. Operating procedure use training is not performed

D. Simulator training is not performed

176. Is role playing used during simulator training to teach the trainee proper interaction with other people, groups or entities during continuing training? 

A. Yes 

B. No 

C. I do not know 

D. Simulator training is not performed

177. Is team training used during simulator training to teach trainees how to coordinate their activities with the activities of team members during continuing training?

A. Yes

B. No

C. I do not know

D. Simulator training is not performed

178. Is team training used during simulator training to teach trainees the importance of proper and complete communication of system conditions and changes to system conditions to other team members during continuing training?

A. Yes

B. No

C. I do not know

D. Simulator training is not performed

179. Is student simulator training performance evaluated immediately following the exercise to reinforce the student's performance and offer correction to any problems noted during performance of the training during continuing training?

A. Yes

B. No

C. I do not know

D. Simulator training is not performed

180. Do you feel that the Simulator training program is effective in enhancing the knowledge and performance skills associated with your job during continuing training?

A. Yes

B. No

C. I do not know

D. Simulator training is not performed

181. Select which of the following best describes Job Re-Qualification program at your location?

A. Re-Qualification is required and completed at our location

B. Re Qualification is required but not followed through at our location

C. Re-Qualification is not required, but will be in the future at our location

D. Re-Qualification is not required; there is no plan in place for job re-qualification at our location

182. If a job re-qualification program is provided at your location, how often must the program be completed?

A. Every Year

B. Every two years

C. Every five years

D. Longer than five years

E. N/A

183. Select which of the following best describes Job Re-Certification program at your location?

A. Re-Certification is required and completed at our location

B. Re-Certification is required but not followed through at our location

C. Re-Certification is not required, but will be in the future at our location

D. Re-Certification is not required; there is no plan in place for job re-certification at our location

184. If a job re-certification program is provided at your location, how often must the program be completed?

A. Every Year

B. Every two years

C. Every five years

D. Longer than five years

E. N/A

## Part G—Training Program Trainee Evaluations and Examinations

185. Does your training program conduct examinations/evaluations to student progress through their initial training program?

A. Yes

B. No

C. I do not know

D. Examination/evaluations are not performed

186. Does your training program conduct examinations/evaluations to student progress during their continuing training program?

A. Yes

B. No

C. I do not know

D. Examination/evaluations are not performed

187. Are examinations/evaluations designed to provide a representative sampling of the knowledge and skills learning objectives presented by your training program?

A. Yes

B. No

C. I do not know

D. Examination/evaluations are not performed

188. Are written test questions, oral evaluations and simulator performance evaluations reviewed by subject matter experts to ensure that technical content, meaning and correct responses are determined prior to administering them to the students?

A. Yes

B. No

C. I do not know

D. Examination/evaluations are not performed

                        189. Does your training program have administrative controls requiring that the content of both written and oral examinations be changed periodically to                               ( print page 354)                            prevent compromise of the material?                    

A. Yes

B. No

C. I do not know

D. Examination/evaluations are not performed

190. Is the development, approval, security, administration, and maintenance of both oral and written examinations and performance evaluations controlled by a program to limit access to the material to only designated personnel to prevent compromise of the material?

A. Yes 

B. No 

C. I do not know 

D. Examination/evaluations are not performed 

191. Is remedial training and reevaluation of students provided by your training program when student examination or performance standards are not met? 

A. Yes 

B. No 

C. I do not know 

D. Examination/evaluations are not performed 

192. Are remedial training plans specified in advance of testing? 

A. Yes 

B. No 

C. I do not know 

D. Examination/evaluations are not performed

193. Is the remedial training program identified to the students and student acknowledgement of the remedial program required prior to testing? 

A. Yes 

B. No 

C. I do not know 

D. Examination/evaluations are not performed

194. Is a method of documenting completion of remedial training provided for in your program? 

A. Yes 

B. No 

C. I do not know 

D. Examination/evaluations are not performed

## Part H—Evaluation of the Training Program 

195. Is your training program structured to provide a systematic evaluation of training effectiveness as it relates to on the job performance by personnel at your location? 

A. Yes 

B. No 

C. I do not know 

D. Evaluations are not performed

196. Is your training program evaluated to ensure that the program conveys all required knowledge and skills to personnel at you location for performance of their duties? 

A. Yes 

B. No 

C. I do not know 

D. Evaluations are not performed

197. Are policies or procedures in place defining the when, how, and by whom of conducting evaluations? 

A. Yes 

B. No 

C. I do not know 

D. Examination/evaluations are not performed

198. Are instructors in your training program evaluated periodically by management against an established set of criteria in all settings in which they provide instruction? 

A. Yes 

B. No 

C. I do not know 

D. Evaluations are not performed

199. Are evaluations of instructors used to ensure consistent instructor performance and/or identify instructional skills in need of improvement? 

A. Yes 

B. No 

C. I do not know 

D. Evaluations are not performed

200. Are trainees provided an opportunity to provide feedback on the effectiveness of the instructor(s) in presentation of training material and the overall quality of the training? 

A. Yes 

B. No 

C. I do not know 

D. Feedback is not solicited 

201. Is feedback from the trainees and the trainee's supervisor after the trainee has had an opportunity to apply his training to actual job duties solicited to help determine the effectiveness of the training provided? 

A. Yes 

B. No 

C. I do not know 

D. Feedback is not solicited 

202. Is the feedback obtained from the trainees and their supervisors used to determine areas in which improvements to the training program are needed? 

A. Yes 

B. No 

C. I do not know 

D. Feedback is not solicited

203. Is a program in place at your location to review changes to procedures, equipment and/or facilities to ensure that changes are reflected in the training program in a timely manner as applicable? 

A. Yes 

B. No 

C. I do not know 

D. Changes are not considered for incorporation into established training plans 

204. Does your training program have in place a policy or procedure to identify required changes to both your initial and continuing training programs and provide guidance on documenting, evaluating, tracking and incorporating changes to your training programs? 

A. Yes 

B. No 

C. I do not know 

D. Changes are not considered for incorporation into established training plans

205. Are the facilities used for training periodically evaluated to determine their adequacy for use as training facilities and to ensure they remain conducive to providing a disturbance and distraction free learning environment? 

A. Yes 

B. No 

C. I do not know 

D. Facilities are not provide for exclusive use of training 

# Footnotes

1.                             16 U.S.C. 825j (2000). Section 311 of the FPA authorizes the Commission to conduct investigations in order to secure information necessary or appropriate as a basis for recommending legislation. Section 311 makes clear                                                 that the Commission's authority in conducting such investigations extends to entities otherwise not subject to the Commission's jurisdiction “including the generation, transmission, distribution and sale of electric energy by any agency, authority or instrumentality of the United States, or of any State or municipality* * *.” “The Commission shall report to Congress the results of investigations made under authority of this section.” 16 U.S.C. 825j.                    
        Back to Citation      

2.                             A list of survey respondents appears in Appendix A to this order. The Commission has hired a contractor to conduct this survey. The contractor will contact all survey respondents with instructions on how to complete the survey.                    
        Back to Citation      

3.                             A copy of this survey is found in Appendix B to this order and at                         *http://www.ferc.gov/​industries/​electric/​indus-act/​reliability/​2004-sys-op-survey.pdf.*
        Back to Citation      

4.                             The Blackout Report, which was made public on April 5, 2004, is available on the Commission's Web site at                         *http://www.ferc.gov/​cust-protect/​moi/​blackout.asp.*
        Back to Citation      

5.                             Blackout Report at 19. The other primary causes identified by the Task Force were inadequate system understanding by FirstEnergy and the East Central Area Reliability Coordination Agreement (ECAR), a North American Electric Reliability Council (NERC) Regional Reliability Council, failure to adequately manage tree growth in transmission rights-of-way, and failure of the interconnected grid's reliability organizations to provide effective diagnostic support.                         *Id.*                         at 17-20.                    
        Back to Citation      

6.                                                     *Id.*                         at 19.                    
        Back to Citation      

7.                                                     *Id.*                         at 20.                    
        Back to Citation      

8.                                                     *Id.*                         at 107.                    
        Back to Citation      

9.                                                     *Id.*                         at 110.                    
        Back to Citation      

10.                                                     *Id.*
        Back to Citation      

11.                                                     *See*                         Recommendation 6. The text of the February 10, 2004 document is available on NERC's Web site,                         *http://www.nerc.com.*
        Back to Citation      

12.                                                     *See*                         Status Report on Recommendation 6a at                         *ftp://www.nerc.com/​pub/​sys/​all_​updl/​docs/​blackout/​Recommendation_​6a.pdf*
        Back to Citation      

13.                             Blackout Report at 157.                    
        Back to Citation      

14.                             OMB Control Number 1902-0209; expiration date June 30, 2005.                    
        Back to Citation      

[FR Doc. 05-18 Filed 1-3-05; 8:45 am] 

BILLING CODE 6717-01-P