OSHA is the federal agency responsible for administering the Occupational Safety and Health Act of 1970, as amended (OSH Act), which was enacted to assure safe and healthful working conditions for the nation’s workers. As authorized by the act, OSHA sets and enforces occupational safety and health standards, which are regulations that set forth specific safety and health requirements with which covered employers must comply. Among its other responsibilities under the OSH Act, OSHA is also required to provide for the establishment and supervision of training programs to help workers and employers recognize, avoid, and prevent workplace safety and health hazards. The Outreach Training Program is the agency’s primary way to offer training for workers in the basics of occupational safety and health, according to OSHA. Outreach Training includes voluntary 10-hour and 30-hour safety courses designed to provide basic hazard awareness training for workers in construction, maritime, and general industry, and a 15-hour course for workers at disaster sites. The training covers how to recognize and prevent hazards on a jobsite, workers’ rights, employers’ responsibilities, and how to file a complaint. The 30-hour courses are intended to provide more in-depth training to workers who have some safety responsibility. Workers that successfully complete an OSHA Outreach Training Program course receive a student course completion card. The student course completion cards in Construction, General Industry, and Disaster Site do not have an expiration date; however, the Maritime Outreach Training student course completion cards expire 5 years after completing the training. While OSHA does not require workers to take its Outreach Training courses, depending on their jobs and where the workers are located, they may be subject to other requirements to show an OSHA course completion card to demonstrate they completed the training. For example, some states, municipalities, unions, and employers may require workers to take an OSHA Outreach Training course as a condition of employment for certain jobs, and OSHA maintains a list of states and municipalities that have such a requirement. According to OSHA’s list, seven states (Connecticut, Massachusetts, Missouri, Nevada, New Hampshire, New York, and Rhode Island), and three municipalities (Miami-Dade County, New York City, and Philadelphia) require certain workers to take some form of OSHA Outreach Training or an equivalent training. For example, all of these states and municipalities require certain workers on public construction projects to take a 10-hour construction safety course. Some, including Nevada and Philadelphia, also require this training for workers on private construction projects. Outreach Training courses are delivered by OSHA-authorized external training providers that receive no funding from OSHA, but instead, rely on tuition and fees from training participants to cover the cost of the training. The types of training providers include Education Centers, which train Outreach trainers to deliver in-person Outreach Training courses to workers, and online training providers. Education Centers: OSHA has nonfinancial cooperative agreements with 27 Education Centers–which are typically universities–to provide occupational safety and health training to workers and employers on behalf of OSHA. The Education Centers have two main roles under the Outreach Training Program: (1) conduct in-person train-the-trainer courses to qualified individuals interested in becoming authorized Outreach trainers; and (2) act as an authorized training organization for the Outreach trainers by monitoring Outreach trainers through records audits and training observations and by processing course completion cards that Outreach trainers request on behalf of the workers who take their courses. OSHA selected the Education Centers through a formal competitive application process. Successful applicants entered into 5-year nonfinancial cooperative agreements with OSHA. Outreach trainers: For in-person training, Education Centers authorize Outreach trainers to provide the training to workers. To become an authorized Outreach trainer, individuals must meet industry safety experience requirements, complete a training course on the applicable OSHA standards, and successfully complete a train- the-trainer course at one of the Education Centers. Outreach trainers are authorized to provide in-person training to workers for 4 years from the date that the trainer course is completed and must successfully complete an update course at an Education Center to maintain authorization. Currently, there are approximately 30,000 authorized Outreach trainers, according to OSHA officials. Online training providers: Online training is provided by nine OSHA- authorized online training providers, including eight companies and one Education Center. OSHA used a multi-phase process to select the current online training providers that is described in OSHA guidance. The guidance provided the requirements for establishing an online Outreach Training course and a process for OSHA authorization. The phases of the process included: (1) submission of a description of the online training program, target audience, and expected impact of the program; (2) submission of a detailed written description of the online training plan; (3) self-verification audit where the online training provider reviews the training against OSHA’s guidance; and (4) OSHA’s review of the completed online training to verify that the program is consistent with OSHA’s guidance. In recent years, OSHA has attempted to change its process for selecting the online training providers, but those attempts have been challenged in court. In 2011 and 2014, the agency issued public solicitations, intending to enter into nonfinancial cooperative agreements or contracts with selected providers. Both solicitations were found to have procedural deficiencies by the U.S. Court of Federal Claims after online training companies challenged them in court. According to OSHA officials, the agency has revised the solicitation to address the court rulings, but no date has been set for issuing the revised solicitation. Currently, the nine online training providers provide OSHA Outreach Training under their 2009 authorization. Participation in the Outreach Training Program has grown substantially over time. The number of workers trained more than quadrupled from 200,522 in fiscal year 2000 to 900,010 in fiscal year 2016. The vast majority of participants take the courses in-person; however, the number of participants taking the courses online has increased in recent years (see fig. 1). In fiscal year 2016, 70 percent of participants took the Outreach Training courses in person, and 30 percent took the courses online. The number of participants taking the courses online increased from 19 percent in fiscal year 2011 to 30 percent in fiscal year 2016. The courses tailored to the construction industry are the most popular. In fiscal year 2016, about 75 percent of the Outreach Training participants took the construction courses, and 24 percent took the general industry courses (see fig. 2). OSHA’s Outreach Training Program reflects attributes of a well-designed training program. GAO’s training guide identifies attributes of a well- designed training program and suggests the kinds of documentation to look for that indicate that a training program has a particular attribute in place. OSHA is not required to follow GAO’s training guide, however, we found that the program’s design reflected at least one indicator for six of the seven attributes of a well-designed training program described in the guide (see table 1). OSHA took steps to design the Outreach Training Program so that workers receive consistent and quality training by using data to identify the content of the training, developing training materials, and issuing detailed requirements for training providers. According to OSHA officials, the content of the training was selected after the agency reviewed data on the leading causes of worker deaths and the most frequently cited OSHA standards. OSHA also developed training materials that training providers can use, including presentations, lesson plans, fact sheets, and tests. In addition, OSHA developed detailed requirements regarding the content and delivery of the training that apply to all Outreach Training providers to improve the quality of the courses and ensure the integrity of its authorized trainers. Specifically, in 2011, OSHA replaced its previous program guidelines with a revised policy document establishing requirements that apply to all Outreach Training providers, including in- person and online training providers. OSHA issued general Outreach Training Program Requirements and separate Industry Procedures for the Construction, General Industry, Maritime, and Disaster Site courses. The requirements and industry-specific procedures identify the learning objectives for the courses, the topics that must be covered, and the amount of time training providers must spend on each topic, among other things. OSHA most recently revised the Outreach Training Program Requirements and industry-specific procedures in January of 2017. OSHA used a combination of centralized and decentralized approaches to design the Outreach Training Program. GAO’s training guide suggests agencies consider the advantages and disadvantages of using centralized and decentralized approaches to designing training programs. Centralizing design can enhance consistency of training content whereas a decentralized approach to training design can enable agencies to tailor training programs to better meet the unique needs of the intended audience. OSHA used a combination of approaches by centrally setting the learning objectives and establishing detailed content requirements, while also allowing training providers to supplement the curriculum to meet the specific needs of their audience. For example, while OSHA requires all Outreach Training courses to include 2 hours of introductory information about OSHA, training providers have some flexibility to develop the rest of the curriculum as long as they cover the required topics and meet the time requirements. Table 2 provides a summary of OSHA’s topic and time requirements for the 10-hour construction Outreach Training course. In addition, although OSHA does not require any accreditation for Outreach Training providers, OSHA’s Directorate of Training and Education, which designed the Outreach Training Program, and several of the online training providers, reported that they are accredited by the International Association for Continuing Education and Training (IACET). IACET accreditation involves an audit of the organization’s training program that benchmarks the organization against a national standard called the IACET Standard for Continuing Education and Training. IACET officials review the training provider’s application and document that the standard is being followed by the organization and that the attestations in the application are accurate by conducting a site visit. IACET-accredited training providers must demonstrate that they have processes in place to ensure that the individuals involved in training design are qualified and that the content and instructional methods used are appropriate for the learning objectives of the training, among other things. According to OSHA officials, having OSHA’s Directorate of Training and Education become IACET-accredited as a continuing education provider demonstrates the agency’s commitment to high standards for the agency’s training programs. The Education Centers we interviewed had a different form of accreditation associated with their universities. To obtain course completion cards, which training providers distribute to workers who complete the training, training providers must submit class documentation through either an automated or paper-based process. Outreach trainers–who deliver in-person training to workers–request cards by submitting class documentation electronically through a web- based system to the Education Center where the trainer was authorized. If approved, the Outreach trainers are sent wallet-sized plastic cards for the workers, each with a Quick Response (QR) code that can be scanned with a smart phone, allowing employers and workers to verify the authenticity of the card. In contrast, online training providers mail class documentation, such as student names, course completion dates, test results, and student evaluations of the courses, to OSHA to obtain cards for the workers who take the training online. If approved, the online training providers are sent wallet-sized paper cards for the workers. The online training provider then prints the student names on the paper cards and distributes them to the workers (see fig.3). OSHA typically processed the course completion cards within 2 weeks of receiving the request from the online training providers, according to our analysis of OSHA data. OSHA has 30 days to process course completion cards after receiving an online training provider’s request for cards, according to an internal deadline set by OSHA. In our analysis of OSHA fiscal year 2012-2016 data, we found that OSHA took 7 days on average to process the cards, and the processing time ranged from processing the cards the day OSHA received the request to 51 days. Two-thirds of the card requests were processed in a week or less, and 91 percent were processed within 2 weeks. However, six of the nine online training providers we interviewed said OSHA’s paper-based system for processing course completion cards contributed to delays in issuing cards to workers. For example, one online training provider said it mails OSHA between 200 and 300 pages of class documentation on a bi-weekly basis and it takes OSHA up to 4 weeks to mail the cards back to them. Three of the nine online training providers we interviewed gave examples of how lengthy processing times can negatively impact workers. One online training provider that serves younger workers said delays can be problematic, especially at the end of the school semester, because if the student does not get the course completion card in time, he or she may not have the required documents needed to start their summer job. Another online training provider said that waiting for OSHA to process the cards causes anxiety for many students, as 3 or 4 weeks is a long time to wait for something required before they can get a job. A third online training provider said there have been instances where their students could not start working on the job site because the student did not have their course completion card and the employer or union refused to accept the online training provider’s certificate of completion. While online training providers have noted these issues, the two national organizations we interviewed that represent employers and unions said that they have not received complaints from workers regarding delays in getting their course completion cards or that delays affected their ability to start working at their job site. Differences in the process for issuing course completion cards stem from the history of the program, according to OSHA officials. When OSHA began partnering with the Education Centers to conduct OSHA Training Institute courses in 1992, the agency required individual Outreach trainers to request course completion cards for workers through the Education Center where they received authorization to serve as trainers. According to OSHA officials, in 2001, when OSHA began allowing the 10- and 30- hour construction and general industry Outreach Training courses to be delivered in an online format, OSHA decided to serve as the authorizer for the online providers. OSHA officials told us that they are planning to allow the online training providers to request cards electronically by having the Education Centers take over processing the course completion cards for the online training providers, but OSHA has not established a timeline for implementing this new process. According to OSHA officials, the agency plans to include a process for electronically requesting course completion cards in conjunction with the revised solicitation for selecting the online training providers that OSHA is finalizing. OSHA also intends to make the plastic cards available to workers who take the courses online. Having the Education Centers issue course completion cards to online providers may help to speed the time for issuing these cards to workers. However, the online training providers we interviewed generally said that they believe that they could issue the cards more quickly if they issued the cards themselves. OSHA officials told us they considered allowing the online training providers to process and print their own OSHA cards but decided not to because they want to limit the number of organizations that have access to the course completion cards. Training providers and OSHA officials reported using automated and manual checks to help ensure that cards are requested only for the students who successfully complete the courses and that the courses meet OSHA’s Outreach Training Program requirements. Both the Education Centers and online training providers reported using information systems that automatically check whether the students have taken courses that met OSHA requirements, such as spending the minimum amount of time on specific topics. Some of the online and in- person training providers reported also using staff to manually verify the information in the class documentation submitted to obtain course completion cards. The online training providers reported using user names and passwords to verify that the person registered for the course is the person completing the course. In addition, some of the online training providers reported that they are capable of using biometric data, such as voice recognition or keystroke analysis, to authenticate students. OSHA manually compares the number of cards requested by the online training providers to the number of student names listed in the class documentation and logs the unique course completion card numbers into a spreadsheet. Use of internal controls in the course completion card process is important to protect the integrity of the program. OSHA works with DOL’s Office of Inspector General to investigate and address fraudulent activity. According to OSHA officials, there have been relatively few cases of fraud in the program. OSHA refers fraudulent activity to DOL’s Office of Inspector General, and trainers caught falsifying information may be subject to criminal prosecution. DOL’s Office of Inspector General investigated four cases of fraud involving OSHA training over the last 3 years, according to Office of Inspector General officials. Examples of fraud cases they identified include an Outreach trainer selling course completion cards to construction workers without providing the training and an individual conspiring with an Outreach trainer to sell fraudulent OSHA course completion cards. OSHA oversees the in-person training providers on an ongoing basis by collecting and assessing data from the Education Centers, including: Course information: OSHA collects data on the train-the-trainer courses the Education Centers deliver to individuals interested in becoming authorized Outreach trainers, including the names of the courses presented each month, training locations and dates, instructor names, total students, total instructional hours, and test results. Course completion card processing: OSHA collects data on the number of course completion cards the Education Centers process each month for their authorized Outreach trainers. Monitoring activities: OSHA collects data on the number of record audits and training observations the Education Centers conduct on their authorized Outreach trainers to check that the in-person training delivered to workers complies with OSHA’s Outreach Training Program requirements and procedures. OSHA assesses the Education Centers’ performance on 11 performance elements (see table 3), which relate to the Education Centers’ Outreach Training Program responsibilities and their other responsibilities outlined in their nonfinancial cooperative agreements with OSHA. OSHA uses the data it collects from the Education Centers to assess the extent to which the Education Centers are meeting their performance goals in an annual appraisal. OSHA rates the Education Centers on a three-point scale of “outstanding,” “satisfactory,” or “area for improvement” for seven performance elements and on a two-point scale of “satisfactory” and “area for improvement” for the remaining four performance elements. For example, in OSHA’s appraisals for fiscal year 2015, OSHA rated 22 of 27 Education Centers an “outstanding” or “satisfactory” rating on their Outreach trainer monitoring and identified this as an “area of improvement” for 5 Education Centers because they did not take certain actions, such as conducting enough records audits and training observations (see fig. 4). OSHA follows up with the Education Centers that have received “area for improvement” ratings to ensure they are taking corrective action. OSHA checks whether corrective actions are implemented by conducting quarterly course documentation audits, annual conference calls, and periodic on-site audits, as well as by developing mid-year status reports. In addition, the program coordinator for the OSHA Training Institute Education Center Program interacts with the Education Centers throughout the year to ensure they are meeting performance goals, according to OSHA officials. OSHA also investigates complaints it receives about Outreach trainers from the general public, referrals from the Education Centers based on their Outreach trainer monitoring, and requests by other OSHA regional offices. OSHA investigates allegations that an Outreach trainer may not be in compliance with OSHA’s Outreach Training Program Requirements and allegations of potential fraud. For example, OSHA may investigate cases where an Outreach trainer fails to respond to audit requests from an Education Center or fails to provide training that meets OSHA’s requirements, such as spending enough time on required topics. OSHA will notify the Outreach trainer of its findings from the investigation and allow the trainer 15 days to address the findings identified in the notification. If the trainer does not respond within this timeframe, OSHA may suspend the trainer indefinitely until a response is received. The trainers can appeal OSHA’s initial decision. If there is an appeal, OSHA will review it and make a final decision, which may result in the Outreach trainer being subjected to disciplinary actions, including probation, suspension, or revocation of the Outreach trainer authorization. As of October 2016, 107 Outreach trainers, which is less than 1 percent of the approximately 30,000 Outreach trainers, have had their status as authorized Outreach trainers suspended or revoked by OSHA as a result of failing to comply with the Outreach Training Program requirements and procedures. OSHA officials said that they oversee the online training providers by collecting updated course information twice per year and investigating any complaints about the training. In 2009, OSHA required that online training providers complete an initial multi-phase process to develop online Outreach Training courses that meet OSHA’s requirements. According to OSHA officials, after they initially approved the online training providers, OSHA monitoring has consisted of responding to complaints about the training, requiring online providers to submit updated information on their courses twice per year, and requiring that providers self-certify twice per year that their training meets OSHA’s requirements. OSHA investigates complaints about the online training providers by first sending them a notification letter describing the issue. Examples of issues OSHA has investigated include cases where an online provider incorrectly advertised the courses or offered the training outside of OSHA’s geographic jurisdiction. The online provider has 15 days to respond to OSHA, and if OSHA determines they have violated applicable requirements, OSHA may suspend, terminate, or put them on probation. Online providers who are placed on probation are required to submit monthly self-certifications to OSHA that their program is in compliance with OSHA’s online training guidance. OSHA did not establish a performance assessment process for online training providers when the current providers were selected in 2009, but according to officials, the agency is in the process of finalizing a public solicitation for a new model for authorizing online providers that they said will include new requirements for the providers. OSHA officials said that having these new requirements in place for the online training providers will improve the providers’ understanding of what is expected of them and help OSHA gather additional information on the providers’ performance. OSHA tracks the number of workers who take the Outreach Training courses to measure the reach of the program. OSHA measures the results of the Outreach Training Program by setting yearly targets for the number of workers trained and uses these data to report its progress in meeting its strategic goal of improving workplace safety and health. While the agency missed its target for fiscal years 2011 and 2012, it exceeded its targets for fiscal years 2013 through 2016 (see fig. 5). OSHA obtains information on the number of workers trained from the Education Centers and online training providers. OSHA also requires some of the Outreach Training providers to evaluate the training they provide using the Kirkpatrick model—a commonly accepted model for evaluating training—which is endorsed by the Office of Personnel Management in its training evaluation guidance and GAO’s training guide. The Kirkpatrick model consists of a four-level approach for soliciting feedback from training course participants and evaluating the impact the training had on individual development, among other things. The Education Centers and the online training providers are required to report on the first two of those levels (see fig. 6) and are encouraged to conduct the other two higher levels of evaluation of the courses they deliver. Individual Outreach trainers, who provide in-person training to workers, are not required to assess the training they deliver. OSHA officials said they decided not to require Outreach trainers conducting in-person training to test student knowledge or use student evaluations—even though this is how the vast majority of training in the program is delivered—because they did not want these activities to take away from instructional time. According to OSHA officials, when delivering 10- and 30-hour Outreach Training courses in the classroom, trainers have the ability to ask questions to encourage student participation and use activities to verify that the transfer of learning has occurred. However, due to the less interactive nature of the online training courses, OSHA determined that the online 10- and 30-hour classes require a testing component to ensure the students understand the material provided. According to OSHA officials, OSHA also requires testing for the Education Centers’ train-the-trainer courses to ensure the trainers have mastered the course content and are able to effectively teach the Outreach Training curriculum to workers. OSHA has taken some steps to assess the results of some of the evaluation information it collects and plans to make greater use of evaluation information it collects from the Education Centers in the future. For the in-person train-the-trainer courses that the Education Centers deliver, OSHA receives test results and copies of student evaluations. The Education Centers use a standard student evaluation form that can be electronically processed by OSHA. OSHA then automatically tabulates the information in the student evaluations and returns them to the Education Centers so that they can use them to make improvements to their courses. According to OSHA officials, OSHA reviewed a sample of the student evaluations and found positive results through these reviews. Beginning in fiscal year 2017, OSHA officials said they plan to begin assessing the student evaluation results from the train- the trainer courses as part of the annual appraisal process for Education Centers by adding a related performance measure to its expectations for the Education Centers. For the online training, OSHA receives test results and copies of student evaluations, which ask the students to identify the extent to which the training will help them better identify hazards and if they had any technical problems accessing the training, among other things. According to OSHA officials, they have not used this information to assess the results of the Outreach Training Program because the agency receives paper copies of the results in different formats, and the agency does not have the resources to analyze the information in its current format. We provided a draft of this report to DOL for its review. DOL did not provide formal comments and did not have any technical comments. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time we will send copies of this report to the appropriate congressional committees, the Acting Secretary of Labor, the Deputy Assistant Secretary for OSHA, and other interested parties. In addition, this report is available at no charge on the GAO website at http://www.gao.gov. Please contact me on (202) 512-7215 or at bawdena@gao.gov if you or your staff have any questions about this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix I. In addition to the contact named above, Mary Crenshaw (Assistant Director), Cathy Roark (Analyst-in-Charge), Hiwotte Amare, Deborah Bland, Carol Bray, David Chrisinger, Scott Coleman, Sarah Cornetto, Cliff Douglas, Holly Dye, Sharon Hermes, Sheila McCoy, Jonathan McMurray, Jean McSween, David Moser, Mimi Nguyen, Stacy Spence, and Bill Woods made significant contributions to this report.

Summary: In fiscal year 2016, approximately 900,000 workers were trained through OSHA's Outreach Training Program, the agency's primary mechanism for training workers on the basics of occupational safety and health. OSHA offers this training through OSHA-authorized in-person and online training providers. GAO was asked to review OSHA's administration of the program. GAO examined (1) the extent to which the program aligns with leading practices in designing an effective training program, (2) the process for documenting successful completion of the training and whether internal controls are in place to assure completion is accurately documented, and (3) how OSHA oversees training providers and assesses the results of the program. GAO compared OSHA's design and evaluation efforts for its training program with leading practices in GAO's training guide (GAO-04-546G) and federal internal control standards, and analyzed fiscal year 2012-2016 OSHA data (the most recent available) on time frames for processing completion card requests from online training providers. GAO also interviewed all nine online training providers and five in-person training providers, selected for having a high number of participants in fiscal year 2015, as well as OSHA officials. GAO is not making recommendations in this report. The Department of Labor's Occupational Safety and Health Administration's (OSHA) Outreach Training Program–which offers training on job hazard recognition and avoidance–reflects many of the attributes of a well-designed training program identified in GAO's training guide. OSHA is not required to follow GAO's training guide; however, the program's design reflects at least one indicator for six of the seven attributes of a well-designed training program GAO has identified. For example, OSHA used an appropriate mix of centralized and decentralized approaches by developing core learning objectives and content requirements for the courses but allowed training providers to modify the curriculum to meet the specific needs of their audience. In addition, OSHA officials told GAO that they took into account the leading causes of worker deaths and the most common workplace safety and health violations to determine topics to be covered in the training. OSHA documents successful course completion differently depending on whether training was delivered in-person or online, but it uses the same controls to prevent fraudulent completion cards from being issued. OSHA officials and the training providers GAO interviewed reported using several checks to prevent fraudulent completion cards from being issued, such as verifying course completion through automated and manual processes and comparing the number of cards requested to the number of registered students. Although OSHA does not require workers to complete Outreach Training, some workers may need to show proof of completion to satisfy requirements by their states, municipalities, employers, or unions. To obtain completion cards, Outreach trainers who deliver in-person training submit course information through a web-based system, while online training providers mail documentation to OSHA. OSHA processed 91 percent of the card requests from online training providers within 2 weeks, which is within the 30-day deadline OSHA has set for itself, according to GAO's analysis of OSHA data. OSHA officials reported that they plan to allow all training providers to request completion cards electronically, but the agency has not established a timeline for implementing this new process. According to OSHA officials, the agency is taking steps to modify its process for selecting online training providers and plans to incorporate electronic requests for completion cards into the new process. OSHA oversees the performance of training providers by routinely collecting and assessing data and investigating complaints and has taken some steps to assess the results of the program. Specifically, OSHA collects data from the training providers to help ensure that the training meets the program requirements and that the trainers are delivering the Outreach Training courses as intended. OSHA also investigates complaints it receives about training providers, including issues such as the trainer not spending enough time on required safety topics. To assess program results, OSHA tracks the number of workers who take the Outreach Training courses to measure the reach of the program. According to OSHA data, from fiscal year 2000 to fiscal year 2016, the number of workers trained more than quadrupled from 200,522 to 900,010. OSHA also receives test results and student evaluations of the courses from some of the Outreach Training providers.