Agencies implement specific elements of laws through regulations. One of the main purposes of guidance is to explain and help regulated parties comply with agency regulations. As shown in figure 1, agencies use guidance documents to explain how they plan to interpret regulations. Agencies also use guidance for circumstances they could not have anticipated when issuing a regulation and when additional clarifications are needed. Similarly, our past work has recommended that agencies issue guidance to clarify policies when we found confusion among grantees or others about a component’s policy or practices. How Does OMB Define Guidance? OMB defines the term “guidance document” as an agency statement of general applicability and future effect, other than a regulatory action, that sets forth a policy on a statutory, regulatory, or technical issue or an interpretation of a statutory or regulatory issue. Guidance documents often come in a variety of formats and names, including interpretive memoranda, policy statements, guidances, manuals, circulars, memoranda, bulletins, advisories, and the like. Guidance documents include, but are not limited to, agency interpretations or policies that relate to: the design, production, manufacturing, control, remediation, testing, analysis or assessment of products and substances, and the processing, content, and evaluation/approval of submissions or applications, as well as compliance guides. Guidance documents do not include solely scientific research. publish the final regulation.creating binding legal obligations. Regulations are generally subject to judicial review by the courts if, for example, a party believes that an agency did not follow required rulemaking procedures or went beyond its statutory authority. Regulations affect regulated entities by To balance the need for public input with competing societal interests favoring the efficient and expeditious conduct of certain government affairs, the APA exempts certain types of rules from the notice and comment process. These include “interpretative rules” and “general statements of policy” that can be made effective immediately upon publication in the Federal Register. and general statements of policy, agencies use guidance documents that may not be published in the Federal Register. In addition, agencies may use the preambles of their regulations to further interpret the regulations. An agency may use any of these documents to provide more explanation on how the agency plans to interpret a regulation. Defining guidance can also be difficult. To illustrate that difficulty, several of the components in our scope told us that they do not consider many of the communication documents they issue to the public to be guidance. 5 U.S.C. §§ 552(a), 553(b), 553(d). There is general agreement that the public interest is served by prompt dissemination of the guidance contained in agency interpretations and policy statements. Jeffrey S. Lubbers, A Guide to Federal Agency Rulemaking, 5th Edition (Chicago, Illinois; American Bar Association, 2012) p. 64. Agency guidance documents are not legally binding. Agencies sometimes include disclaimers in their guidance to note that the documents have no legally binding effect on regulated parties or the agencies. Even though not legally binding, guidance documents can have a significant effect on regulated entities and the public, both because of agencies’ reliance on large volumes of guidance documents and the fact that the guidance can prompt changes in the behavior of regulated parties and the general public. Due to the potential for these effects, agencies’ use of guidance has been the subject of scrutiny from policymakers and the public. Despite the general distinctions between regulations and guidance documents, legal scholars and federal courts have at times noted that it is not always easy to determine whether an agency action should be issued as a regulation subject to the APA’s notice and comment requirements or is guidance or a policy statement, and therefore exempt from these requirements. Among the reasons agency guidance may be legally challenged are procedural concerns that the agency inappropriately used guidance rather than the rulemaking process or concerns that the agency has issued guidance that goes beyond its authority. Other concerns raised about agency use of guidance include consistency of the information being provided, currency of guidance, and whether the documents are effectively communicated to those affected. Although the APA does not generally prescribe processes for review of agency guidance, the OMB Bulletin establishes policies and procedures for the development, issuance, and use of “significant” guidance documents. The Bulletin defines “significant guidance document” as a guidance document disseminated to regulated entities or the general public that may reasonably be anticipated to (1) lead to an annual effect on the economy of $100 million or more or adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or state, local, or tribal governments or communities; (2) create a serious inconsistency or otherwise interfere with an action taken or planned by another agency; (3) materially alter the budgetary impact of entitlements, grants, user fees, or loan programs or the rights and obligations of recipients thereof; or (4) raise novel legal or policy issues arising out of legal mandates, the President’s priorities, or the principles set forth in Executive Order 12866, as further amended. Pursuant to a memo issued by the Director of OMB in March 2009, OMB’s Office of Information and Regulatory Affairs (OIRA) reviews some significant guidance documents prior to issuance. All significant guidance documents, whether reviewed by OIRA or not, are subject to the OMB Bulletin. “Economically significant guidance documents” (those guidance documents under the first item in the definition above) are also published in the Federal Register to invite public comment. The OMB Bulletin directs each agency to develop written procedures for the approval of significant guidance, establishes standard elements that must be included in significant guidance documents, and requires agencies to maintain a website to assist the public in locating significant guidance documents. Non-significant guidance is not subject to the OMB Bulletin, and guidance procedures are left to agency discretion. How can guidance be significant? Guidance documents are considered “significant” when they have a broad and substantial impact on regulated entities, on the public or on other federal agencies. Guidance can have coercive effects or can lead parties to alter their conduct. For example, under a statute or regulation that would allow a range of actions to be eligible for a permit or other desired agency action, a guidance document might specify fast track treatment for a particular narrow form of behavior, but subject other behavior to a burdensome application process with an uncertain likelihood of success. Even if not legally binding, such guidance could affect behavior in a way that might lead to an economically significant impact. Similarly, an agency might make a pronouncement about the conditions under which it believes a particular substance or product is unsafe. While not legally binding, such a statement could reasonably be anticipated to lead to changes in behavior by the private sector or governmental authorities such that it would lead to a significant economic effect. For example, the following guidance documents issued by our audited agencies were considered significant: In response to questions from state officials, DOL’s Employment and Training Administration issued guidance in 2005 clarifying that distance learning can be considered approvable classroom training under the Trade Adjustment Assistance program. In response to requests for technical assistance, Education’s Office for Civil Rights determined that elementary and secondary schools and postsecondary institutions could benefit from additional guidance concerning Title IX obligations to address sexual violence. In 2014, the office issued guidance that included questions and answers to further clarify legal requirements, and discussed the interplay between applicable laws and proactive efforts schools can take to prevent sexual violence. Certain provisions of the OMB Bulletin were informed by written agency practices established by the Food and Drug Administration (FDA) for the initiation, development, issuance, and use of its guidance documents. In 1997, Congress established certain aspects of FDA’s guidance processes as law and directed the agency to evaluate the effectiveness of its practices and develop and issue regulations specifying its procedures.FDA’s Good Guidance regulations define the subset of guidance that must be published in the Federal Register.internal policies and practices to ensure appropriate adherence to their good guidance practices, including a written process to document decisions about the appropriate level of review for each guidance document. FDA officials told us that by default, guidance will receive a higher level of review unless a justification is presented to warrant lesser review. FDA officials told us that they use tools, such as “guidance initiation forms” or “concept papers” to, among other things, ensure they avoid duplicative or overlapping guidance and to prioritize proposed guidance. Component officials used guidance for multiple purposes, including interpreting regulations, disseminating suggested practices, and providing grant administration information. Interpret new regulations. Component officials told us they used guidance to summarize regulations or explain ways to meet regulatory requirements. For example, Education officials told us that they often follow their regulations with guidance issued to restate the regulation in plainer language, to summarize requirements, to suggest ways to comply with the new regulation, or to offer best practices. Occupational Safety and Health Administration (OSHA) officials told us that they issue guidance to help employers and workers understand their legal rights and responsibilities. Distribute information on suggested practices. Officials told us that they used guidance to distribute information on program suggestions sometimes called best practices. In particular, component officials who administered formula grants in which wide discretion is given to grantees, such as states, told us that they often used guidance to encourage certain leading practices. For example, the Administration for Children and Families (ACF) Office of Child Care issued Information Memorandums encouraging partnerships between state child care and child welfare agencies. Provide guidance on grant administration. Components that administered grants also issued procedural guidance related to grant administration. For example, the Bureau of Labor Statistics (BLS) issued routine administrative memoranda to remind state partners of federal grant reporting requirements. The impetus for developing and issuing guidance varied, including reasons such as: (1) explaining new regulations, (2) responding to questions from external stakeholders, (3) clarifying policies in response to compliance findings, and (4) disseminating information on leadership priorities and initiatives. Explaining new regulations. Components initiated guidance in coordination with publication of a new regulation to help regulated entities understand new requirements. For example, in 2014, DOL’s Wage and Hour Division released both a fact sheet and Frequently Asked Questions (FAQs) to coincide with the issuance of its final rule establishing a minimum wage for federal contractors. Clarifying policies in response to questions. Multiple component officials told us that they used guidance to clarify policies in response to questions received from the field or regional office input about questions they are receiving from grantees or regulated entities. DOL’s Office of Labor-Management Standards officials told us that ideas for guidance often come from questions from the field or the regulated community, particularly if multiple unions had similar questions about a new regulation. Clarifying policies in response to compliance findings. Officials at Education’s Office for Civil Rights and OSHA told us that they often initiated guidance in response to findings resulting from their investigatory or monitoring efforts, among other things. USDA Food and Nutrition Service officials told us they have issued guidance in response to our or Office of Inspector General audits. Disseminating information on leadership priorities and initiatives. In some cases, component officials told us they may issue guidance in response to directives from senior management or in response to administration priorities. ACF’s Office of Child Care officials told us that they use Information Memorandums to emphasize leadership or other legislative priorities and changes. Officials at Education’s Office of Postsecondary Education told us that component leadership initiates guidance related to priorities the administration wants to accomplish. When we asked for an example, they noted that a Dear Colleague letter the Office issued explaining that students confined or incarcerated in locations such as juvenile justice facilities were eligible for federal Pell grants was issued in response to inquiries from the field as well as administration priorities. We found that agencies did not use standard terminology for their guidance. Components we reviewed within USDA, HHS, Education, and DOL used many names for their external guidance documents. Some departments or components generally used uniform names for guidance; at other departments components used different names for guidance. Education offices often issued Dear Colleague letters or FAQs, among other types of guidance. Within HHS, ACF officials told us that they consistently used Program Instructions and Information Memorandums to communicate information to grantees and other recipients. While many DOL components issued documents to assist with regulatory compliance, they used varied terms for that guidance, including bulletins, Administrator Interpretations, directives, fact sheets, and policy letters. The amount of guidance components issued varied, ranging from about ten guidance documents to over a hundred documents issued by a component in a single year. BLS officials told us that they issued about ten routine administrative memorandums each year related to the operation of two cooperative agreement statistical programs. In contrast, OSHA officials told us they could easily produce 100 new or updated products each year to provide guidance to stakeholders. Component officials cited the varying missions or types of programs as one explanation for the different amounts of guidance. OSHA regularly issued guidance to assist with regulatory compliance, while BLS officials told us that, as a non-regulatory component, they rarely issued guidance. Although the Office of Workers’ Compensation Programs has regulatory authority, officials told us that they did not frequently issue guidance because their authorizing statutes have not changed recently and their programs focus on administering benefits. Officials at the Office of Child Care and at DOL’s Employment and Training Administration (ETA) told us they rarely issued guidance for formula or block grant programs or were generally limited to issuing guidance with encouragements and recommendations. These programs include the Office of Child Care’s Child Care Development Fund Grants and ETA’s Workforce Investment Act formula grants (both to states), which provide wide discretion to grantees. Officials considered a number of factors before deciding whether to issue guidance or undertake rulemaking. However, a key criterion in making this decision was whether they intended for the document to be binding. Officials from all components that issue regulations told us that they understood when guidance would be inappropriate and when regulation was necessary and consulted with legal counsel as they decided whether to initiate rulemaking or issue guidance. Officials told us that they often based the decision between guidance and regulation on whether the direction was meant to be binding (in which case they issued a regulation). In some cases, issued guidance clarified existing regulations, educated the public, addressed particular circumstances, or shared leading practices. According to DOL officials, if components determined that current regulations could not reasonably be interpreted to encompass the best course of action, the solution was not case-specific, or the problem was very wide-spread, then they may determine that issuing a new regulation was necessary. An Education department official told us that they considered multiple factors, including the objective to be achieved, when choosing between guidance and regulations. For example, they used a regulation to fill in gaps in statutory provisions. Following issuance of regulations, they also provided guidance in the form of technical assistance. Similarly, HHS Administration for Community Living officials told us that they considered a number of factors, including whether the instructions to be disseminated were enforceable or merely good practice. For example, they noticed that states were applying issued guidance related to technical assistance and compliance for the state long-term care ombudsman program differently. Administration for Community Living officials decided it would be best to clarify program actions through a regulation, as they could not compel the states to comply through guidance. They believed that a regulation would ensure consistent application of program requirements and allow them to enforce those actions. They issued the proposed rule in June 2013. USDA’s Food and Nutrition Service (FNS) officials told us that the decision to issue guidance or undertake rulemaking depended on (1) the extent to which the proposed document was anticipated to affect stakeholders and the public, and (2) what the component was trying to accomplish with the issued document. OIRA staff concurred that agencies understood what types of direction to regulated entities must go through the regulatory process. In a few cases, components used guidance to alert affected entities about immediate statutory requirements or to anticipate upcoming requirements to be promulgated through the rulemaking process. While this may provide timely information about new or upcoming requirements, it also may cause confusion as details are revised during the rulemaking process. FNS officials told us that when a new statute becomes effective immediately and there is little ambiguity in how the statute can be interpreted, they use a “staging process.” In this process, they issue informational guidance so their partners are aware of and consistently understand new requirements before the more time-consuming rulemaking process can be completed. In 2014, we reported that five FNS memorandums related to new statutory requirements for the content of school lunches were distributed prior to the issuance of the final rule on the changes to the content and nutrition standards for school lunches, in part because of statutory timeframes. As FNS implemented the finalized regulation, it also issued guidance containing new flexibilities or substantive changes to previously issued guidance. While state and school food authority officials said that some of these changes were likely made by USDA to respond to problems they were having implementing the new lunch requirements, the guidance changes were difficult to keep up with and led to increased confusion about the requirements. Education department officials told us they often used guidance to help the field understand and apply new statutory requirements. Other officials told us that in rare instances, they have issued guidance while a proposed rule is out for comment. They noted that statutory deadlines for implementation may necessitate the issuance of guidance prior to the issuance of a final rule. For example, in 2014 DOL’s ETA issued an Unemployment Insurance Program Letter interpreting statutory provisions on permissible drug testing of Unemployment Insurance applicants. ETA noted that the program letter did not provide interim guidance on the substance of the rule, but interpreted related statutory provisions. The guidance stated that certain provisions related to the regulation could not be implemented until publication of the final regulation. ETA officials told us they will issue revised guidance after issuance of the final regulation. Unemployment Insurance Program Letter 1-15, Permissible Drug Testing of Certain Unemployment Compensation Applicants Provided for in Title II, Subtitle A of the Middle Class Tax Relief and Job Creation Act of 2012. 79 Fed. Reg. 61,013 (Oct. 9. 2014) (proposed rule). future guidance. However, the website was not updated in a timely manner when the regulation was issued as final, which could have caused confusion. ACF’s Office of Child Care issued recommendations to its grantees that foreshadowed future binding requirements. For example, in September 2011 the Office issued an Information Memorandum recommending criminal background checks. It later published a proposed rule in May 2013 to mandate the recommendations as requirements. Selected departments considered few of their guidance documents significant as defined by OMB. Education considered a greater number of its guidance documents significant, while USDA and DOL issued relatively few significant guidance documents. We were unable to determine the number of significant guidance documents issued by HHS. As of February 2015, Education listed 139 significant guidance documents on its website, while DOL listed links to 36 pieces of significant guidance on its website. USDA listed links to 34 significant guidance documents on its webpage for significant guidance (see figure 2). All selected components told us that they did not issue any economically significant guidance. OIRA staff told us they accepted departments’ determinations of which types of guidance meet the definition of significant guidance. However, given the unique circumstances of each program, the selected departments differed in their interpretation of the significance of guidance issued to explain eligibility changes resulting from the Windsor Decision Education deemed its initial guidance recognizing same-sex marriages.explaining eligibility changes for student aid in response to Windsor to be non-significant, but considered later guidance providing clarifications significant. HHS and USDA did not consider corresponding guidance they released to be significant. Officials at several components told us that, rather than invest resources in OMB review of significant guidance, they would typically decide to propose a regulation, which would allow them to assert binding requirements. Employee Benefit and Security Administration officials told us they would ordinarily use a regulation if they were considering a guidance project that would meet the OMB bulletin definition of significant or economically significant guidance. Officials at Labor’s Employment and Training Administration told us they would typically opt to issue a rule rather than guidance if the content was considered significant. OIRA staff told us that OIRA examiners work closely with department officials and may discuss what types of documents warrant OIRA review. What are the OMB Good Guidance Practices for Approval of Significant Guidance? OMB’s Final Bulletin for Agency Good Guidance Practices requires agencies to develop the following procedures for approval of significant guidance: Agencies should develop or have written procedures for the approval of significant guidance documents. Those procedures shall ensure that the issuance of significant guidance documents is approved by appropriate senior agency officials. Education and USDA had written procedures for the approval of significant guidance as directed by OMB. However, HHS did not. While DOL had written approval procedures, they were not available to appropriate officials and DOL officials noted that they required updating. Education and USDA’s written procedures explained the approval and clearance procedures for significant guidance. DOL officials told us that, although their written procedures were not readily available during our audit, officials had been trained in 2007 on review and approval of significant guidance documents. HHS officials told us that each component tracked guidance development differently and a lack of written procedures did not mean that guidance did not receive appropriate departmental review. However, without written procedures or wide knowledge of these procedures—a basic internal control standard—HHS and DOL may be unable to ensure that their components consistently follow other requirements of the OMB bulletin, such as required standard elements for significant guidance, and cannot ensure consistency in their processes over time. As previously discussed, the Administrative Procedures Act does not establish standards for the production of guidance. Therefore, departments and components produce and issue the bulk of guidance— guidance that is considered non-significant—without government-wide standards for those processes. In the absence of government-wide standards for the production of non-significant guidance, officials must rely upon internal controls—which are synonymous with management controls—to ensure that guidance policies, processes, and practices achieve desired results and prevent and detect errors. By incorporating internal control standards, departments and components can promote consistent application of management processes. We identified four selected components of internal control and applied them to agencies’ guidance processes (see table 2 below). Departments and components identified diverse and specific practices that addressed these four components of internal control. While all departments and components identified standard processes for internal review of their guidance documents, these processes were typically not documented. Further, agencies did not consistently apply other components of internal control. The following sections identify practices that select components have used to address these internal controls, as well as opportunities for broader application of these practices. Internal controls help to guide departments and components in the guidance production process in a fluid environment where processes and anticipated goals for non-significant guidance vary. Even within components, types of guidance may need to be treated differently or may warrant differing levels of review. A level of standardization of the process that may be appropriate for one component or type of guidance may be inappropriate for another. For information on specific components’ processes for initiation, development, review, dissemination, and tracking and evaluation of guidance, see appendix II. Agencies identified and discussed risk as they initiated guidance, prioritized among different guidance documents to be developed, and made decisions about the necessary level of review. Although no component can insulate itself completely from risks, it can manage risk by involving management in decisions to initiate guidance, prioritizing among proposed guidance, and determining the appropriate level of review prior to issuance. According to internal control standards, agencies should use techniques and processes to identify and manage risk. Agencies face multiple risks when going through the guidance production process. Risks include legal challenges that issued guidance is asserting binding requirements without having gone through the rulemaking process, or that a guidance document goes beyond the agency’s statutory authority. In addition, if leadership is not included in discussions related to initiation of guidance, agencies risk expending resources developing guidance that is unnecessary or inadvisable. At a few components, officials told us that leadership (such as component heads and department-level management) decided whether to initiate certain guidance and guidance did not originate from program staff for these components. Employee Benefits Security Administration office directors presented guidance proposals considering legal, policy, and programmatic factors to their Assistant Secretaries and Deputy Assistant Secretaries for approval to start developing guidance. In most other cases, ideas for additional guidance originated from program staff and field offices or from leadership, depending on the nature of the guidance. Education officials told us that component program staff and leadership work together to identify issues to address in guidance. USDA Food and Nutrition Service officials told us they may decide to initiate guidance based on both input from regional offices and directives from senior leadership. In a recent example, the Under Secretary for Food, Consumer, and Nutrition Services initiated guidance to remind state agencies of existing requirements prohibiting the online sale of benefits for the Special Supplemental Nutrition Program for Women, Infants, and Children. Officials at several components indicated that they prioritized the development of certain guidance documents over others to ensure that guidance expediently responded, for example, to constituents with pressing needs to comply with current regulations and that staff resources were allocated in alignment with these priorities. Education officials told us they prioritized among possible guidance documents based upon conversations about staffing resources as well as the needs of their constituents. For example, Education’s Office of Management officials told us they used an ongoing list to prioritize certain guidance documents, considering factors including (1) whether the guidance document explained new statutory requirements, (2) whether it responded to questions from their constituents, (3) the importance of the guidance to other Education programs, or (4) whether the proposed guidance addressed issues identified through technical assistance or calls to their compliance hotline. HHS Administration for Children and Families’ (ACF) Office of Child Care officials explained that they prioritized guidance in areas that (1) had the largest positive impacts; (2) needed clarification to address questions from grantees, stakeholders, and others; and (3) were most relevant to the program’s rules or requirements. At most components, officials told us that they determine the appropriate level of review and final clearance of proposed guidance, and in many cases guidance was reviewed at a higher level if the document was anticipated to affect other offices or had a particular subject or scope. Risk was one factor agency officials considered when determining the anticipated appropriate level of review and final clearance of proposed guidance. At the Employee Benefits Security Administration, for example, the need for department-level clearance depended on various factors. These included likely congressional interest, potential effects on areas regulated by other DOL components, expected media coverage, and whether the guidance was likely to be seen as controversial by constituent groups. Two other factors that a few agencies reported considering in determining whether the guidance warranted a higher level of review were whether it was related to a major priority or would be “impactful.” Most components did not have written procedures for guidance initiation, development, and review. Control activities (such as written procedures) help ensure that actions are taken to address risks and enforce management’s directives. In the absence of written procedures, components relied on officials’ understanding of the guidance process, including when certain guidance documents should have been reviewed by leadership and when it was unnecessary to have that review. In these cases, officials told us that the guidance process was well understood by program staff or followed typical management hierarchies. For example, HHS’s Administration for Community Living, DOL’s Office of Labor- Management Standards and Education’s Office of Special Education and Rehabilitative Services officials told us program staff have a good understanding of the processes involved in developing and obtaining approval of guidance. Control activities may help components assure that management officials approve of the content of the guidance and concur on the document’s relative priority. Control activities outlined in written procedures can provide a central approach to guidance initiation, development, and review. A total of 6 of the 25 components had written procedures for the entire guidance production process, and several of these components highlighted benefits of these procedures for their guidance processes. These components included ACF’s Office of Head Start and five components at DOL: the Occupational Safety and Health Administration (OSHA), the Mine Safety and Health Administration, the Employment and Training Administration, the Office of Federal Contract Compliance Programs, and the Bureau of Labor Statistics. Education’s Office of Innovation and Improvement and Office of Elementary and Secondary Education and Labor’s Veterans’ Employment and Training Service had written procedures only for the review and clearance phase. The Mine Safety and Health Administration’s written procedures contain information that they describe as essential to the effective and consistent administration of the component’s programs and activities. As shown in figure 3, OSHA’s written procedures are designed to ensure that the program director manages the process for a specific policy document by considering feedback and obtaining appropriate concurrence to ensure that guidance incorporates all comments and has been cleared by appropriate officials. The Deputy Assistant Secretary resolves any disagreements about substance, potential policy implications, or assigned priority of the document. Documented procedures are not just an internal controls issue; agencies benefit from them. OSHA’s procedures were meant to ensure effective management of the issuance of guidance documents. Internal control standards do not prescribe either centralization or decentralization for managing guidance processes, and the departments we reviewed had varied approaches. One department, Education, had centralized processes for guidance development, review, and dissemination, while the other three departments were decentralized. At Education, an Office of the General Counsel official told us she was involved in decisions about whether guidance is considered significant and Education’s Office of the Executive Secretariat managed the document clearance and approval processes for all guidance. This office also maintained frequently asked questions to explain the process to components. In contrast, officials told us that DOL gave its components the flexibility to develop individual procedures for developing and issuing non-significant guidance. In addition, HHS departmental officials told us they played a secondary review and approval role and that each HHS component approached the development and issuance of guidance documents differently. USDA officials told us that its guidance process was also decentralized, as guidance was typically initiated, developed, and approved at the program level, while significant guidance was shared with the department for review. Although a few components had written procedures for guidance initiation, development, and review, officials from all components could describe standard review practices to provide management the opportunity to comment and ensure that its comments were addressed by program staff. For example, the Administration for Community Living had its officials circulate draft guidance for internal review and typically required three to four officials to sign off on the draft, including center directors and its Executive Secretariat. At Education’s Office of Innovation and Improvement, program staff shared draft guidance with senior leadership, who in turn provided feedback. Once senior leadership officials and program staff were satisfied with and approved the document, it was sent to the Office of the Executive Secretariat to be placed into clearance. Most selected components had guidance practices to ensure either intra- agency and interagency review (or both) of guidance documents before issuance. Internal controls require (1) that information is recorded and communicated to management and others, and (2) the components have an adequate means of communicating with and obtaining information from external stakeholders that may significantly affect the component’s ability to achieve its goals. Intra-agency communications. To ensure that management concurrence was recorded, most components we reviewed used communication tools, such as electronic or hard-copy routing slips, to document approval for guidance clearance or to communicate with management and other offices about proposed or upcoming guidance. In particular, officials at 20 components used a routing slip to document management concurrence. For example, the Mine Safety and Health Administration used two forms to track the clearance of guidance documents. Education’s Office of Management used a routing slip to document internal component approvals and convened a working group to resolve comments and edits on the guidance documents. The two components within ACF used a “policy calendar,” a tool for communicating with management about the guidance documents being drafted and their projected issuance dates. ACF’s policy calendar listed the name and status of guidance, whether it was a presidential or secretarial priority, whether the affected program was mandatory or discretionary, and the proposed date of issuance to alert appropriate ACF officials of upcoming guidance and to facilitate appropriate review. Interagency communications. Most component officials told us that they conferred with other affected components or federal departments to ensure consistency of their guidance during the development of guidance. Officials at Education’s Office for Civil Rights told us they sometimes reached out to other federal agencies and interested stakeholders to have “listening sessions” on new guidance documents, such as a 2013 pamphlet on academic success for pregnant and parenting students. DOL’s Office of Disability and Employment Policy officials told us they often worked with multiple departments that addressed disability issues and contributed to the fact sheets or other guidance documents issued by these departments. External stakeholders. Officials told us that feedback from external nonfederal stakeholders often served as the impetus for the initiation of guidance, and 15 of the 25 selected components cited examples in which they conferred with external nonfederal stakeholders during the guidance development process. At OSHA, external stakeholders were not involved in developing directives or policy issuances, but assisted with developing educational, non-policy guidance, such as hazard alerts. Food and Nutrition Service officials told us that state and local agency staff, industry representatives, advocacy organizations, and the general public were involved in the development of their guidance, generally through a comment period. For example, during the development of a policy memorandum on vendor management, the Special Supplemental Nutrition Program for Women, Infants, and Children solicited comments from regional offices and all state agencies through a 45-day comment period. Although some components did not have a formalized process to assess the effectiveness of their guidance, many of these components told us they have updated or revised certain guidance documents. According to internal control standards, agencies benefit from procedures to continually reassess and improve guidance processes and documents to respond to the concerns of regulated entities. In the absence of monitoring and evaluation strategies, components cannot assess whether guidance meets intended goals or whether they need to provide additional guidance to supplement and improve upon prior guidance. Nearly half of the components we reviewed (11 of the 25) did not regularly evaluate whether issued guidance was effective and up to date. Without a regular review of issued guidance, components can miss the opportunity to improve their guidance. DOL’s Office of Labor- Management Standards officials told us they had not evaluated the relative success of old guidance and did not often revise guidance. ACF’s Office of Child Care regularly tracked and updated guidance on grantee reporting requirements. However, officials said there was little need to track or update other guidance, as it had been 18 years since its authorizing statute was changed. However, with the recent passage of the Child Care and Development Block Grant reauthorization, officials said they intended to assess their old guidance and update it to reflect the new law. GAO, Fair Labor Standards Act: The Department of Labor Should Adopt a More Systematic Approach to Developing Its Guidance. GAO-14-69 (Washington, D.C.: Dec. 18. 2013). systematic effort to evaluate guidance. For example, DOL’s Wage and Hour officials updated guidance to reflect new standards during their 2010 regulatory initiative on the temporary agricultural employment of H-2A immigrant workers. A few selected components had initiated or established a process for tracking and evaluating guidance to identify necessary revisions. For example, in November 2011, DOL’s Office of Federal Contract Compliance Programs officials initiated a 2-year project to review their directives system to ensure that they only posted up-to-date guidance. As a result of the project, in 2012 and 2013 officials identified necessary updates to guidance, clarified superseded guidance, and rescinded guidance where appropriate. Officials told us that these actions reduced the original number of directives by 85 percent. Officials also told us that they did this to ensure that their guidance was more accurate and correct, and the actions resulted in officials posting only relevant and current guidance information on the component’s website. Officials told us they now routinely monitor their directives about once a year and review other guidance documents each time they issue new regulations or change a policy to decide if they need to revise them. The Employment and Training Administration used a checklist to review a list of active guidance documents and identified whether to continue, cancel, or rescind the guidance. In addition, officials indicated which documents were no longer active on their website. The Mine Safety and Health Administration also ensured that programs periodically reviewed and updated guidance documents and canceled certain guidance. All components told us they relied primarily on their websites to disseminate guidance but also used many other dissemination methods. As shown in figure 4, the components in our review used various strategies to distribute guidance to the public. While all agencies posted guidance online, a few components also made documents available to specific audiences on intranet websites. For example, USDA’s Food and Nutrition Service officials told us that they posted operational guidance on upcoming or proposed regulations to their PartnerWeb, an intranet site that is only accessible by state agencies. One component, DOL’s Bureau of Labor Statistics, e-mailed guidance to state agencies and posted it on an intranet site for state agencies that is not publicly accessible. Components also relied on other government websites to distribute guidance. For example, Education used ADA.gov to jointly issue guidance related to disability discrimination with the Department of Justice and Stopbullying.gov to publicize guidance related to antibullying laws and policies. Components also designed specialized websites to disseminate guidance on specialized topics. For example, DOL’s Office of Disability Employment Policy posted information about disability programs on disability.gov and Education’s Federal Student Aid used a separate websites to serve different audiences. Almost all components used e-mail as another key dissemination method. Components also used listservs (which manage e-mails to and from a list of subscribers), e-mail delivery service (such as GovDelivery) or newsletters. Officials told us they compiled listservs of individuals interested in specific issues. They also explained that these lists were developed in a number of ways, including by program offices that add interested parties or directly from members of the public who sign up to be on these lists through component websites. These listservs could be very large. For example, DOL’s Employee Benefits Security Administration list has 336,000 subscribers. Recognizing the importance of listservs as a dissemination method, officials at several components told us they periodically verify and update their e-mail lists. Components also used other methods to disseminate guidance. Some held press conferences or issued press releases, while others distributed and discussed guidance during conferences, webinars, or conference calls. Components also reported using social media, such as Facebook, Twitter, or blogs. A few components told us that they posted guidance in the Federal Register. Lastly, component officials said that external partners—such as states, advocacy groups, and trade associations— sometimes distributed guidance for them at their request. Officials used different strategies to reach certain groups and noted that it was more resource intensive to distribute guidance to a wider audience. For example, officials from HHS’s Administration for Community Living explained that because their subgrantees are defined in statute, they were able to effectively target their guidance to that group. Similarly, Education’s Office for Civil Rights officials had readily available e-mail lists for the purpose of sending guidance to all public school superintendents or college presidents. DOL Employee Benefits Security Administration officials noted that disseminating guidance to financial institutions was fairly easy because that audience was receptive to receiving information through their website and generally vocal when they were unable to find the information they were seeking. On the other hand, OSHA officials told us they use social media to communicate with hard- to-reach populations, such as non-English speakers and temporary/contract workers who were more likely to be working in dangerous jobs, and used hard-copy guidance during disaster recovery efforts or to reach those who did not have access to the Internet. Officials noted that states and stakeholder groups were helpful in reaching wide audiences, especially when disseminating guidance to large groups nationwide, such as parents or students and all employers or employees. Components also reached wider audiences by engaging with the public directly through conferences, webinars, media outreach, or public awareness campaigns. Our ability to access and find significant and non-significant guidance online varied. We found that Education, USDA, and DOL consistently applied OMB Bulletin requirements for public access and feedback for significant guidance while HHS did not. HHS’s website did not link to significant guidance documents. In addition, we were unable to find these documents by searching the department’s website. HHS officials could not explain why these documents were not posted on its website. Because components rely on their websites to disseminate guidance, it is important that they generally follow requirements and guidelines for online dissemination. For significant guidance, agencies are required by the OMB Bulletin to maintain a current list of their significant guidance on their websites. Agencies must also provide a means for the public to submit comments on significant guidance through their websites. Without providing the public an easy way to access significant guidance, agencies cannot ensure that the public can know about or provide feedback on these documents. an agency’s website to the electronic list posted on a component website—a current list of its significant guidance documents in effect. The list shall include the name of each significant guidance document, any document identification number, and issuance and revision dates. The agency shall provide a link from the current list to each significant guidance document that is in effect. The list shall identify significant guidance documents that have been added, revised or withdrawn in the past year. While the OMB Bulletin does not have requirements for agencies related to the online dissemination of non-significant guidance, there are several resources agencies can use to improve how they post and update those documents. One such resource is the Guidelines for Improving Digital Services developed by the federal Digital Services Advisory Group. These guidelines are aimed at helping federal agencies improve their communications and interactions with customers through websites (see table 3). website a means for the public to submit comments electronically on significant guidance documents, and to submit a request electronically for issuance, reconsideration, modification, or rescission of significant guidance documents. Public comments under these procedures are for the benefit of the agency. No formal response to comments by the agency is required. Made guidance easily accessible from component home pages. All components linked key guidance documents on their websites so that guidance could be easily found. We were able to navigate from the homepage to the guidance itself in just a few clicks for all the websites we reviewed. All components also used common terms for guidance—including publications, resources, policy, grant guidance, fact sheets, memorandums, and reports—to help users identify those documents. Components used these terms to create links or menus to facilitate users’ ability to find guidance. : HHS reported using its department-wide governance structure for developing and delivering digital services. Governance, policies, and standards: Education established guiding principles to reinforce a governance structure for developing and delivering digital services and managing data. Cross-agency collaboration and shared services and tools website to showcase digital strategy best practices and to test new technology and tools. Technical considerations: that it was modernizing its technical infrastructure by adhering to business requirements and technical trends, such as increased use of and support for mobile devices. Usability and accessibility: target for all its websites and digital content to become accessible and compliant with Section 508—which requires that federal electronic and information technology are accessible to people with disabilities—by May 31, 2013. As of September 2013, the Administration for Children and Families website was 92 percent compliant and the Administration for Community Living’s website was 99 percent compliant. Privacy and security: that the information it collects is protected by the privacy and confidentiality provision of federal statutes, including the Family Educational Rights and Privacy Act, the Individuals with Disabilities Education Act, the Education Sciences Reform Act, and the Privacy Act of 1974. It has also implemented a disclosure review process and established a review board to ensure that data are reviewed and approved before they are publicly released. Improved search. We found that most components had search tools on their websites that generally functioned well. Searches are a key way that users access guidance. A number of components had taken steps to improve their website searches. These included adding meta tags to the code on their pages so that the most relevant content appeared higher in the results of external searches and adding a wider range of keywords to internal search engines to improve searches. Components that had not made improvements explained that this was because they used the department’s search tools and did not have the ability to make changes on their own. Highlighted new or important guidance. Components highlighted new or important guidance on their homepages to draw users’ attention to that information. For example, HHS’s Office of Child Care highlighted the passage of the Child Care and Development Block Grant Act of 2014 on its homepage by providing reauthorization resources. The website included key guidance related to the act, including program instructions, technical assistance, and trainings. Posted contact information to allow for questions and feedback from the public. Components used websites to provide contact information to the public. Specifically, components posted toll-free numbers which could facilitate the public’s ability to ask questions or provide feedback on published guidance. A few components provided direct e-mails or phone numbers for specific offices and key program staff. Opportunities for affected parties and other stakeholders to submit questions and feedback on guidance documents are important because, as discussed above, public interactions have served as the impetus for new guidance. Further, because not all components we reviewed provided examples of taking steps to solicit and respond to public comments as guidance was developed, ensuring effective mechanisms for affected parties and others to submit feedback is crucial. Categorized guidance. Components organized guidance by type, topic, date, or audience to help users sort through the sometimes long lists of guidance posted online, as shown in figure 5. Several factors hindered the ease of access to component guidance online. Components posted long lists of guidance, which could make it difficult for users to find particular guidance documents. In addition, we found that few components effectively distinguished whether their online guidance was current or outdated to ensure the relevance of their online information. As discussed earlier, we found that DOL’s Office of Labor- Management Standards did not update its website in a timely manner to reflect guidance that would be affected by finalized regulation. Clearly marking whether guidance is current is important. As previously discussed, DOL’s Office of Federal Contract Compliance Programs efforts to ensure the relevancy and currency of its directives system resulted in the removal of 85 percent of their documents. Ensuring that online content is accurate and relevant is one of the guidelines for federal digital services (see table 3 above). Easy access to current and relevant guidance could also facilitate opportunities for affected parties and stakeholders to provide feedback on those documents. Another factor that hindered public access was that it was not always clear where to find guidance on a component website. We found guidance was sometimes dispersed across multiple pages within a website, which could make guidance hard to find and could contribute to user confusion. The labeling of these links was not distinctive enough for users to know where to go for the various guidance documents or topics they may be seeking (see figure 6). These issues could be a result of the requirement that Education components use a departmental template for their websites. A few components created navigational links to supplement departmental toolbars. Education officials told us that they have learned from their grantees that the department’s guidance was not easy to find and that online resources were hard to navigate. Federal digital services guidelines direct agencies to publish digital information so that it is easy to find and access (see table 3 above). While components used web metrics to evaluate their online guidance dissemination strategy, many did not use that information to change their existing approach. Further, many component officials told us that they did not have a systematic way to evaluate whether the public could access their guidance online. Web and customer satisfaction metrics—data which allow agencies to measure performance, customer satisfaction, and engagement to make continuous improvements to serve its customers—could be a good source of this information.web metrics can inform officials about which guidance is being accessed For example, and searched. Similarly, customer satisfaction metrics could provide qualitative information about how easily users were able to find the guidance they were seeking. The thoughtful analysis and application of these data would allow components to regularly evaluate the effectiveness of disseminating guidance through their websites (see table 3 above). Further, internal controls call for the continual monitoring of results and for management to take proper actions in response to findings. All components collected web metrics. This data could help agencies evaluate its online products, which is a guideline for federal digital services (see table 3 above). Every department in our review used Google Analytics to collect website performance data. Many components also used web metrics to evaluate the effectiveness of how they disseminated guidance online. For example, USDA’s Food and Nutrition Service used web metrics to track overall use of its guidance online. In another example, HHS’s Administration for Community Living learned about ways to drive traffic to its website when new guidance is posted through its evaluation of web metrics. However, many components did not use web metrics to improve how they disseminated guidance through their websites (see table 4). Only 8 of the 24 components that made guidance publicly available online reported using web metrics to improve how they used websites to disseminate these documents. Because all components we studied primarily relied on their websites to disseminate guidance, there is an opportunity for them to build on their use of web metrics to improve how they disseminate guidance online. Doing so will also facilitate components’ efforts to evaluate its online guidance, which is also a guideline for federal digital services (see table 3 above). Components found ways to evaluate the effectiveness of guidance dissemination outside their websites. This included seeing how many applications had been viewed, downloaded, or submitted through grants.gov, and how many e-mails and newsletters were opened through GovDelivery. Components also conducted usability tests, focus groups, and surveys of users, adhering to the federal guideline calling for agencies to collect and address customer feedback (see table 3 above). Other components convened internal task teams to identify and implement changes while others hired contractors to redesign their websites. Further, officials reported receiving useful feedback directly from the public at conferences, webinars, stakeholder/grantee meetings, or from monitoring visits. For example, Education’s Federal Student Aid officials changed their search function to allow guidance to be searched by relevance and date based on feedback received during training with outreach groups. Additionally, OSHA officials are in the process of surveying subscribers of its biweekly e-newsletter to seek feedback and improvements so that they can provide useful, educational, and up-to- date information to the public. and Adult Education learned through the use of metrics that many users sought information about states and reorganized its website to make that information more prominent and easier to locate. Administration updated its pages to facilitate user access to guidance related to lead because of web metrics indicating that users were searching the topic. The component also reconfigured its publications page based on the number of downloads. It also used metrics to guide how often to reprint guidance and which ones to translate. Administration used metrics to determine how long to keep guidance on its homepages and provided navigational links to reflect the different ways visitors search its website. metrics to populate its list of trending topics and most requested pages. It also used metrics on high-traffic pages to inform decisions about where to post new guidance. Further, decisions about what materials should be translated into other languages and whether more guidance is needed on certain topics are based on web metrics. Guidance documents are an important tool that agencies use to communicate timely information about the implementation of regulatory and grant programs to regulated parties, grantees, and the general public. Guidance documents also provide agencies valuable flexibility to clarify their requirements and policies, and to address new issues and circumstances more quickly than may be possible using rulemaking. However, agencies must also exercise diligence when using guidance. Although guidance documents are not legally binding, they can affect the actions of agencies’ staffs, stakeholders and other interested parties, because guidance articulates agencies’ interpretations and policy choices. The potential effects of these documents—and the risks of legal challenges to agencies—underscore the need for consistent and well- understood processes for the development, review, dissemination, and evaluation of guidance. We found mixed compliance with requirements established by the Office of Management and Budget’s (OMB) Final Bulletin for Agency Good Guidance Practices (OMB Bulletin) for the subset of guidance documents considered “significant” under OMB’s definition. Education and USDA had written departmental procedures for the approval of significant guidance, as directed by the OMB bulletin. DOL officials had not made their procedures available to component staff to ensure consistent application of review processes for significant guidance, and those procedures required updating. HHS had no procedures for significant guidance approval. Though officials from both departments told us that they believed their components understood the OMB requirements, HHS and DOL could better ensure that their components consistently followed OMB’s requirements for significant guidance if they made departmental written procedures available. Education, USDA, and DOL consistently applied other OMB Bulletin requirements on public access and feedback for significant guidance, but HHS did not. HHS did not explain why the department had not posted a website for significant guidance online. HHS should ensure that the public can easily access and provide feedback on its significant guidance, as required by OMB. Without providing an easy way to access and comment on significant guidance, HHS cannot ensure that the public is aware of or can provide feedback on these documents. Government-wide guidance that specifically addresses processes for non-significant guidance does not exist. Non-significant guidance accounts for the bulk of components’ guidance documents. In the absence of government-wide guidance specifically targeted at non- significant guidance, internal control principles and standards provide the key criteria for components to apply to their policies and procedures. Component officials identified many practices that they use to address internal control standards regarding risk assessment, control activities, communication, and monitoring. In particular, officials at most components told us that they determine the appropriate level of review and final clearance of proposed guidance documents. However, the components less consistently identified practices to address other elements of internal controls. For example, though all components could describe standard practices for developing guidance, only 6 of the 25 components had written procedures for the entire process, and another 3 only had written procedures for the review and clearance phase. Written procedures could help components define management roles in decisions to initiate development of guidance documents, prioritize among them, and determine their appropriate level of review to manage risk. Further, not all components documented approval for guidance clearance and nearly half of them did not regularly evaluate whether issued guidance was effective and up to date. Opportunities exist for components to strengthen their internal controls. For example, components could adapt practices that others already use and have found to be an effective use of resources. Wider adoption of these practices could better ensure that components have internal controls in place to promote quality and consistency of their guidance development processes. To be effective, guidance documents must also be accessible by their intended audiences. The departments and components primarily relied on their websites to disseminate guidance. Consequently, components’ application of relevant federal guidance and best practices for web dissemination is particularly important for ensuring that the intended audiences can access and are aware of these documents. Certain component websites for disseminating guidance were easy to use—for example, because guidance was well organized or clearly marked—but others were hard to navigate or did not effectively distinguish between current and outdated guidance. Further, components did not always leverage the web and customer satisfaction metrics they collected to evaluate their guidance and its dissemination. By more consistently analyzing the metrics they have already collected, components could better ensure that their online guidance is easy to access, accurate, and relevant. Ensuring effective mechanisms for affected parties and other stakeholders to submit feedback on guidance documents is also crucial. Opportunities for feedback on issued guidance are important, not only because public comments and questions are often the impetus for components initiating new or revised guidance, but also because components we reviewed did not consistently take steps to confer with external stakeholders while guidance was being drafted and reviewed. To better ensure the adherence to requirements for approval and public access to and feedback on significant guidance in accordance with OMB’s Final Bulletin for Agency Good Guidance Practices (M-07-07), we recommend that the Secretary of HHS take the following two actions: 1. Develop written procedures for the approval of significant guidance documents. 2. Ensure that the department’s significant guidance is accessible online and that the public can provide comments on significant guidance documents. To better ensure the adherence to requirements for approval of significant guidance in accordance with OMB’s Final Bulletin for Agency Good Guidance Practices (M-07-07), we recommend that the Secretary of Labor take the following action: 1. Review and update the department’s written procedures for approval of significant guidance and make them available to appropriate component staff. To improve agencies’ guidance development, review, evaluation, and dissemination processes for non-significant guidance, we recommend that the Secretaries of USDA, HHS, DOL, and Education take the following two actions: 1. Strengthen their selected components’ application of internal controls to guidance processes by adopting, as appropriate, practices developed by other departments and components, such as assessment of risk; written procedures and tools to promote the consistent implementation and communication of management directives; and ongoing monitoring efforts to ensure that guidance is being issued appropriately and has the intended effect. Examples of practices that could be adopted more widely include written procedures for guidance production to, among other things, clearly define management roles; improved communication tools, such as routing slips to document management review; and consistent and ongoing monitoring to determine if guidance is being accessed and having the intended effect. 2. Improve the usability of selected component websites to ensure that the public can easily find, access, and comment on online guidance. These improvements could be informed by the web and customer satisfaction metrics that components have collected on their websites. Some examples of changes that could facilitate public access to online guidance include improving website usability by clarifying which links contain guidance; highlighting new or important guidance; and ensuring that posted guidance is current. We provided a draft of this report to the Secretaries of Agriculture, Education, Health and Human Services, and Labor. We received written comments from Education, HHS, and DOL, which are reprinted in appendixes III, IV, and V, respectively. USDA provided oral comments. In addition, Education, DOL, and USDA provided technical comments, which we incorporated as appropriate. We also shared a copy of the report with the Office of Management and Budget, and incorporated its technical comments as appropriate. Education concurred with our recommendations. Education stated that, although it believes that its internal controls for developing and producing guidance are effective and that its online guidance can be easily accessed by the public, it is committed to continuously looking for opportunities to improve its processes. Education stated that it will review components’ procedures for guidance development and production and develop and provide to its components standard protocols they can use to clarify management roles, document management review and approval of guidance, and review posted guidance to ensure it is current and accessible to the public. In addition, Education will review the presentation of guidance on Education’s and its components’ websites and identify best practices to improve the online presentation and accessibility of guidance documents. HHS concurred with our recommendations. While HHS pointed to its established practices for developing and internally reviewing significant guidance, it stated that it would explore the best mechanism for distributing written procedures for approval of these documents. HHS noted that it regularly engages with the public and regulatory stakeholders to receive feedback and distributes its guidance in accordance with this feedback, but will work with its agencies to update links to published guidance and explore ways to make published guidance easier to find on HHS webpages. In response to our recommendation on internal controls, HHS stated that it will continue to work with its subagencies to share best practices and ensure that agency practices are aligned with departmental standards. HHS concurred with our recommendation on improving website usability and stated that it will review current links to guidance documents and explore ways to enhance their visibility and usability. Labor concurred with our recommendations. Labor stated that it will update the department’s written procedures for the approval of significant guidance, disseminate them to component agencies, and ensure they are easily accessible. In addition, Labor stated that it will work with component agencies to share best practices and promote more consistent application of internal control standards in the guidance production process and encourage agencies to consider website improvements and better use web metrics to ensure access and public comments on guidance. On March 19, 2015, USDA officials representing the Food and Nutrition Service and the Department’s Office of Budget and Program Analysis provided oral comments on the report. USDA generally concurred with our recommendations. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretaries of Agriculture, Education, Health and Human Services, and Labor, and other interested parties. We are also sending copies of this report to the appropriate congressional committees. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-6806. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI. You asked us to examine guidance processes at the four departments under your jurisdiction. For this report, we reviewed how the Departments of Agriculture, Education, Health and Human Services, and Labor and selected subagencies or components at these four departments (1) use guidance and the processes and criteria they use to determine whether to issue guidance or undertake rulemaking; (2) follow applicable criteria and leading practices in their policies, procedures, and practices for producing guidance; and (3) ensure they follow dissemination requirements and facilitate end users’ access to and comment on documents. We limited our review to guidance agencies provide to external parties. The scope of review included the 25 selected subagencies, or components, in the four selected departments that (1) were within the requesting committee’s jurisdiction, and (2) engaged in regulatory or grantmaking activities, as the components engaged in these activities were likely to issue guidance interpreting regulations or other requirements to external parties (in contrast to agencies that issue only informational guidance or other resources). To identify these components, we searched in the Federal Register and the Catalogue of Federal Domestic Assistance to confirm regulatory and grantmaking activities. We confirmed the resulting list of identified components with department officials. See table 5 below for the full list of audited department components. We determined that computer-processed data were not expected to materially affect our findings, conclusions, or recommendations, thus rendering a data reliability assessment unnecessary. We used computer-processed data solely to confirm that the departments and components within our scope engaged in regulatory or grant-making activities. To describe how selected department and component processes used guidance and the processes and criteria they used to determine whether to issue guidance or undertake rulemaking, we reviewed agency written procedures, guidance documents, and websites. We also interviewed department and component officials on guidance practices.themes and examples from our documentary and testimonial evidence for all objectives, we analyzed information from relevant documents and interviews to identify and confirm common patterns as well as differences across selected agencies. To evaluate whether agency policies, procedures, and practices for producing guidance followed applicable criteria and leading practices, we assessed the extent to which agencies adhered to requirements for written procedures for approval of significant guidance under the Office of Management and Budget’s (OMB) Final Bulletin for Agency Good Guidance Practices (OMB Bulletin). To do so, we reviewed applicable agency written procedures and agency websites and spoke to officials about their practices for development and review of significant guidance. Office of Management and Budget, Final Bulletin for Agency Good Guidance Practices, 72 Fed. Reg. 3432 (Jan. 25, 2007). authority, addressed in our discussion of the level of review of guidance, are reflected in our application of the other internal controls. To ensure we applied selected internal controls to guidance processes appropriately, we reviewed applicable literature and spoke to OMB staff and legal scholars identified due to their published work on the subject. In addition, we spoke with officials at the Food and Drug Administration (FDA) to gain a better understanding of their related statutory requirements for guidance. We spoke with FDA because certain provisions of the OMB Bulletin were informed by written FDA practices for the initiation, development, issuance, and use of their guidance documents. To evaluate guidance dissemination strategies, we assessed the extent to which departments adhered to OMB’s public feedback and comment requirements; reviewed agency websites and digital government strategy reports; evaluated written statements from officials on components’ use of web and customer satisfaction metrics; and interviewed relevant agency officials. We used Guidelines for Improving Digital Services developed under the President’s digital government strategy to assess the usability of component websites for accessing guidance documents. We conducted this performance audit from March 2014 to April 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. This appendix describes guidance processes at the 25 components we reviewed. Agency profiles include the following information: Component Mission. In almost all cases we used information from the 2013 United States Government Manual. Target Audience and Dissemination Methods. Component officials identified their intended audience and dissemination methods. We did not corroborate this testimonial evidence. Yes/No Questions. We asked the agency for evidence of these practices and reported on our analysis based on this support. Agency Use of Guidance. We relied on component officials to explain the types of guidance they use most often and how they use these types of guidance. In most cases, we corroborated this by reviewing agency websites. Guidance Processes (Initiation, Development, Review, Dissemination, and Feedback on Guidance and Dissemination). We provided information from component interviews about processes for initiation, development, review, dissemination, and tracking and evaluation. We did not corroborate this testimonial evidence. In cases in which components referred to written procedures, tracking sheets, or formalized review processes, we requested and reviewed relevant documentation. Highlighted Practices. In certain cases, we presented practices identified by components that (1) applied or strengthened identified internal controls and (2) could be helpful to other components if similarly adopted. These practices are not exhaustive, but rather are meant to illustrate useful practices. Food and Nutrition Service (FNS) Overview What FNS Does FNS administers the Department of Agriculture’s (USDA) domestic food assistance programs. Target Audiences Agency officials told us the guidance issued by FNS was targeted to state agencies, local agencies, and other partner organizations to assist them with policy implementation and compliance with regulations. 3. General Guidance: Covered such topics as eligibility, program management, breastfeeding, and nutrition. Initiation, Development, Review, and Dissemination Initiation: FNS officials told us that guidance was typically initiated and drafted at headquarters by program staff. Guidance was often used to interpret regulations. Officials at the FNS Special Supplemental Nutrition Program for Women, Infants and Children (WIC) told us that they issued guidance in response to the release of new regulatory and legislative provisions or regulations issued by another agency that may have affected WIC programs. They also issued guidance in response to input from FNS regional offices as well as questions and other input from WIC state agencies to clarify requirements and expectations for program operations and directives from senior management, or in response to Inspector General or GAO audits. Dissemination methods FNS website, hard copy (pre-2004 guidance), PartnerWeb internal website, and e-mails to regional offices. Development and Review: FNS officials told us that after guidance was drafted and depending on the nature of the guidance, agencies other than the Department of Agriculture (USDA) FNS may be consulted or contacted as appropriate. Guidance was cleared by the Deputy Administrator for Supplemental Nutrition and Safety Programs and Office of General Counsel. FNS obtained input from General Counsel to ensure that it did not overstep its legal authority as provided under the program’s governing legislation. Guidance was cleared by the Deputy Administrator, the Administrator, or the Undersecretary if it contained sensitive issues of interest to management or had broader implications. Guidance documents that contained specific, extensive requirements for state agencies administering the relevant program went through the same review process as for rulemaking, going first to the division director and then to the Chief of the Division of Planning and Regulatory Affairs for clearance. The clearance process was tracked within the agency using an e-routing system. FNS used a routing slip that accompanied all guidance documents for clearance through the Administrator’s Office. Food and Nutrition Service (FNS) Initiation, Development, Review, and Dissemination, cont. website. Afterwards, the officials notified state offices. FNS WIC officials told us that sometimes guidance documents were e-mailed to offices that did not have consistent access to the website, such as the Indian Health Service offices. FNS also sent out notices if information published in the Federal Register affected state program implementation. USDA officials told us that because their programs were high profile, they got a lot of public comments. They added that there was no routine process for reviewing public comments on guidance received through USDA’s website. FNS also received feedback from state and local agencies through its regional offices, which had a program-specific point of contact for the agencies. FNS officials told us that going through the regional offices was the most effective way to receive feedback from state agencies (i.e. the end users), as the regional offices worked with the states doing day-to-day technical assistance and monitoring and evaluation. State agencies contacted their regional offices when they had difficulty accessing the web site or had questions about issued guidance and policies. FNS officials told us this type of feedback served as a reference point for future documents. FNS also occasionally issued optional surveys or asked regional offices to poll state agencies. Supplemental Nutrition Assistance Program: Enhanced Detection Tools and Reporting Could Improve Efforts to Combat Recipient Fraud. GAO- 14-641. Washington, D.C.: August 21, 2014. School-Meals Programs: USDA Has Enhanced Controls, but Additional Verification Could Help Ensure Legitimate Program Access. GAO-14-262. Washington, D.C.: May 15, 2014. School Lunch: Implementing Nutrition Changes Was Challenging and Clarification of Oversight Requirements Is Needed. GAO-14-104. Washington, D.C.: January 28, 2014. School Lunch: Modifications Needed to Some of the New Nutrition Standards. GAO-13-708T. Washington, D.C.: June 27, 2013. WIC Program: Improved Oversight of Income Eligibility Determination Needed. GAO-13-290. Washington, D.C.: February 28, 2013. Office for Civil Rights (OCR) Overview What OCR Does OCR’s mission is to ensure equal access to education and to promote educational excellence through enforcement of civil rights. It serves student populations facing discrimination and the advocates and institutions promoting systemic solutions to civil rights problems. OCR also provides technical assistance to help institutions achieve voluntary compliance with the civil rights laws that OCR enforces. Agency Use of Guidance 1. Dear Colleague Letters and Frequently Asked Questions: Covered topics related to OCR-enforced regulations. 2. Pamphlets: Offered general informational guidance directed toward a wider audience, typically students and parents. Initiation, Development, Review, and Dissemination Initiation: OCR relied on field offices, other programs, and the public for ideas for new guidance. OCR used its Program Legal Group (PLG) to collect and assess the need for new guidance. PLG submitted guidance ideas to OCR leadership for approval. Some factors considered to determine whether to issue guidance included: (1) the number of people who would be affected; (2) the need for guidance; and (3) how resource intensive guidance development would be, such as the availability of technical expertise or amount of collaboration needed. School officials, parents, students, and the general public. Development and Review: OCR informally involved external stakeholders, such as associations representing educational institutions and officials, students, and civil rights advocates, and has used listening sessions to get their views on draft guidance. However, it did not share drafts externally until they were finalized. OCR followed the department’s clearance processes for all of its guidance. Review decisions documented? Yes ✔ No Dissemination methods Website, e-mails, listservs, conferences, webinars, press conferences/releases, social media, and external partners. Dissemination: OCR disseminated guidance through its website, press releases, e-mail listservs, and social media. OCR has scheduled calls with press and stakeholders as necessary to announce the public release of guidance. For guidance documents written for school officials, OCR has created and distributed a less technical fact sheet to describe the guidance for parents and school staff. OCR staff also disseminated and answered questions about guidance during conferences, webinars, and through the provision of technical assistance. Feedback on Guidance and Dissemination OCR officials monitored (1) statistics on the number of clicks on e-mailed links, (2) the number of website hits, and (3) mentions of guidance in the news and specialized publications. OCR also distributed feedback forms on its guidance after technical assistance sessions and webinars. Examples of Recent Guidance-Related GAO Reports Child Welfare: Federal Agencies Can Better Support State Efforts to Prevent and Respond to Sexual Abuse by School Personnel, GAO-14-42 (Washington, D.C.: January 27, 2014). Charter Schools: Additional Federal Attention Needed to Help Protect Access for Students with Disabilities, GAO-12-543 (Washington, D.C.: June 7, 2012). Students with Disabilities: More Information and Guidance Could Improve Opportunities in Physical Education and Athletics, GAO-10-519 (Washington, D.C.: June 23, 2010). Office of Career, Technical, and Adult Education (OCTAE) Overview What OCTAE Does OCTAE administers and coordinates programs that are related to adult education and literacy, career and technical education, and community colleges that enable adults to acquire the basic literacy skills necessary to function in today’s society. OCTAE also helps students acquire academic and technical skills and prepare for high-skill, high-wage, and high-demand occupations in the 21st-century global economy. OCTAE provides national leadership and works to strengthen the role of community colleges in expanding access to postsecondary education for youth and adults and in advancing workforce development. Agency Use of Guidance 1. Frequently Asked Questions: Guidance relevant to OCTAE’s two formula grant programs. 2. State Plan Guidance: Information provided to states on the requirements for plans that must be submitted before a state can access federal funding under certain programs. 3. Reports and Resources: Fact sheets that highlight efforts currently underway that support OCTAE’s top priorities, and reports to Congress on state performance under the Carl D. Perkins Vocational and Technical Education Act and the Adult Education and Family Literacy Act. OCTAE also published research and data. Target Audiences The primary audience for guidance is states that are grantees of OCTAE’s formula and discretionary grant programs. Written guidance review policy? same departmental process for significant and non-significant guidance. Program officials consulted with policy staff, the Office of the General Counsel (OGC), and the Office of Planning, Evaluation, and Policy Development on guidance development. The draft guidance then went to OCTAE’s executive officer, the Chief of Staff, and the Deputy Assistant Secretary for review. The Office of the Assistant Secretary cleared the guidance and consulted with the OGC to determine if the guidance was significant. Review decisions documented? Yes ✔ No Dissemination: OCTAE sent guidance directly to state directors via OCTAE’s listserv. They also posted guidance to OCTAE’s website and its blog. OCTAE also included links to new guidance in its newsletter. OCTAE officials tracked and evaluated users’ awareness and understanding of existing guidance through interactions with grantees including monitoring visits, regular telephone communications, audits, and meetings and conferences. Used web metrics to change online guidance dissemination? Office of Elementary and Secondary Education (OESE) Overview What OESE Does OESE directs, coordinates, and formulates policy relating to early childhood, elementary, and secondary education. OESE also focuses on supporting states, school districts, and schools in improving K–12 education; providing children with language and cognitive development, early reading, and other readiness skills; and improving the quality of teachers and other instructional staff. Agency Use of Guidance 1. Frequently Asked Questions (FAQ): Issued to states and school districts to answer questions spanning multiple programs. 2. Dear Colleague Letters: Addressed issues concerning a particular program or multiple programs outlining new policies. Initiation, Development, Review, and Dissemination Initiation: OESE officials said that they often produced guidance in response to grantee inquiries or to questions from stakeholders. Further, OESE may receive feedback from quarterly meetings with grantees and technical advisors which result in the development of guidance. Target Audiences State and local educational agencies, school administrators, teachers and support staff, parents, and the general public. Development and Review: OESE officials told us that they follow the departmental guidance on determining whether other guidance should be categorized as significant or non-significant. Officials said that they have formed a working group to draft and review significant guidance that included the Office of the General Counsel and staff from OESE and other components. OESE subcomponents were responsible for the drafting of non-significant guidance. From there, the review and approval process differed depending on the type and content of the guidance. OESE has developed an internal guide to advise staff on who should be approving different types of guidance and the time frames required for that review. See the highlighted practices section on the next page for more information. Dissemination methods OESE website, e-mails, webinars, and newsletters. Dissemination: Officials said that OESE typically posted guidance on its website and issued a message from either the Assistant Secretary or the Director of the issuing office to grantees or state contacts stating that the new guidance was available. OESE also held webinars and frequently communicated with national associations to provide information about the guidance. In addition, OESE has highlighted new guidance in the department’s “ED Review” newsletter. Finally, OESE typically held quarterly calls with grantees and technical assistance advisors who suggested possible improvements to guidance on grant program implementation. Feedback on Guidance and Dissemination Officials noted that OESE did not formally track how it disseminates guidance although some offices within the component may have their own tracking systems. OESE officials said that they met with national associations each month to discuss clarifications and technical issues. OESE sought feedback from grantees about their experiences with receiving guidance and have used those responses to improve dissemination strategies. Office of Elementary and Secondary Education (OESE) Highlighted Practice In addition to using a routing slip to track the clearance of draft guidance, OESE has developed an internal guide to advise staff on who should be approving different types of guidance and the time frames required for that review. For example, this document calls for a 3-to-5 business day window for the Office of the Secretary to approve nonregulatory guidance and FAQs. There is also an internal contact identified who is responsible for coordinating clearances within the Office of the Assistant Secretary. Feedback on Guidance and Dissemination, cont. OESE officials also noted that they received frequent and helpful feedback on guidance directly from the public through e-mails and phone calls. A point of contact is identified on each piece of guidance to whom feedback could be submitted. Additionally, OESE staff regularly informed departmental leadership and policy officials of the need to revise guidance when gaps and inconsistencies were identified. Federal Student Aid (FSA) Overview What FSA Does FSA partners with postsecondary schools and financial institutions to deliver programs and services that help students finance their education beyond high school. This includes administering postsecondary student financial assistance programs authorized under Title IV of the Higher Education Act of 1965, as amended. Agency Use of Guidance 1. Electronic Announcements: Announced administrative information. 2. Operational and Policy Clarifications: Clarified issues such as reporting requirements. Initiation: FSA officials told us that questions from schools, program partners, and the public often serve as the impetus for FSA guidance. FSA officials met with OPE staff to discuss how to address questions that have been raised and whether it was appropriate to issue guidance. FSA took the lead on developing guidance that was operational in nature, while OPE was the lead if the guidance was policy related. Guidance is typically developed in a collaborative manner and often includes Education’s Office of the General Counsel. Target Audiences The primary audience for FSA guidance is students, parents, borrowers, financial institutions, counselors, mentors, schools, and institutions of higher education that disburse Direct Loans and other federal aid authorized under Title IV. Other audiences include postsecondary education associations and interest groups, lenders and guarantors, federal and state agencies, legal rights advocates, and Title IV program partners. Development and Review: While operational guidance may be signed by managers, officials told us that any guidance of importance was reviewed at all levels. FSA officials told us they do not generally work with external stakeholders during the development of electronic announcements. Dissemination: According to FSA officials, FSA’s primary mechanism for disseminating guidance was through the Information for Financial Aid Professionals website at http://www.ifap.ed.gov/ifap/index.jsp. Officials said all finalized guidance was posted on this website, which featured a “What’s New” page on which guidance issued in the past 14 days was posted. FSA also used e-mail blasts and newsletters to send out new guidance and disseminated guidance during trainings and webinars. According to FSA officials, FSA managers and staff have considerable interaction with program participants, program partners, and recipients. FSA receives a significant amount of feedback informally. Feedback also came through FSA customer care centers. Websites, e-mail blasts, webinars, and newsletters. Managing for Results: Selected Agencies Need to Take Additional Efforts to Improve Customer Service. GAO-15-84. Washington, D.C.: October 24, 2014. Used web metrics to change online guidance dissemination? Department of Education: Improved Oversight and Controls Could Help Education Better Respond to Evolving Priorities. GAO-11-194. Washington, D.C.: February 10, 2011. Higher Education: Stronger Federal Oversight Needed to Enforce Ban on Incentive Payments to School Recruiters. GAO-11-10. Washington, D.C.: October 7, 2010. Office of Innovation and Improvement (OII) Overview What OII Does OII oversees competitive grant programs that support innovations in the educational system and disseminates lessons learned. OII administers, coordinates, and recommends programs and policy for improving the quality of activities designed to support and test innovations throughout the K–12 system in areas such as parental choice, teacher quality, use of technology in education, and arts in education. Agency Use of Guidance 1. Policy Guidance and Policy Letters: Intended to help state and local educational agencies, educational service agencies, consortia of these agencies, nonprofit organizations or institutions of higher education that receive federal financial assistance to fulfill their obligations under specific federal laws and regulations. 2. Publications: Included guides, booklets, fact sheets, and brochures on a variety of topics, such as innovative school models, promising practices, school choice, private education, and supplemental educational services. 3. Frequently Asked Questions (FAQ): Related to grant competitions and new statutory requirements. Target Audiences The audience for OII guidance includes parents, administrators, teachers, students, and grantees/applicants. Initiation, Development, Review, and Dissemination Initiation: OII officials told us that grant competitions were the impetus for most guidance it issues. In addition, questions from external stakeholders have led OII officials to initiate new guidance or use FAQs to quickly respond to questions. When deciding whether to issue a rule or guidance, officials noted working with the Department of Education’s (Education) Office of the General Counsel (GC). However, if OII made legally binding programmatic changes, officials would use the rulemaking process. Officials also stated that they worked with GC on all guidance to ensure they were providing clear and accurate information to their grantees and to the field. Website, e-mail blasts, listserv announcements, and press releases. Used web metrics to evaluate online guidance dissemination? Yes No ✔ Used web metrics to change online guidance dissemination? Development and Review: Program staff were responsible for drafting guidance in consultation with GC and budget officers. Drafts were shared with the office director and submitted to the OII Executive Office where they were reviewed for clarity and consistency with other initiatives. The draft guidance was then reviewed by the Assistant Deputy Secretary, Associate Assistant Deputy Secretary, and Executive Officer who then provided feedback to program staff to make required revisions. OII officials said that they did not use a routing slip to document concurrence on draft guidance. Once approved, the guidance was sent to the Office of the Executive Secretariat in the Office of the Secretary where it was finalized (see figure 7). This process was also used for the development and approval of significant guidance. For OII’s significant guidance, the Office of the Executive Secretariat sought reviews from relevant Education components and then GC forwarded it to OMB for review. Officials told us external stakeholders were not involved in commenting on drafts before issuance. Dissemination: OII officials told us that they posted all guidance on their website and notified the public about new guidance in a number of ways. For example, OII sent out links to the guidance to its listserv subscribers and used e-mail blasts to inform interested parties of the issuance of the guidance. Office of Innovation and Improvement (OII) Initiation, Development, Review, and Dissemination, cont. Agency Use of Guidance 1. Privacy Technical Assistance (TA): Provided TA through responses to written inquiries, e-mails, and telephone calls on the Family Educational Rights and Privacy Act (FERPA), the Protection of Pupil Rights Amendment (PPRA), and the military recruiter provisions of the Elementary and Secondary Education Act of 1965, as amended (ESEA). 2. Dear Colleague Letters: Addressed topics related to privacy matters. 3. Frequently Asked Questions: Answered questions related to privacy matters. Target Audiences School officials and parents. Initiation, Development, Review, and Dissemination Initiation: OM created new guidance based on feedback or questions from the field, and addressed statutory and regulatory amendments made to the laws it administers. OM held weekly meetings to discuss complex inquiries received from the field and to address the need for any new guidance. Review decisions documented? Yes ✔ No Website, listservs, webinars, and conferences. Development and Review: OM officials told us that they follow departmental guidance on determining whether other guidance should be categorized as significant or non-significant. OM developed guidance documents on FERPA, PPRA, and ESEA military recruiter provisions. A working group provided input into the draft guidance documents and recommended the appropriate level of departmental review. This group included officials from OM and the Office of the General Counsel (OGC) and other department program offices. OM has reached out to other federal agencies when necessary. For example, OM has worked with the Department of Justice on juvenile justice issues and with the Department of Health and Human Services (HHS) on guidance on the amendments made to FERPA by the Uninterrupted Scholars Act. Together, OM and HHS hosted a joint webinar on this amendment. OM also circulated draft guidance to OGC and appropriate components for review. OM followed the department’s significant guidance clearance process for all documents considered significant by OMB. Generally, non-significant guidance documents received a less formal clearance process. Dissemination: OM officials told us that they posted guidance on their website and e-mailed it out through a listserv. OM also introduced new guidance through webinars and conferences. Feedback on Guidance and Dissemination OM officials told us that they have received feedback on guidance through e-mails, interactions during conferences, and through OM’s technical assistance network. Office of Postsecondary Education (OPE) Overview What OPE Does OPE formulates federal postsecondary education policy and administers programs that address national needs in support of the mission to increase access to quality postsecondary education. Agency Use of Guidance 1. Dear Colleague Letters: Clarified regulations. 2. Electronic Announcements: Provided administrative information. 3. Frequently Asked Questions: Answered questions on postsecondary education policies and programs. Target Audiences The primary audience for OPE guidance is entities and individuals involved in postsecondary education. This includes institutions of higher education, postsecondary education associations and interest groups, lenders and guarantors, students, federal and state agencies, legal rights advocates, program partners, and grantees. Initiation, Development, Review, and Dissemination Initiation: OPE officials told us that questions from schools, program partners, and the public were typically the impetus for issuing guidance. This was particularly true if OPE had received a number of similar questions on a topic. Typically, it decided to issue guidance in close consultation with the department’s Office of the General Counsel (OGC). OPE officials also said that agency leadership initiated guidance development to accomplish administration priorities. Written guidance review policy? Review decisions documented? Development and Review: OPE officials told us that they used a standardized process to review all guidance and used a routing slip to document the review. OPE officials told us they used different routing slips for higher education program guidance and for guidance related to policy, planning, or innovation. OPE officials reviewed the draft guidance and, after final approval from the Assistant Secretary, submitted it for further review in the department and at OGC. OPE officials did not generally work with external stakeholders during the development of Dear Colleague letters and electronic announcements. However, if the guidance addressed an operational issue, OPE may consult with the National Association of Student Financial Aid Administrators or other relevant nongovernmental organizations. Website, e-mails, and conferences. Used web metrics to evaluate online guidance dissemination? Used web metrics to change online guidance dissemination? Dissemination: According to agency officials, OPE’s primary mechanism for disseminating regulatory guidance was through the Information for Financial Aid Professionals website. Guidance to grantees was mainly distributed through the OPE website. For example, OPE distributed guidance through national conferences for grantees, e-mail blasts to inform grantees of program changes and upcoming grant competitions, technical assistance webinars, and newsletters. OPE officials also said that they may contact external stakeholders (including those that represent students) as new guidance is being released to explain the guidance, establish the objective of issuing the guidance, and answer any questions that the stakeholders may have. OPE said guidance that has been superseded by new guidance was clearly marked accordingly to prevent confusion about which policies were in effect. Office of Postsecondary Education (OPE) OPE officials described a variety of ways in which they obtained feedback on guidance dissemination. For example, OPE was able to track how many members were on its listserv and to track undeliverable e-mail. According to OPE officials, program staff had considerable interaction with people in the field and they received a significant amount of feedback informally. Postsecondary Education: Many States Collect Graduates' Employment Information, but Clearer Guidance on Student Privacy Requirements Is Needed. GAO-10-927. Washington, D.C.: September 27, 2010. Grant Monitoring: Department of Education Could Improve Its Processes with Greater Focus on Assessing Risks, Acquiring Financial Skills, and Sharing Information. GAO-10-57. Washington, D.C.: November 19, 2009. Office of Special Education and Rehabilitative Services (OSERS) Overview What OSERS Does OSERS helps ensure that people with disabilities have equal opportunities and access to education, employment, and community living. Its mission includes administering the Individuals with Disabilities Education Act and the Rehabilitation Act of 1973. These statutes help states meet early intervention and educational needs of disabled children and youth, support state and private programs that provide people with disabilities the resources they need to gain meaningful employment and lead independent lives, and support research and development programs. Agency Use of Guidance 1. Dear Colleague Letters: Highlighted departmental or administration initiatives or provided program-related information. 2. Grants and Funding: Provided information on grants, funding opportunities, and other resources. 3. Frequently Asked Questions: Contained issue-specific guidance about a pending funding opportunity, provided background information, or clarified certain topics. 4. Model Individualized Education Program (IEP): Created in response to a statutory mandate to be used by advocates, parents, grantees, and school administrators. Initiation, Development, Review, and Dissemination Initiation: OSERS officials said that guidance was often produced in response to grantee inquiries or questions from stakeholders. OSERS has received feedback from quarterly meetings with grantees and technical assistance providers which resulted in the development of guidance. Further, OSERS has identified the need to clarify or issue guidance during the course of monitoring its grant programs. It has also convened a focus group with external stakeholders to identify needed guidance. Target Audiences Educational administrators, vocational rehabilitation administrators, grantees and potential applicants, and the special education and vocational rehabilitation communities, including advocates and parents of students with disabilities. Development and Review: OSERS officials told us that they followed departmental guidance on determining whether documents should be categorized as significant or non-significant. OSERS has an informal process for developing and reviewing guidance. Officials explained that there was no need to have a documented process because it had a congenial and close group of experienced OSERS staff with a clear understanding of policy. As needed, OSERS staff coordinated with staff from other departmental components. Written guidance review policy? Yes No ✔ Review decisions documented? Yes ✔ No Dissemination methods Websites, newsletters, webinars, e-mails, and mail. Dissemination: Officials said that OSERS typically posted guidance on its homepage as well as on idea.ed.gov. OSERS also uses its listserv to announce new guidance to grantees or contact information. Additionally, program officers held meetings with project directors during which new guidance was announced. OSERS also held webinars and frequently communicated with state associations to provide information about guidance. Lastly, OSERS highlighted new guidance in the department’s “Ed Review” newsletter or in the OSERS’ monthly newsletter. Feedback on Guidance and Dissemination OSERS officials noted that they received frequent and helpful feedback on guidance directly from the public through e-mails and phone calls. Specifically, a point of contact was identified on each piece of guidance through which feedback could be submitted. Office of Special Education and Rehabilitative Services (OSERS) Examples of Recent Guidance-Related GAO Reports Charter Schools: Additional Federal Attention Needed to Help Protect Access for Students with Disabilities. GAO-12-543. Washington, D.C.: June 7, 2012. Students With Disabilities: More Information and Guidance Could Improve Opportunities in Physical Education and Athletics. GAO-10-519. Washington, D.C.: June 23, 2010. Administration for Children and Families (ACF) Office of Child Care (OCC) OCC supports low-income working families by providing access to affordable, high-quality early care and afterschool programs. OCC administers the Child Care and Development Fund (CCDF) and works with state, territory, and tribal governments to provide support for children and their families juggling work schedules and struggling to find child care programs that will fit their needs and that will prepare children to succeed in school. transmit requirements to grantees: Information Collections: Typically related to reporting requirements for grantees. Information on Related Legislation: Used to transmit information about new legislation that affected the program. For example, OCC used PIs to issue information on American Recovery and Reinvestment Act (ARRA) funding. OCC has also issued PIs to grantees about targeted funds appropriated for specific activities. 2. Information Memorandums (IM): Used IMs to emphasize leadership or other legislative priorities and changes, including recommendations or encouragements, flexibilities in use of funds, and related information on partner agencies. Audiences for OCC guidance include grantees, the 50 states, the District of Columbia, five territories and 260 federally-recognized tribes. (Some of these tribes represent consortia of tribes) 3. Policy Interpretation Questions: Provided policy guidance in response to questions from the field. OCC officials told us they rarely use this type of guidance. Written guidance review policy? Initiation, Development, Review, and Dissemination Initiation: Information guidance was typically developed in response to feedback provided by regional office officials about questions received from grantees. Officials at the central office held monthly calls with the regional offices and a biweekly call with the regional program managers to supplement day-to-day communication. Review decisions documented? Website, regional office meetings with state representatives, regional and national conference calls, and webinars. Used web metrics to evaluate online guidance dissemination? Development and Review: The OCC Director was involved in the development of OCC guidance and reported to the Deputy Assistant Secretary. The Assistant Secretary reviewed any guidance that was considered new, or novel. ACF’s General Counsel reviewed all OCC guidance. PIs were flagged for leadership review at the beginning of the clearance process. Officials typically created a routing slip for each document. Officials did not typically provide draft guidance to external stakeholders for comment prior to issuance. They often shared draft guidance with regional offices. OCC officials cleared documents with other ACF offices when the subject matter was directly relevant to other programs. Used web metrics to change online guidance dissemination? Dissemination: Because the audience for guidance was the OCC grantees, officials typically posted new guidance on their website and then e-mailed it to the grantees. If the guidance was of interest to grantees, they often held a conference call with the grantees to explain the guidance. Administration of Children and Families (ACF) Office of Child Care (OCC) Highlighted Practices The Child Care and Development Block Grant (CCDBG) Act of 2014 reauthorized the child care program for the first time since 1996. In response to new requirements outlined in the law, OCC created a “Reauthorization Resource” webpage. The page featured an overview of the law and detailed the new health and safety requirements for child care providers and changes to eligibility policies. It also provided answers to frequently asked questions (FAQs). In those FAQs, OCC explained provisions of the law, clarified who is affected by the law, and stated that more detailed guidance on effective dates for certain program requirements in the new law would be forthcoming. OCC officials told us they had processes to ensure the currency of their guidance, including labeling if the guidance was not current or applied only to a certain fiscal year. OCC also maintained a Technical Assistance Network that could identify implementation issues through its work with grantees. This allowed OCC to receive comments from grantees online. OCC regional offices communicated more with the grantees through quarterly calls and often relayed any issues to headquarters officials. Guidance documents listed regional office officials as the point of contact for questions. Because the guidance was largely informational, officials had not issued revisions to their guidance. Administration for Children and Families (ACF) Office of Head Start (OHS) OHS administers grant funding and oversight to the agencies that provide OHS services. OHS also provides federal policy direction and a training and technical assistance system to assist grantees in providing comprehensive services to eligible young children and their families. requirements and submissions essential to program function. OHS officials told us they used PIs when discussing requirements of the OHS Act. 2. Information Memorandums (IMs): Provided informational and qualitative updates. 3. Policy Clarifications: Responded to questions received through the Early Childhood Knowledge and Learning Center (ECLKC) website. Audiences for OHS guidance included Head Start grantees. Initiation, Development, Review, and Dissemination Initiation: OHS officials told us they initiated guidance in response to confusion identified in the field. In addition, officials might also issue a piece of guidance if there was a change to statute or regulation of which their grantees should be aware. Feedback leading to guidance came from a variety of sources, including regional offices, conferences, directly from programs, and from calls to congressional offices that were communicated to the office. Review decisions documented? Yes ✔ No Electronically though ECLKC and e-mails to grantees. Development and Review: OHS officials told us that guidance development started at the program office level unless, for some types of guidance, the decision was made to engage earlier with General Counsel (GC). The unit’s division director and the Office of the Executive Secretary approved guidance. Guidance then went to the Director of the Office of OHS, GC, and finally to the Deputy Assistant Secretary or Assistant Secretary of ACF. The guidance process was documented and a review slip was used (see figure 8 below). Officials told us that they input information on the potential guidance onto the ACF policy calendar so that ACF officials could determine who should review and approve the documents. If guidance was relevant to another federal agency, OHS would work with the agency to write and approve the guidance. Dissemination: OHS officials told us they disseminated guidance electronically through ECLKC, which provided all grantees with access to key documents. They maintained a directory of grantees and could send information to the entire directory (anyone could sign up to receive information). They may broadly disseminate items or share them only with management staff, depending on the content. They also sent e-blasts with the guidance based on the audience and subject matter. Administration of Children and Families (ACF) Office of Head Start (OHS) Initiation, Development, Review, and Dissemination, cont. Initiation, Development, Review, and Dissemination Initiation: Before drafting, ACL officials told us they typically checked whether similar guidance had been issued. This allowed them to take prior guidance into consideration and to consult with lawyers to check on any legal issues they needed to be aware of before drafting began. Review decisions documented? Website, regional office meetings with state representatives, regional conferences, webinars, and newsletters. Used web metrics to evaluate online guidance dissemination? Development and Review: ACL officials told us that they circulated guidance for internal review to center directors and ACL’s Executive Secretary using an e-mailed routing slip. Officials wrote a memorandum to accompany the guidance for review if background or additional context was needed. ACL officials involved their lawyers during the internal review process when legal interpretation was needed. During this phase, ACL officials discussed whether external review was needed. External reviews were not typical, but ACL worked with OMB to ensure that its guidance accurately reflected new OMB instructions and federal requirements for grant programs. Used web metrics to change online guidance dissemination? Dissemination: Officials told us that all guidance was posted on ACL’s website. ACL also distributed new and updated guidance through regional conferences, webinars, newsletters, and the Federal Register. Regional offices met with their states in group meetings to review new guidance each quarter. From there, it was the responsibility of state representatives to pass that information on to their respective partners, with whom they have had more direct contact. Officials explained that sub-grantees were identified in the law and thus easily identifiable. Administration for Community Living (ACL) Highlighted Practices Guidance production at ACL often included a discussion of how best to publish and disseminate that guidance. In one example, ACL officials used a decision memorandum to accompany draft guidance that explained to reviewers (1) the issue the policy guidance document addressed, (2) provided background on the policy and impetus for the guidance, and (3) requested the reviewer’s signature. The background discussion in the decision memorandum included a plan for posting the policy guidance on the website and a discussion of other dissemination methods to publicize the guidance. The decision memorandum suggested that a blog post could be developed to publicize the guidance, or that ACL staff could reach out to key stakeholders when the policy was posted. ACL officials stated that for any type of policy guidance they would generally discuss how best to disseminate it and would follow up with staff to provide technical assistance or answer questions. Feedback on Guidance and Dissemination ACL officials told us that they received feedback on their guidance in a number of ways. ACL conducted quarterly and biennial meetings with grantees during which guidance was discussed. The field office also organized conference calls that covered guidance. ACL officials stated that one-on-one interactions were the most effective manner to receive feedback. There were program updates for grant programs that were re- released each year. Officials told us that the most common reason for guidance updates was to address questions received from the states, territories, and tribal organizations or from their regional offices; however, they noted that guidance did not change frequently. ACL received information from its information technology and communications offices on website analytics. Meanwhile, it was also redesigning its website. ACL officials told us that web metrics will be an important tool used to guide how it shares its content, including guidance, once the site is redesigned. In addition, officials told us that when ACL issued state plan guidance on the Older Americans Act, ACL headquarters and regional staff coordinated the dates for release, the timeline, and the process staff would use as follow- up. Bureau of International Labor Affairs (ILAB) Overview What ILAB Does ILAB improves working conditions, raises living standards, protects workers’ rights, and addresses the workplace exploitation of children and other vulnerable populations. Target Audiences Grantees that are subject to cooperative agreements with ILAB, including non-governmental organizations and members of the public who have knowledge of labor conditions and practices in countries with which the U.S. signs free trade agreements. Guidance related to cooperative agreements was generally not intended for widespread public consumption. Covered the receipt and handling of public submissions on labor provisions of U.S. free trade agreements. Written guidance review policy? Yes No ✔ 3. Research-Related Resources Initiation, Development, Review, and Dissemination Initiation: ILAB officials told us that its program offices initiated the development process when guidance was needed. The impetus for new guidance could be grantee questions, or a program office could determine that an adjustment in language was needed to clarify a requirement or that a new requirement was needed. Most ILAB guidance clarified government-wide grant regulations and was prompted by a change in those grant regulations. ILAB officials told us little, if any, clarification was required if there had not been a change in regulation. Review decisions documented? Yes ✔ No Dissemination methods Website, listserv, and press releases. Development and Review: ILAB officials told us that once the guidance was drafted, the Office of the Solicitor reviewed it. If the guidance was related to grants, the grant office also reviewed it. External stakeholders were not typically involved in developing grant-related guidance. In one instance, ILAB put forward guidance related to submissions on labor provisions of U.S. free trade agreements through formal notice and comment in the Federal Register. In another instance, for research-related resources ILAB sought feedback from interagency partners and through a peer review by external experts from business, academia, unions, and civil society groups. Officials told us that they did not have written procedures for their guidance processes. Dissemination: ILAB officials told us that they disseminated guidance directly to grantees as part of their cooperative agreement and via the office’s webpage and various listservs. Grant solicitations were posted on grants.gov. The Department of Labor’s Office of Public Affairs could also issue a press release. Officials told us that grantees could provide feedback to ILAB through its project managers. Depending on the subject matter, ILAB could also obtain public feedback through publishing a notice in the Federal Register that solicits public comments. Officials considered this to be an effective practice. Bureau of International Labor Affairs (ILAB) Examples of Recent Guidance-Related GAO Reports International Labor Grants: DOL's Use of Financial and Performance Monitoring Tools Needs to Be Strengthened. GAO-14-832. Washington, D.C.: September 24, 2014. International Labor Grants: Labor Should Improve Management of Key Award Documentation. GAO-14-493. Washington, D.C.: May 15, 2014. Bureau of Labor Statistics (BLS) BLS collects, analyzes, and disseminates economic information to support public and private decision making. BLS serves as a statistical resource for the Department of Labor. Agency Use of Guidance BLS officials told us that BLS awarded cooperative agreements to state agencies to conduct two cooperative statistical programs- Labor Market Information and Occupational Safety and Health Statistics. BLS issued routine administrative memoranda that contained reporting requirements and closeout procedures targeted to grantees. Initiation, Development, Review, and Dissemination Initiation: Because guidance was routine and issued annually, BLS officials told us that their process did not involve a specific impetus for initiation. The audience for BLS administrative memoranda was the 50 state agencies and territories that receive cooperative agreement funds and BLS Regional Commissioners. Written guidance review policy? Development and Review: BLS officials told us the routine administrative memoranda were sent out to the appropriate offices for review, including the BLS Branch of Grants and Funds Management, the Office of Field Operations, and BLS program offices. In addition, officials told us BLS had written procedures for the development of guidance, but no external stakeholders were involved during development. Review decisions documented? Dissemination: BLS officials told us BLS administrative memoranda were posted on the internal “Stateweb” website and e-mailed to the state agencies. BLS provided an intranet link to its grantees to access the stored documents. These documents were neither disseminated to the general public nor posted to the BLS public website. Internal “Stateweb” web site and e- mails to grantees with an intranet link. Feedback on Guidance and Dissemination Officials told us that most administrative memoranda were issued annually, so it was unnecessary to revise or update them or issue correction memoranda. BLS officials told us that sometimes BLS got feedback or questions on funding that it then answered. We did not evaluate BLS’s use of web metrics because it did not use its public website to disseminate guidance. Employee Benefits Security Administration (EBSA) Overview What EBSA Does EBSA promotes and protects the retirement, health, and other benefits of the more than 141 million participants and beneficiaries in more than 5 million private sector employee benefit plans. EBSA develops regulations, assists and educates workers, plan sponsors, fiduciaries, and service providers, and enforces the law. The Employee Retirement Income Security Act is enforced through regional and district offices nationwide and a national office in Washington, D.C. Agency Use of Guidance 1. Compliance Assistance Documents: Typically issued in response to requests for advisory opinions and included advisory opinions, information letters, interpretations, frequently asked questions, and technical releases. 2. Field Assistance Bulletins: Typically issued in response to issues identified by EBSA personnel, including the regional and enforcement staff who review them. 3. Technical Guidance for Consumers: Typically provided information to the public and included brochures, handouts, participant information, and press releases. Target Audiences Guidance was targeted to employee benefit plan participants and beneficiaries, sponsors, administrators, fiduciaries, service providers including large financial services firms, institutional record keepers and asset custodians and their representatives, EBSA regional and enforcement staff, and individual members of the public. Initiation, Development, Review, and Dissemination Initiation: Officials told us that any official in EBSA could initiate an idea for a new piece of guidance. For example, each EBSA office had a chain of command that employees could use to suggest ideas. However, the Assistant Secretary decided whether to start developing guidance based on discussion at regular executive staff meetings or with office directors and their management staff. Officials told us EBSA sometimes issued companion guidance to documents issued by other employee benefit regulators, such as the Pension Benefit Guaranty Corporation or the Internal Revenue Service. Written guidance review policy? Yes No ✔ Review decisions documented? Yes ✔ No Dissemination methods EBSA website, e-mails to website listserv, press releases, e-mails to stakeholders, calls, webcasts, meetings with constituents, and presentations at industry meetings. Used web metrics to evaluate online guidance dissemination? Yes ✔ No Used web metrics to change online guidance dissemination? Development and Review: EBSA officials told us office directors considered legal, policy, and programmatic factors and then developed guidance proposals to present to EBSA’s Assistant Secretary and Deputy Assistant Secretaries for approval. Their procedures for guidance clearance were dependent on the type of guidance. Guidance interpreting regulations triggered a different level of review than informational guidance and was cleared through the Director of the Office of Regulations and Interpretations and/or the Director of the Office of Health Plan Standards and Compliance Assistance, the Office of the Solicitor, and the Deputy Assistant Secretary for Program Operations. EBSA’s Assistant Secretary cleared all guidance except for very routine matters and typically alerted department leadership upon release of the guidance. EBSA officials regularly discussed the status of draft guidance using a written agenda of pending regulations, exemptions, and active guidance products in a weekly meeting with key EBSA and departmental officials. Officials told us that the need for departmental review depended on various factors, including likely congressional interest, potential impacts on areas regulated by other Department of Labor (DOL) agencies, and expected media coverage. EBSA did not use a formal or codified routing slip. Instead, it used e-mail to contact the Assistant Secretary and Deputy Assistant Secretaries after guidance was developed and vetted through the appropriate national and/or regional office components and the Office of the Solicitor. Initiation, Development, Review, and Dissemination, cont. keepers and asset custodians, and the financial services industry. Dissemination to these groups was relatively easy. Officials told us this audience closely followed updates to the EBSA website and characterized this audience as resourceful and very vocal. EBSA officials usually knew whether the guidance had been received and was clear. However, officials also told us that EBSA had challenges reaching their other audience, which included small- to medium- sized employers and the participants in the 5-to-6 million existing employee plans. To reach this audience, they used a “multi-pronged strategy,” including posting guidance on their website, which has a dedicated page for guidance, e-mailing and meeting with stakeholders, and webcasts. Any special guidance could be posted on the “New and Noteworthy” portion of their website and could be e- mailed to EBSA’s website listserv, which had about 336,000 subscribers. Labor’s Public Affairs office assisted by drafting press releases and handling press calls. EBSA officials told us that EBSA sometimes revised and updated its guidance and identified which documents had been superseded by the new guidance. EBSA officials told us that both new and replaced guidance documents were posted on EBSA’s website, which was actively monitored by the regulated community and media outlets that focus on labor and benefit issues. Various media reports and benefit-specific websites could also provide information on new guidance. Officials also told us that the most effective means of soliciting feedback on guidance had generally been to post the guidance documents on EBSA’s website. To obtain input on how to improve the quality of guidance and gauge whether guidance has reached the intended audiences, EBSA met regularly with stakeholder associations, individual companies, and consumer groups. EBSA also participated in educational conferences sponsored by industry groups and interacted with the Employee Retirement Income Security Act Advisory Council. 401(K) Plans: Improvements Can Be Made to Better Protect Participants in Managed Accounts. GAO-14-310. Washington, D.C.: June 25, 2014. Private Pensions: Clarity of Required Reports and Disclosures Could Be Improved. GAO-14-92. Washington, D.C.: November 21, 2013. Private Pensions: Revised Electronic Disclosure Rules Could Clarify Use and Better Protect Participant Choice. GAO-13-594. Washington, D.C.: September 13, 2013. 401(K) Plans: Labor and IRS Could Improve the Rollover Process for Participants. GAO-13-30. Washington, D.C.: March 7, 2013. Employment and Training Administration (ETA) Overview What ETA Does ETA provides job training, employment, labor market information, and income maintenance services, primarily through state and local workforce development systems. ETA also administers programs to enhance employment opportunities and business prosperity. Agency Use of Guidance 1. Training and Employment Guidance Letters (TEGL): Issued to a broad audience of states and sub-grantees, they transmitted both policy and operational guidance. Often used for programs to identify funding allotments. 2. Training and Employment Notices (TEN): Informational notices directed to broad audiences. Officials told us that they used TENs to communicate technical assistance resources, publications, and updates on research and evaluation and the status of available agency issuances. Target Audiences Audiences for guidance at ETA are both narrow and broad, depending on the program. ETA funds American Jobs Centers, which has guidance targeting a narrow audience of those that administer these centers, while other programs are broader and have guidance that could be directed to a wide range of people. Other audiences include Workforce Investment Boards, governors, both discretionary grantees and formula subgrantees, and state workforce agencies, among others. 3. Unemployment Insurance Program Letters (UIPL): Interpreted statute or policy in the form of guidance. For example, when unemployment insurance was extended and then terminated, ETA issued guidance more than a dozen times from June 2008 to December 2013. 4. Program Information Notices (PINs): Information for Job Corps Centers, which were operated both by private contractors and the Forest Service in the Department of Agriculture in partnership with the Department of Labor (DOL). Written guidance review policy? Yes ✔ No 5. Frequently Asked Questions Initiation, Development, Review, and Dissemination Initiation: ETA officials told us they initiated guidance in response to issues that arose in the field, and worked with ETA program and policy officials as well as discussed the issue with their Office of the Solicitor (SOL) colleagues and the program office. Review decisions documented? Yes ✔ No Dissemination methods Website, e-mail blasts to the public, e-mails to regional offices, webinars, site visits, phone calls, and conferences. Development and Review: ETA used a standard operating procedure for reviewing guidance documents and a routing slip for internal review. Officials told us guidance went through multiple review processes. The officials included in the review process varied depending on the nature of the guidance. Usually, the program office, policy office, SOL, and leadership office all signed off on the guidance. The Office of the Assistant Secretary for Policy (OASP) signed off on guidance documents that were “major priorities.” Officials also discussed what level of review the guidance required with the Office of the Executive Secretary. Officials told us that when making review decisions they considered (1) the amount of money affected by the guidance, (2) how much of a priority the guidance was, and (3) whether other sub-agencies within DOL would be interested in the guidance. If the proposed guidance required internal clearance, they included OASP, the Offices of Congressional Affairs, the Secretary, and Public Affairs and alerted Cabinet Affairs staff. If regional offices would be affected, they concurred on the draft guidance. Employment and Training Administration (ETA) Initiation, Development, Review, and Dissemination, cont. sign up to receive an e-mail when a new advisory was available. Separate e-mails were sent to the regional offices, which then followed up with grantees about new guidance. ETA officials told us they also informed OASP and the Office of Congressional Affairs. ETA officials hosted a webinar to provide technical assistance and answer any questions that grantees had. The regional offices also had site visits and phone calls with grantees in which they discussed advisories and guidance as needed and occasionally attended conferences. Feedback on Guidance and Dissemination Officials told us they were asked to identify whether to continue, cancel, or rescind the guidance on the ETA website annually. They told us they routinely monitored grantees and had conversations with intergovernmental organizations to gain insights into potential changes to guidance. They received feedback from program offices, regional offices, intergovernmental organizations, and agency leadership on the content of their guidance. Workforce Investment Act: DOL Should Do More to Improve the Quality of Participant Data. GAO-14-4. Washington, D.C.: December 2, 2013. H-2A Visa Program: Modernization and Improved Guidance Could Reduce Employer Application Burden. GAO-12-706. Washington, D.C.: September 12, 2012. Mine Safety and Health Administration (MSHA) Overview What MSHA Does MSHA promulgates and enforces mandatory health and safety standards by thoroughly inspecting mines; targeting the most common causes of fatal mine accidents and disasters; reducing exposure to health risks from mine dusts and other contaminants; improving training, particularly for inexperienced miners and contractors; strengthening MSHA and the industry’s emergency response preparedness; enforcing miners’ rights to report hazardous conditions without fear of retaliation; and emphasizing prevention. MSHA also assists states in the development of effective state mine safety and health programs and contributes to mine safety and health research and development. Agency Use of Guidance 1. Program Information Bulletins: Provided information and best practices to mine operators, miners, and MSHA enforcement officials. 2. Program Policy Letters: Explained regulations to mine operators, miners, and MSHA enforcement officials. 3. Procedural Instruction Letters: Instructed MSHA’s staff on procedures for enforcing applicable standards. 4. Program Policy Manual: Consolidated MSHA policies. 5. Handbook: Provided instructions to MSHA inspectors and specialists. 6. Compliance Guidance and E-Laws: Posted on MSHA’s website. 7. Best Practice Pocket Cards: Provided miners with health and safety information, including an explanation of their rights 8. Frequently Asked Questions: Explained MSHA standards and regulations to operators and miners. Target Audiences Coal mine operators, metal and non-metal mine operators, unions, associations, and safety and health professionals. Initiation, Development, Review, and Dissemination Initiation: MSHA officials told us that new or revised guidance was typically initiated in response to questions from the field or issues identified by miners, operators, or MSHA Field Managers. MSHA officials discussed whether to issue guidance or undertake the rulemaking process with the Office of the Solicitor, the Office of Standards, Regulations, and Variances and the Office of the Assistant Secretary. MSHA also issued guidance as part of the normal rollout of new standards or regulations. Written guidance review policy? Review decisions documented? Dissemination methods Website, e-mails, compliance visits and other meetings, and through partners. Development and Review: MSHA had written procedures for guidance formulation, distribution, and maintenance. Administrators and Directors initiated guidance that was reviewed by appropriate officials. MSHA officials told us that guidance went through multiple reviews by affected programs, and significant guidance was flagged during the review process. For urgent guidance documents that need to be disseminated quickly (for example, hazard alerts and information on respiratory protective devices), the review process was shortened and senior management were involved earlier in the process. The Directorate of Program Evaluation and Information Resources (PEIR) coordinated and monitored guidance development and clearance. PEIR’s Office of Program Policy Evaluation (OPPE) officials managed the directives process and used a form to manage and track the review and dissemination of directives. Mine Safety and Health Administration (MSHA) Initiation, Development, Review, and Dissemination, cont. miners, labor organizations, industry associations, and other stakeholders when guidance was developed or to introduce new standards or regulations to explain and discuss the guidance. MSHA’s Field Managers also discussed guidance with miners and operators at multiple compliance visits each year. Feedback on Guidance and Dissemination MSHA officials told us they had developed procedures to ensure that programs periodically reviewed and updated guidance documents. They revised previously issued guidance if they determined that the guidance was out of date due to advances in technology or if other new information was available from stakeholders. MSHA officials told us that the most effective way to gather feedback on guidance is by speaking with stakeholders. MSHA officials told us that OPPE also had developed policies to review existing guidance to ensure that it was valid and that MSHA had made changes to its website to help the public easily find guidance information. Examples of Recent Guidance-Related GAO Reports Mine Safety: Basis for Proposed Exposure Limit on Respirable Coal Mine Dust and Possible Approaches for Lowering Dust Levels. GAO-14-345. Washington, D.C.: April 9, 2014. Mine Safety: Additional Guidance and Oversight of Mines' Emergency Response Plans Would Improve the Safety of Underground Coal Miners. GAO-08-424. Washington, D.C.: April 8, 2008. Occupational Safety and Health Administration (OSHA) Agency Use of Guidance Policy issuances—established internal policies or policy interpretations. OSHA assures safe and healthful working conditions for men and women by promulgating protective health and safety standards; enforcing workplace safety and health rules; providing training, outreach, education, and assistance to workers and employers in their efforts to control workplace hazards; preventing work-related injuries, illnesses, and fatalities; and partnering with states that run their own OSHA-approved programs. OSHA officials told us policy issuance documents have been used to explain internal procedures for inspections and interpretations of regulations for specific programs. An example of a policy issuance was the inspection procedure directive, which contains procedures used to investigate and cite violations of particular OSHA regulations. Non-policy issuances—provided information consistent with regulations. 1. Fact sheets, information sheets, hazard alerts, and small entity compliance guidance: Provided hazard identification and prevention information on critical safety and health hazards that often must be disseminated quickly. 2. Booklets: Provided information for constituents at all education levels. 3. Fatal Facts: Contained information about how to identify and prevent hazards that lead to fatalities at worksites. Written for employers, safety and health professionals, and workers. Employers and their representatives, such as trade associations; covered workers and their representatives, such as unions, community groups and worker centers; and other safety and health professionals. 4. Quick cards: Small laminated cards that provided safety and health information for employers, professionals, and workers with some safety and health background. 5. Low-literacy materials: For workers and employers with limited English proficiency and young workers. Written guidance review policy? Review decisions documented? 6. Letters of interpretation: Clarified ambiguities in regulations. Initiation, Development, Review, and Dissemination Initiation: OSHA officials told us that they were often prompted to issue or revise guidance for clarification in response to feedback from regional offices and external stakeholders with questions on existing guidance. OSHA website, hard copy delivery to area offices, mass mailings to employers, webinars, outreach in the school system, social media, newsletter (QuickTakes), and e- mail. Used web metrics to evaluate online guidance dissemination? Development and Review: Officials told us policy issuances were cleared by the Office of the Solicitor and OSHA leadership, and were sometimes sent to the Office of the Executive Secretariat for coordination of department-level review. OSHA program directors obtained input and technical and policy clearance for both policy and non-policy issuances from each of the other program directors and their directorate offices and resolved any comments. The final draft was sent to the Director of Administrative Programs for approval. The Deputy Assistant Secretary addressed unresolved disagreements concerning the substance or policy implications of proposed policy guidance. Officials told us that in some circumstances, OSHA sought expert input or input of the target audience for non-policy guidance materials to provide the most accurate and applicable information—and to make sure it is accessible—for workers and employers on a specific topic. Occupational Safety and Health Administration (OSHA) Highlighted Practices OSHA had separate written procedures in the form of instructions for both policy and non-policy issuances. Policy issuances are internal directives and supplementary guidance that have implications for internal statements of policy and procedure, while non-policy issuances are technical and educational guidance documents that provide information consistent with regulations and include such supplementary guidance materials as letters of interpretation and other non-policy statements issued by OSHA. These procedures outlined the roles and responsibilities for the Assistant Secretary of Labor for Occupational Safety and Health, OSHA program directors, and the Director of Administrative Programs and identified conditions under which guidance should be published. Initiation, Development, Review, and Dissemination, cont. produces an e-mail based newsletter called QuickTakes that publicized new policy and non-policy guidance documents. Area offices distributed new educational guidance materials to the stakeholders who were difficult to contact electronically. They also conducted mass mailings and webinars, posted on social media sites, and reached out at conferences and schools. Dissemination also occurred with cooperative program participants such as Alliance and Partnership members and state partners. Participants and partners disseminate materials and guidance to members and constituencies. Feedback on Guidance and Dissemination OSHA officials told us they tracked and evaluated guidance to determine whether to revise guidance. If a guidance product was written for a specific OSHA standard that had not changed, revisions were infrequent. Guidance would be updated if based on a standard that has changed or a hazard where new information was available to assure workers are protected. Workplace Safety and Health: OSHA Can Better Respond to State-Run Programs Facing Challenges. GAO-13-320. Washington, D.C.: April 16, 2013. Workplace Safety and Health: Further Steps by OSHA Would Enhance Monitoring of Enforcement and Effectiveness. GAO-13-61. Washington, D.C.: January 24, 2013. Office of Disability Employment Policy (ODEP) Overview What ODEP Does ODEP seeks to increase the number and quality of employment opportunities for people with disabilities by promoting the adoption and implementation of its policy strategies and effective practices and bringing focus to the issue of disability employment. Target Audiences People with disabilities, employers (in both the private and public sectors), service providers, and government entities. employers (in both the private and public sectors), service providers, and government entities. While ODEP does not have regulatory authority, it has assisted enforcement agencies in reaching stakeholders in the disability community regarding regulations that impact them. ODEP also provided input on guidance issued by other Department of Labor (DOL) components, such as the Employment and Training Administration, related to disabilities. Review decisions documented? Yes No ✔ Dissemination methods Website, e-mail, auxiliary websites such as Disability.gov, listening sessions, and webinars, social media, public service announcements, speaking engagements, and press releases. Guidance Initiation, Development, Review, and Dissemination Initiation: ODEP officials told us that items addressing how to comply with regulations, if developed, would emanate from ODEP’s policy or research team supervisors. The item would then require input and approval from other team leads at the agency, the Executive Officer, the Deputy Assistant Secretary, the Chief of Staff, and the Assistant Secretary. ODEP would work closely with DOL’s enforcement agencies and the Solicitor’s Office in developing all such items. All non-regulatory items produced for the public would emanate from ODEP’s policy, outreach or research team supervisors. The item would also require input and approval from other team leads at the agency, the Executive Officer, the Deputy Assistant Secretary, the Chief of Staff, and the Assistant Secretary. Development and Review: According to ODEP officials, all guidance was reviewed by either the Outreach Supervisor or the Executive Officer to determine the level of both internal or external review needed (both intraagency and interagency). Guidance was cleared internally by the Outreach Supervisor or the Executive Officer, all relevant Policy, Administrative and/or Research Supervisors, the Deputy Assistant Secretary, the Chief of Staff, and the Assistant Secretary. In addition, there is an expectation that items affecting stakeholders or touching legal or policy issues were cleared by the relevant DOL agencies, such as the Office of the Assistant Secretary of Policy, the Office of Public Affairs, the Solicitor’s Office, the Office of Congressional and Intergovernmental Affairs, the Office of the Secretary, or affected DOL program agencies. ODEP worked closely with all these agencies to determine if further external review was required, including the Office of Management and Budget or other relevant federal agencies. Office of Disability Employment Policy (ODEP) Initiation, Development, Review, and Dissemination, cont. disseminated guidance through their website. Officials used the “Gov Delivery” e-mail subscription service to disseminate guidance and information to about 50,000 subscribers. ODEP officials also reached out to their employer stakeholders and maintained an online community of practice. ODEP held primary responsibility for managing the Disability.gov website. In addition to these methods of dissemination, ODEP officials said that each of their four technical centers maintained its own website and offered webinars throughout the year. Feedback on Guidance and Dissemination ODEP officials told us that items shared with the public were reviewed regularly and updated so that the information was current and relevant. ODEP officials told us they had just started to widely use web metrics for their website and intended to use the information gathered to improve how they communicated with the public. Survey tools were consistently used to evaluate how Disability.gov served the public and that information was used to improve its service model. Office of Federal Contract Compliance Programs (OFCCP) OFCCP administers and enforces equal opportunity mandates prohibiting federal contractors and subcontractors from discriminating on the basis of race, color, religion, sex, national origin, disability, or protected veteran status and require federal contractors and subcontractors to take affirmative steps to ensure equal employment opportunities. anticipated to have operational impact or to result in enforcement action. 2. Federal Contract Compliance Manuals: Explained broad OFCCP policy. 3. Frequently Asked Questions (FAQ) 4. Fact Sheets and Brochures: Targeted to a more general audience or the public. 5. Technical Assistance Guides: Assisted federal contractors and subcontractors in complying with laws and regulations on employment discrimination and equal employment. The audience for most OFCCP guidance and technical assistance is contractors, particularly new contractors. Guidance was also targeted to communities from whom OFCCP most often received violation complaints. These communities represent all protected classes. 6. Equal Employment Opportunity Posters: Employers covered by non-discrimination and equal employment opportunity laws were required to display posters on their premises. Initiation, Development, Review, and Dissemination Initiation: OFCCP officials told us that after they identified an issue that might require guidance, they discussed it with the Office of the Solicitor (SOL). Written guidance review policy? Yes ✔ No Review decisions documented? Yes ✔ No Website, e-mails, webinars, public meetings, and media outreach. Development and Review: OFCCP officials told us that the national office typically developed guidance with feedback from SOL and comments from the regions. OFCCP guidance received varying levels of review based on the type of guidance, but SOL reviewed almost all guidance. Directives were more formal, used a template, and were always reviewed and signed by the OFCCP director. FAQs were generated internally and might not rise to the level of director review. Departmental officials and officials from other Department of Labor (DOL) or components reviewed guidance (1) on a sensitive subject, (2) for which heightened scrutiny was anticipated, (3) that might affect other DOL programs or agencies, (4) that might be considered newsworthy, or (5) that is part of an initiative of the Administration. OFCCP officials used a routing slip to document clearance and its processes for review were written in administrative procedures. See figure 12 below. Dissemination: OFCCP officials told us they had recently standardized the process to centrally monitor and control the guidance dissemination process. To supplement sending new guidance to the regional office, officials also sent an e-blast to more than 54,000 e-mail subscribers explaining (1) why guidance was issued, (2) what the guidance was, and (3) where the guidance could be found. If appropriate, the e-blast directed the subscriber to a hyperlink to guidance on OFCCP’s website. OFCCP also used media packets, toolkits, webinars, and public appearances to further publicize new guidance. OFCCP posted all guidance intended for the public online. Office of Federal Contract Compliance Programs (OFCCP) Highlighted Practices In 2011, OFCCP officials started a 2-year project to review their directives system. They told us that this effort was intended to make their guidance more accurate and correct. As part of these efforts, they identified necessary updates to guidance, clarified superseded guidance, and rescinded guidance when appropriate, reducing the original number of directives by 85%. Initiation, Development, Review, and Dissemination, cont. OLMS conducts criminal and civil investigations to safeguard the financial integrity of unions and to ensure union democracy. The Office conducts investigative audits of labor unions to uncover and remedy criminal and civil violations of the Labor-Management Reporting and Disclosure Act and related statutes and explains the reporting, election, bonding, and trusteeship provisions of the act. The OLMS promotes labor union and labor-management transparency through reporting and disclosure requirements. either stand alone or be issued to accompany a regulation. Standalone facts sheets often explained what statute required. 2. Guides for Union Officers: Provided general information on requirements that applied to unions and union officers, and offered suggestions on how to comply with those requirements. 3. Guidance: Information on how a complaint with OLMS could be filed. Initiation, Development, Review, and Dissemination Initiation: OLMS officials told us that the impetus for guidance varied. Guidance was initiated if (1) numerous unions had similar questions after a new regulation had been finalized about compliance, (2) officials had issued a regulation or were about to issue one, or (3) field personnel encountered a consistent issue through OLMS’s compliance assistance programs. OLMS received feedback when its program staff reached out to union officials. However, OLMS did not typically initiate new guidance and rather answered questions individually. If officials saw questions come into their OLMS-Public@dol.gov e-mail box on a related issue, OLMS program officials flagged the questions as a potential impetus for new guidance. 1) Union officials and union members, 2) employers with a union or with a workforce attempting to unionize, and 3) labor relations management organizations. Written guidance review policy? Review decisions documented? OLMS website, field staff, webinars, and listserv e-mails. Used web metrics to evaluate online guidance dissemination? Development and Review: Officials stated that guidance development depended on the impetus. OLMS officials stated that most often the director or other senior manager decided to initiate guidance and then tasked the drafting of the guidance to program staff. OLMS had no written procedures for the guidance production process. After the program staff drafted the guidance, it went to the Division Chief, then the Deputy Director, then the Director. OLMS divisions and the Office of the Solicitor (in particular the Division of Civil Rights and Labor Management) were also involved at this stage of the guidance process. Documentation of concurrence on draft guidance depended on the type of guidance. Some documents were typically approved through e-mail, while for other documents a physical folder with the sign-off chart was used for stakeholder initials of concurrence. OLMS officials stated that they drafted decision memorandums (typically used for regulation) to accompany draft guidance when departmental clearance was required. OLMS officials told us they rarely coordinated with other federal agencies when developing and reviewing guidance, although they recently coordinated with the Department of Labor’s Office of Federal Contract Compliance Programs (OFCCP) on development and review of guidance. Used web metrics to change online guidance dissemination? Office of Labor-Management Standards (OLMS) Initiation, Development, Review, and Dissemination, cont. hosted webinars. OLMS had a listserv to notify interested parties when there was a new regulation or guidance being issued. Feedback on Guidance and Dissemination OLMS officials told us they found webinars to be the most effective way to solicit feedback. Officials solicited questions from listeners at webinars and contacted those with questions directly. Officials also received questions and comments from the public through the OLMS- Public@dol.gov e-mail box. Questions typically related to how forms should be filled out, and officials contacted the person commenting directly. If the question was substantive, they would raise it to a higher level to address the comment. As of October 2014, OLMS officials told us they had not received substantive comments related to guidance dissemination. However, they had received feedback on technical issues related to locating materials, navigating the website, and ease of accessibility. OLMS officials said that they had not assessed whether their guidance has been effective. Office of Workers’ Compensation Programs (OWCP) Overview What OWCP Does OWCP protects the interests of workers who are injured or become ill on the job. OWCP serves specific employee groups that are covered under four major disability compensation statutes by mitigating the financial burden resulting from workplace injury or illness and promoting return to work when appropriate. Agency Use of Guidance 1. Training materials: OWCP posted training materials directed to internal claims examiners to the web. 2. Industry notices: Procedural information sent to employers, carriers, medical providers, and injured workers. 3. Compliance Guidance: Issued for Black Lung and Longshore programs to explain certain new regulations and how small entities could comply with them. 4. Educational Guidance: Included medical benefit guides, frequently asked questions, webinars, and other educational materials. Target Audiences The audience for OWCP guidance included beneficiaries and employers. Initiation, Development, Review, and Dissemination Initiation: OWCP officials told us that program directors decided to initiate guidance in response to (1) questions from users or confusion in the field, (2) new procedures, (3) new initiatives, (4) litigation, or (5) results of accountability reviews. Written guidance review policy? Yes No ✔ Review decisions documented? Yes ✔ No Dissemination methods Website, emails, specific task forces, and meetings with unions. Development and Review: OWCP officials told us they developed and issued guidance with the assistance of the Office of the Solicitor. The development of guidance was typically informal and was conducted through email, revisions, and comments in documents. OWCP officials told us they had an informal review process for guidance within each program. OWCP officials used a routing slip and hierarchical process for document review. A guidance document could be reviewed by the Department based on the significance and sensitivity of the issue. Used web metrics to evaluate online guidance dissemination? Yes No ✔ Used web metrics to change online guidance dissemination? Dissemination: OWCP officials told us that guidance documents were posted to the OWCP website and disseminated through email blasts to stakeholders. For example, there were 12,000 subscribers to their Longshore Electronic Filing email list. OWCP officials told us that they discussed who may want or need the information before placing the information on their website. OWCP had a joint task force which disseminated information regarding the Energy program primarily to the concentrated population affected in jurisdictions where there have been nuclear activities. OWCP also held routine meetings with unions, advocacy groups, and other stakeholder groups and maintained lists of these groups for dissemination purposes. Feedback on Guidance and Dissemination OWCP officials told us that program directors reviewed guidance on an ongoing basis and updated guidance as necessary. Veterans’ Employment and Training Service (VETS) Overview What VETS Does VETS is responsible for administering veterans’ employment and training programs and compliance activities that help veterans and service members succeed in their civilian careers. Target Audiences Program service providers, including agency grantees, sub- grantees, and other entities that provide services to veterans, transitioning service members, and other eligible persons. grant administration, including formula funding updates. 3. Solicitations for Competitive Grant Applications 4. Frequently Asked Questions: Clarified application and program details. Written guidance review policy? Guidance Initiation, Development, Review, and Dissemination Initiation: According to VETS officials, the primary impetus for guidance was to maximize the effectiveness of their programs by issuing updated and clarifying guidance that is easily understood and can be carried out effectively. Review decisions documented? Dissemination methods Website, e-mail, social media, and through regional administrators and state workforce grantees. Development and Review: VETS officials told us that because the component is small, it was easy to complete internal review of their documents. Officials used a routing slip to circulate draft guidance to the Offices of the Solicitor, Deputy Secretary, Assistant Secretary, and Secretary. Departmental officials reviewed guidance if it represented a major policy change or affected other Department of Labor agencies and other departments. Used web metrics to evaluate online guidance dissemination? Dissemination: VETS posted all publicly available documents on its websites. Officials told us they also relied on regional administrators and state workforce grantees to disseminate guidance related to grants. When VETS issued technical assistance on competitive grants, it sent out an e- mail blast to notify grantees. Used web metrics to change online guidance dissemination? Feedback on Guidance and Dissemination VETS officials told us they conducted periodic meetings and listening sessions with Veterans Service Organizations and stakeholders. VETS coordinated closely with the Departments of Defense and Veterans Affairs. The coordination requirements were included in a memorandum of understanding. VETS officials also participated in forums such as the Interagency Council on Homelessness and the Transition Assistance Program Executive Committee. VETS’ practice had been to issue new guidance annually. As of January 2015, it was now working to give guidance a longer “shelf life” by, for example, only revising due dates for routine reports rather than reissuing the entire guidance. Examples of Recent Guidance-Related GAO Reports Veterans’ Employment and Training: Better Targeting, Coordinating, and Reporting Needed to Enhance Program Effectiveness. GAO-13-29. Washington, D.C.: December 13, 2012. Wage and Hour Division (WHD) WHD enforces federal minimum wage, overtime pay, recordkeeping, and child labor law requirements of the Fair Labor Standards Act. WHD enforces a number of other worker protection laws including those concerning family and medical leave, migrant and seasonal farm workers, and several temporary foreign worker visa programs. WHD also administers and enforces the prevailing wage requirements applicable to federal contracts for construction and for the provision of goods and services. Employers and employees. Written guidance review policy? Yes No ✔ Review decisions documented? Yes ✔ No WHD website, webinars, trainings, and outreach. Initiation, Development, Review, and Dissemination Initiation: WHD officials told us they initiated new guidance in response to (1) issues identified by WHD’s investigators in their field offices, (2) questions from outreach and education sessions with employers and employee groups, (3) recurring questions received in correspondence, (4) feedback from stakeholders on specific programs, (5) information collected on the types and frequency of questions at WHD’s national call center, and (6) interaction with other federal and state agency partners. Many officials and offices within the Department of Labor (DOL) were involved in WHD’s decisions to initiate guidance, including the Office of the Solicitor, Office of the Assistant Secretary for Policy, national and regional WHD offices, and other enforcement personnel. Development and Review: WHD officials told us that they had a multilayered review process that differed based on the substance of the proposed guidance. Officials documented review with a routing slip. To clear guidance materials, WHD worked with the Office of the Solicitor and, depending on the substance or public interest, may have worked with other offices within the Department as appropriate. Some of the factors that may be considered in guidance being reviewed at the departmental level include: whether the guidance involved a new interpretation or policy or whether it could impact other DOL programs. Wage and Hour Division (WHD) Initiation, Development, Review, and Dissemination, cont. depended on the type of guidance. WHD staff routinely conducted webinars, training, and outreach during which guidance was distributed. For example, they have hosted webinars, conference calls, meetings and presentations, including engaging relevant state associations, consumers, disability and senior citizens’ advocates, worker representatives, and industry groups. Feedback on Guidance and Dissemination WHD officials told us that they revised guidance “as appropriate.” As of January 2015, WHD officials told us they did not have a systematic way to determine whether end users were accessing their guidance. WHD officials told us that they meet with stakeholders, including employers, human resources organizations, attorneys, employees, worker organizations, and unions to hear their views on areas WHD can provide additional guidance to achieve better compliance with the worker protection laws it administers. Examples of Recent Guidance-Related GAO Reports Fair Labor Standards Act: Department of Labor Needs a More Systematic Approach to Developing Its Guidance, GAO-14-629T. Washington, D.C.: July 23, 2014. Fair Labor Standards Act: Department of Labor Needs a More Systematic Approach to Developing Its Guidance, GAO-14-69. Washington, D.C.: December 18, 2013. Overview What Women’s Bureau Does The Women’s Bureau is responsible for promoting the status of wage-earning women, improving their working conditions, increasing their efficiency, and advancing their opportunities for profitable employment. The Women’s Bureau also focuses on the needs of vulnerable women in the workforce. Agency Use of Guidance 1. Informational Fact Sheets: Used to support the Women’s Bureau’s role in disseminating its own research. 2. Technical Assistance: Women’s Bureau officials told us 10 regional offices as well as staff in the national office provided technical assistance. Target Audiences Either targeted groups or broadly to more than 50,000 people. The target audiences for its informational fact sheets are typically working women, their employers, and organizations that represent them. 3. Frequently Asked Questions Initiation, Development, Review, and Dissemination Initiation: When deciding to initiate guidance, officials told us that the relevant program office typically decided whether a fact sheet was needed. It then either researched the subject itself or contracted out for the research. Leadership sometimes initiated guidance if it was on a “burning issue.” The ideas for potential Women’s Bureau initiatives and/or academic research came from all staff—both in the national and regional offices. These ideas stemmed from internal research, including current news and events. Written guidance review policy? Yes No ✔ Review decisions documented? Yes ✔ No Dissemination methods Website, e-mail, social media, workshops, and listening sessions. Development and Review: Women’s Bureau officials told us that after initiating guidance, the relevant program office drafted the guidance document. Managers and staff in the Women’s Bureau’s Office of Policy and Programs reviewed the draft guidance document before it was reviewed by the Deputy Directors and Director. Officials used a routing slip or memo to guide the draft through the review process and to document the review. Women’s Bureau officials told us they worked closely with the Office of the Solicitor, the Office of the Assistant Secretary for Policy, the grant officer, and other impacted Department of Labor components as they drafted and reviewed guidance. They also used the Departmental “Secretary’s Information Management System” program for internal tracking. Fact sheets were shared with external stakeholders when the guidance was issued. Dissemination: Women’s Bureau officials told us that most documents and other resources were available on their website and they used the “Gov Delivery” system to disseminate new information and documents. Officials also disseminated information through their Director’s blog and through workshops conducted by regional office staff, such as those that led up to the national White House Summit on Working Families. Listening sessions were another example of dissemination conducted by the regional offices. Officials told us they received feedback from end users though e-mails, phone calls, and comments made by participants at conferences. Michelle Sager at (202) 512-6806 or sagerm@gao.gov. In addition to the contact named above, key contributors to this report were Tim Bober, Assistant Director, Alexandra Edwards, Sandra Baxter, Shirley Hwang, Shelby Kain, Andrea Levine, Sarah Sheehan, and Wesley Sholtes. In addition, Jacques Arsenault, James Bennett, Marcia Crosse, Rachel Frisk, Ricky Harrison, Anne K. Johnson, Sarah Kaczmarek, Jacqueline M. Nowicki, Cynthia Saunders, Stewart W. Small, and Betty Ward-Zukerman provided key assistance.

Summary: Agencies rely on guidance to clarify regulatory text or statutes, to respond to the questions of affected parties in a timely way, and to inform the public about complex policy implementation topics. Unlike regulations, guidance is not legally binding. GAO was asked to examine guidance processes at four departments. This report reviews how (1) agencies use guidance and decide to issue guidance rather than regulations; (2) follow applicable criteria and leading practices in their policies, procedures, and practices for producing guidance; and (3) agencies disseminate guidance to ensure public access and feedback. GAO reviewed guidance processes at all 25 components in the four departments that (1) were within the requesting committee's jurisdiction, and (2) engaged in regulatory or grant activities. GAO reviewed relevant requirements, written procedures, guidance and websites, and interviewed agency officials. The four departments—Agriculture (USDA), Education (Education), Health and Human Services (HHS), and Labor (DOL)—and their selected components used guidance for multiple purposes, such as clarifying or interpreting regulations and providing grant administration information. The terminology used for agency guidance varied and agency components issued varying amounts of guidance, ranging from about 10 to over 100 guidance documents each year. The key criterion used when deciding whether to issue a regulation or guidance was whether it needed to be binding; in such cases agencies proceeded with regulation. Officials reported that they routinely consulted with legal counsel when making these choices. Departments typically identified few of their guidance documents as “significant,” generally defined by the Office of Management and Budget (OMB) as guidance with a broad and substantial impact on regulated entities. All four departments identified standard practices to follow when developing guidance. They addressed OMB's requirements for significant guidance to varying degrees and could strengthen internal controls for issuing guidance. Education and USDA had written departmental procedures for approval of significant guidance as required by OMB. DOL's procedures were not available to staff and required updating. HHS had no written procedures. Ensuring these procedures are available could better ensure that components consistently follow OMB's requirements. In the absence of specific government standards for non-significant guidance—the majority of issued guidance—the application of internal control standards is particularly important. The 25 components GAO reviewed addressed some control standards more regularly than others. For example, few components had written procedures to ensure consistent application of guidance processes. All components could describe standard review practices and most used tools to document management approval of draft guidance. Of the 25 components, 15 cited examples in which they conferred with external nonfederal stakeholders while developing guidance and nearly half did not regularly evaluate whether issued guidance remained current and effective. Components used different strategies to disseminate guidance and all relied primarily on posting the guidance on their websites. As such, components should follow applicable requirements for federal websites. One of these requirements—easy access to current and relevant guidance—could also facilitate opportunities for affected parties and stakeholders to provide feedback on those documents. USDA, DOL, and Education posted their significant guidance on a departmental website as directed by OMB; HHS did not. Components used several strategies—including organizing guidance by audience or topic and highlighting new or outdated guidance—to facilitate access. However, GAO identified factors that hindered online access, including long lists of guidance and documents dispersed among multiple web pages. All components GAO studied collected web metrics and many used them to evaluate online guidance dissemination. However, many of these components did not use metrics to improve how they disseminated guidance through their websites. Beyond their websites, components found other ways to disseminate and obtain feedback on issued guidance, including focus groups, surveys, and direct feedback from the public at conferences, webinars, and from monitoring visits. GAO is recommending that HHS and DOL ensure consistent application of OMB requirements for significant guidance. GAO also recommends that USDA, Education, HHS, and DOL strengthen the use of internal controls in guidance production processes and improve online guidance dissemination. USDA, Education, HHS and DOL generally agreed with the recommendations.