Electronic Rulemaking: Progress Made in Developing Centralized E-Rulemaking System
Published: 2005-09-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The E-Government Act of 2002 requires regulatory agencies, to the extent practicable, to ensure there is a Web site the public can use to comment on the numerous proposed regulations that affect them. To accomplish this, the Office of Management and Budget named the Environmental Protection Agency (EPA) as the managing partner for developing a governmentwide e-Rulemaking system that the public can use for these purposes. Issues GAO was asked to address include: (1) EPA's basis for selecting a centralized system, (2) how EPA collaborated with other agencies and agency views of that collaboration, and (3) whether EPA used key management practices when developing the system.
section_title: What GAO Found
paragraphs: E-Rulemaking officials and the e-Rulemaking Initiative Executive Committee considered three alternative designs and chose to implement a centralized e-Rulemaking system based on cost savings, risks, and security. Officials relied on an analysis of the three alternatives using two cost and risk assessment models and a comparison of the alternatives to industry best practices. Prior to completing this analysis, officials estimated the centralized approach would save about $94 million over 3 years. They said when they developed this estimate, there was a lack of published information about costs related to paper or electronic rulemaking systems. They used their professional judgment and information about costs for developing and operating EPA's paper and electronic systems, among other things, to develop the estimate. E-Rulemaking officials extensively collaborated with rulemaking agencies and most officials at the agencies we contacted thought the collaboration was effective. E-Rulemaking officials created a governance structure that included an executive committee, advisory board, and individual work groups that discussed how to develop the e-Rulemaking system. We contacted 14 of the 27 agencies serving on the advisory board and most felt their suggestions affected the system development process. Agency officials offered several examples to support their views, such as how their recommendations for changes to the system's design were incorporated. While managing the development of the centralized system, e-Rulemaking officials followed all but a few of the key practices for successfully managing an initiative. For example, officials did not have written agreements with participating agencies that included system performance measures. The first agencies began migrating to the centralized system in May 2005 with the public scheduled to have access in September 2005. Eventually, all rulemaking agencies will migrate to the centralized system; however, the schedule is tentative due in part to funding issues. As agencies migrate, e-Rulemaking officials are planning changes to the system including adding capabilities that exist in electronic systems operated by some agencies.

section_title: Letter
paragraphs: Each year, federal agencies publish thousands of regulations that affect  many aspects of citizens’ lives.  The regulatory process allows the public to  play a role in the promulgation of rules, primarily by requiring agencies to  publish proposed rules for public comment in accordance with the  Administrative Procedure Act of 1946, as amended.  However, in order to  be effectively involved, the public needs to know whether a proposed rule  is open for comment.  Modern information technology holds the potential  to make it easier than ever before to inform the public whether proposed  rules are open for comment.  Technology can also improve the federal  government’s ability to analyze and respond to those comments.
Since 2001, several legislative and executive branch initiatives have  encouraged public participation in rulemaking.  Notably, the E-Government  Act of 2002 included provisions designed to encourage electronic  rulemaking and the administration identified electronic rulemaking as one  of many electronic government initiatives.  The first module of the  e-Rulemaking initiative resulted in a governmentwide Web site at  www.regulations.gov, which enables the public to find and submit  comments on federal rules that are open for comment and published in the  Federal Register.  The goal of the second module of the e-Rulemaking  initiative is to establish a governmentwide electronic docket management  system into which all relevant regulatory supporting materials and public  comments will be placed.
While the Office of Management and Budget (OMB) oversees the e- Rulemaking initiative, it named the Environmental Protection Agency  (EPA) managing partner for the initiative.  As managing partner, EPA  heads the Program Management Office (PMO), which is staffed by officials  from several rulemaking agencies participating in this initiative.  The PMO  staff manages the day-to-day activities of the e-Rulemaking initiative while  the e-Rulemaking Initiative Executive Committee oversees the initiative’s  strategy, resources, and timetable.  Hereafter, we refer to PMO officials as  e-Rulemaking officials.
In response to your request that we continue to monitor the status of the  second module of the e-Rulemaking initiative, this report focuses on:  the approach EPA used to identify alternative designs for the  e-Rulemaking system, and how the decision to proceed with a single,  centralized e-Rulemaking system rather than another alternative design  was made;  how EPA collaborated with other rulemaking agencies to obtain input  about the e-Rulemaking system and those agencies’ views regarding the  collaboration; and  whether key practices for successfully managing a project have been  utilized by EPA and EPA’s future plans for developing and implementing  the centralized e-Rulemaking system.
To address our objectives, we reviewed relevant federal legislation; capital  asset and business cases developed for the e-Rulemaking initiative;  contractor assessments regarding the capabilities of existing electronic  rulemaking systems and the three alternative designs considered for the  e-Rulemaking system; and documents related to the roles and  responsibilities of OMB, e-Rulemaking officials, the Executive Committee,  and federal rulemaking agencies for the design and implementation of the  e-Rulemaking initiative.  We also interviewed OMB, e-Rulemaking officials,  and officials from several rulemaking agencies involved in the  e-Rulemaking initiative.  To identify key practices for successfully  managing a project we reviewed previous GAO work related to the  e-government initiatives, which was based in part upon reviews of  government, private sector, and academic research.  We compared  information provided by e-Rulemaking officials to these key practices to  determine what practices they appeared to have engaged in as part of  managing the e-Rulemaking initiative.  Our analysis was limited to a  determination concerning the presence of the practice.  We did not assess  how well or extensively the practice was followed.  We performed our  work from December 2003 through June 2005 in accordance with generally  accepted government auditing standards.  Our scope and methodology are  described in more detail in appendix I.
subsections: section_title: Background
paragraphs: For many years, federal agencies have been encouraged to use information  technology to improve their communications with the public and to  increase participation in the rulemaking process.  One of the  recommendations of the National Performance Review in September 1993  was to “se information technology and other techniques to increase  opportunities for early, frequent, and interactive public participation during  the rulemaking process ...”  The Paperwork Reduction Act of 1995 states  that the Director of OMB should promote the use of IT “to improve the productivity, efficiency, and effectiveness of Federal programs…”   Similarly, a December 17, 1999, presidential memorandum on “Electronic  Government” noted that “as public awareness and Internet usage increase,  the demand for online Government interaction and simplified, standardized  ways to access Government information and services becomes increasingly  important.”
Between June 2000 and February 2001, as information technology began to  play an increasingly important role in agencies’ regulatory management  and the facilitation of public participation in federal rulemaking, we  reported on a number of associated opportunities and challenges.  We  pointed out that the use of information technology could reduce regulatory  burden, improve the transparency of regulatory processes, and, ultimately,  facilitate the accomplishment of regulatory objectives.  We reported on  innovative uses of information technology that representatives of federal or  state agencies and nongovernmental organizations believed could be more  widely used by federal regulatory agencies, and we recommended actions  that could facilitate innovation, avoid duplication of effort, and potentially  result in a broader and more consistent approach across federal agencies.
The President, through the President’s Management Agenda, has advanced  greater use of information technology across a range of government  functions, including rulemaking.  In July 2001, the President identified the  expansion of e-government as one of the five priorities of his management  agenda.  To support this, OMB has developed an implementation strategy  and identified 25 e-government initiatives, one of which is e-Rulemaking.     In May 2002, the Director of OMB issued a memorandum to the heads of  executive departments and agencies advising them of “our intention to  consolidate redundant IT systems relating to the President’s on-line  rulemaking initiative.”  Citing OMB’s authority under the Clinger-Cohen  Act of 1996, the Director said OMB had identified “several potentially  redundant systems across the federal government that relate to the  rulemaking process,” and indicated that consolidation of those systems  could save millions of dollars.  The administration said e-Rulemaking  would “democratize an often closed process.”
In late 2002, the President signed into law the E-Government Act of 2002.   The act requires agencies, to the extent practicable, to accept public  comments on proposed rules “by electronic means” and to ensure that a  publicly accessible federal Web site contains “electronic dockets” for their  proposed rules.  The act also established an Office of Electronic  Government within OMB, headed by an Administrator appointed by the  President and required to work with the Administrator of OMB’s Office of  Information and Regulatory Affairs in establishing the strategic direction of  the e-government program and to oversee its implementation.
Initially, OMB named the Department of Transportation (DOT) as the lead  agency, or managing partner, for the e-Rulemaking effort, entrusting DOT  with the day-to-day management of the project as well as collaboration  with other e-Rulemaking agencies.  In a previous report, we noted that  “DOT had the most developed electronic docket system of the agencies  that we contacted, covering every rulemaking action in the department and  including all public comments received regardless of medium.”  DOT  served as managing partner until an independent consultant’s assessment  concluded in August 2002 that the system operated by EPA was the optimal  platform to serve as the consolidated electronic docket platform for the  entire federal government.  As a result of this assessment, OMB transferred  managing partner responsibilities from DOT to EPA in late 2002.
The e-Rulemaking initiative provides a single portal for businesses and  citizens to access the federal rulemaking process and comment on  proposed rules. It is composed of three modules.  The first module was  completed in January 2003 with the launch of the www.regulations.gov  Web site.   The second module, which is the focus of this report, will move  beyond rule identification and a comment mechanism by establishing a  governmentwide electronic docket management system into which all  relevant regulatory supporting materials and public comments will be  placed.  The third and final module will create an electronic regulatory  desktop to facilitate the rule development process.
section_title: Officials Considered Three E-Rulemaking Designs and Chose a Single, Centralized Design Based on Cost, Deployment Risks, and Security
paragraphs: E-Rulemaking officials and the e-Rulemaking Initiative Executive  Committee considered three alternative designs for the federal e- Rulemaking system and ultimately chose to develop and implement a  single, centralized design.  This decision was based upon studies that  examined the costs, deployment risks, and security aspects of the three  designs.  Throughout the management of the e-Rulemaking initiative, e- Rulemaking officials have estimated that the centralized design would save  money; the latest estimate is that it would save approximately $94 million  over 3 years.  Officials said they used their best professional judgment and  information about the costs to develop and operate paper and electronic  rulemaking systems.
subsections: section_title: Centralized, Tiered, and Distributed Designs Considered
paragraphs: E-Rulemaking officials and the e-Rulemaking Initiative Executive  Committee considered three designs for the system architecture of the  governmentwide e-Rulemaking system.  These were a centralized, tiered,  or a distributed design.   The key characteristics of the governmentwide e- Rulemaking system architecture would:  enable agencies to manage content and workflow processes using  variable access controls and role definitions;   provide a robust and scaleable Web-based solution that supports the  capture, conversion, and dissemination of high volumes of information;   provide the public and stakeholders with the ability to perform  electronic docket searching, viewing, and commenting across multiple  agencies; and  minimize the total cost of system ownership and management while  delivering responsive service to agencies/subagencies, businesses, and  citizens.
Anyone with the ability to connect to the Internet with a standard industry  browser would be able to access the e-Rulemaking system.
Centralized design.  The centralized design uses a minimal number of  governmentwide e-Rulemaking system components in the same location to  provide consistent access and services for citizens and agency staff across  all dockets.  Under this design, the features and functions of the  governmentwide e-Rulemaking system are provided by component-based  application architecture and all standard components are centrally located.   These application components would reside on servers in a single location  for delivery of services to the public and the agencies.  Unlike the tiered  and distributed designs, which are discussed in the next sections, the  centralized design will not support existing agency docket systems nor will  it distribute components of the governmentwide e-Rulemaking system in  agency facilities.
Tiered design. The tiered design utilizes a centralized governmentwide e- Rulemaking system to deliver all agency and citizen services, but is  different from the centralized design because common hardware and  software components installed in the governmentwide e-Rulemaking  system are also installed at different agency sites to enhance system  performance.  For example, components like the database management or  document management system may be placed on separate servers in the  same or different locations to process information more efficiently to boost  system performance, and the responsibility for the data is dispersed across  multiple entities or agency sites and would be maintained locally at each  site.  Because the tiered design is based on the use of common hardware  and software components, the governmentwide e-Rulemaking system is  not linked to any existing agency specific docket system with its unique  hardware and software. As a result, no customized software interfaces are  needed.
Distributed design. The distributed design integrates a centralized  governmentwide e-Rulemaking system with existing agency-specific  electronic docket systems while satisfying all governmentwide  e-Rulemaking system requirements.  This design links agencies with  existing docket systems to the governmentwide e-Rulemaking system using  customized software (middleware) to allow interconnectivity between  agencies’ systems and the governmentwide e-Rulemaking system.  This  design provides the public with a means of searching across agency  dockets and establishes the governmentwide e-Rulemaking system as an  access point to the public and agencies for searching, reviewing, and  commenting on dockets that reside on the agencies’ existing docket  systems.  In addition, agencies with docket systems will continue to  perform their own workflow/business/docket life-cycle processing.
section_title: Officials Used Contractor’s Analysis When Preparing Business Case Recommending Centralized Design
paragraphs: In 2002, DOT, which was the managing partner for the e-Rulemaking  initiative, contracted with a consulting firm to assess the capabilities of  seven existing agency e-Rulemaking systems and prepare a business case  describing alternative designs for the system and the recommended design.   The firm was to provide advice to DOT about the best technical approach  for the initiative, along with a full analysis of alternatives that leverage the  use of existing technology to meet the vision, goals, and objectives of the  initiative.  Based on its assessment, the consulting firm’s August 2002  report recommended that EPA’s eDocket system would be the optimal  platform for a governmentwide centralized e-Rulemaking system.  The  business case discussed two other alternative designs as well.  DOT  submitted this business case to OMB in September 2002.
Based on the recommendation to use EPA’s e-Rulemaking system as the  platform for a centralized system, OMB transferred managing partner  responsibilities for the e-Rulemaking initiative from DOT to EPA.  After  EPA became the managing partner in late 2002, EPA submitted three  additional business cases to OMB—in December 2002, September 2003,  and September 2004—and recommended the centralized design.  During  this time period, e-Rulemaking officials also hired a contractor to refine the  three alternative designs, summarize the costs and risks for each  alternative, and recommend one alternative for implementation.  E- Rulemaking officials used the contractor’s January 2004 report  recommending the centralized alternative when preparing the last of the  three business cases that was submitted to OMB.
section_title: Three Assessments Favored Centralized Design
paragraphs: E-Rulemaking officials and the e-Rulemaking Initiative Executive  Committee based the decision to select the centralized design over the  tiered and distributed designs on three assessments that generally found  that the centralized design was most cost effective, had the lowest risk for  deployment and support instability, was the most secure, and was most  likely to deliver the breadth and functionality sought by agencies and the  public.  E-Rulemaking officials hired a contractor to conduct two analyses  based on cost and risk models, and the contractor obtained from a  consulting firm a third assessment that was based on industry best  practices and experiences in assessing similar architectures.
The first assessment used a cost-analysis modeling tool used by the private  sector and government agencies to analyze and estimate the costs  associated with software application development.  Using this model, the  cost estimates to deliver and operate the three designs were $18.7 million  for the centralized design, $21.1 million for the tiered design, and $87.2  million for the distributed design (assuming 10 agencies were integrated  with the centralized governmentwide e-Rulemaking system).  The model  predicted it would take 1 year to deliver the centralized design and almost 3  years to deliver the distributed design.  Also, according to the model, the  complexity of the distributed design would present a high risk of instability  to the overall operation and maintenance of the systems.
The second assessment used a model that estimates the total cost of  implementing and supporting a specific application within a commercial  enterprise and assesses the expected risk of successful deployment and the  benefit to the organization of adopting the new capability.  Using this  model, the cost to deliver and operate the three designs was $20.1 million  for the centralized design, $22.8 million for the tiered design, and $94.9  million for the distributed design (assuming 10 agencies were integrated  with the centralized governmentwide e-Rulemaking system).  The model  estimated that there was a 50 to 60 percent greater likelihood of  unsuccessful deployment of the distributed design when compared to the  other designs.  The most significant risk factor accounting for this  increased likelihood of failure is system complexity, resulting in major  integration, infrastructure, and change management issues that tend to be  difficult to resolve.
The third assessment was done by a leading information technology firm  with the intent of predicting the cost, risk, security, and supportability of  the three designs based on industry best practices and the firm’s  experience in assessing similar architectures.  The firm recommended the  centralized design because it provided for a consistent implementation  across the government and was the lowest cost and least risky design.  The  firm rated the tiered design as “feasible but not recommended” for various  reasons including the following.  First, the firm said it would be difficult to  maintain consistent data quality in a tiered design because the  responsibility for data would be spread across multiple entities.  Second,  consistent security and business continuity policies could not be  implemented because each agency would most likely manage its data  according to its security policies and business continuity models.  Third,  data retrieval could be complex because data would be maintained locally  at each agency. The firm did not recommend the distributed design for  several reasons, including the highly complex implementation and high  delivery risk and higher cost for maintenance and support due to the large  number of disparate systems involved in the design.
section_title: Officials Used Professional Judgment to Develop Estimated Savings
paragraphs: E-Rulemaking officials estimated that the federal government would save  approximately $94 million over 3 years by deploying the e-Rulemaking  system—$56 million in savings from eliminating duplication of systems and  $38 million in savings from annual maintenance fees.  Officials said that  they developed this estimate prior to completing the three assessments  previously described and that at the time they developed this estimate,  there was a lack of published information about how much it cost to  develop or operate paper or electronic rulemaking systems.  They primarily  used their professional judgment, information about costs for developing  and operating EPA’s paper and electronic rulemaking systems, and  information provided by OMB about the number of rulemaking entities to  develop the estimate.  The estimate assumes, among other things, that all  rulemaking entities would either develop an electronic rulemaking system  or, if they already had an electronic rulemaking system, they would  continue to operate those systems.
section_title: Agencies Generally Expressed Satisfaction with E-Rulemaking Officials’ Efforts to Solicit Their Input
paragraphs: E-Rulemaking officials extensively collaborated with rulemaking agencies  and used several methods to solicit the participation of those agencies  during the e-Rulemaking effort, notably the use of interagency working  groups.  Most officials at these other rulemaking agencies said they had  adequate opportunity to collaborate on the initiative and described the  process as effective.  They provided us with examples that supported their  opinions.
subsections: section_title: Officials’ Collaborative Efforts Included Three Levels of Panels with Broad Representation
paragraphs: E-Rulemaking officials have successfully used a variety of collaboration  tools to encourage participation and arrive at consensus among the  participating agencies, particularly the use of interagency working groups.   Two core working groups facilitate discussion and decision making for the  project: (1) the e-Rulemaking Advisory Board, composed primarily of  technical and policy staff from the participating agencies; and (2) the e- Rulemaking Initiative Executive Committee, composed of upper-level  agency managers such as Chief Information Officers and individuals at the  Assistant Secretary level, which was added after the project had begun to  obtain upper management support from participating agencies—especially  those that already have an electronic system in place.  Additionally,  e-Rulemaking officials have convened specialized working groups to tackle  unique areas of concern, such as system design, project funding, or legal  issues.  These specialized working groups are open forums designed to  foster consensus on decisions made on the e-Rulemaking initiative.
Under e-Rulemaking officials’ guidance, specialized working groups are  charged with developing specific proposals for the project and transmitting  these proposals to the Advisory Board for discussion.  The Advisory Board  may consider revisions to the proposal before reaching a decision that  represents a consensus of the participants.  This decision is then sent to the  Executive Committee, which makes a final decision on the proposals and  recommends a course of action to OMB.  Agency officials we spoke with  noted that representatives of all agencies are free to participate on the  Executive Committee, the Advisory Board, or any working group.
The three levels of panels are supplemented by a variety of other  collaborative tools and methods, including: one-on-one meetings with  agency officials, surveys, e-mail communication, teleconferencing, and an  online library of documents related to the initiative.  EPA also provided  public notice of its work in the Federal Register and held public forums on  the East and West coasts to obtain the views of businesses, citizens, and  interest groups regarding the design of the e-Rulemaking system.
section_title: Officials Praised for Their Collaboration, Responsiveness to Unique Agency Needs, and Explanations for Suggestions That Were Rejected
paragraphs: The tenor of our discussions with officials of 14 of the 27 agencies serving  on the Advisory Board was that they were satisfied with the level of  collaboration.  Participating agencies indicated that they had adequate  opportunity to provide input and described the collaboration of e- Rulemaking officials as effective.  Officials from a few agencies even said  that in terms of the e-government initiatives, the e-Rulemaking initiative  was one of the better collaborative efforts in which they have participated.
Agencies often praised e-Rulemaking officials’ concern for the unique  needs of their agency.  For example, one agency communicated its concern  that it was not clear how the e-Rulemaking system would accommodate  interim rules.  While most rules are not effective until after the rulemaking  process is concluded, some interim rules are effective immediately, for  example, due to emergency conditions or other concerns related to health  and safety.  After receiving this comment, e-Rulemaking officials  recognized the need to be clear about how interim rules would be  processed and how comments on such rules would be addressed.  The  system was revised in accordance with the agency’s suggestions.
In another instance, one agency disagreed with the implementation  schedule it was given for its transition to the e-Rulemaking system.  The  agency’s officials anticipated that the transition date would overlap with a  critical stage in their annual rulemaking cycle and expressed concern that  any errors in processing and adopting rules during that time would be  particularly harmful.  E-Rulemaking officials readily agreed to change the  schedule to account for this timing problem.
Even when an agency’s suggestion was not incorporated into the system  design, they acknowledged that e-Rulemaking officials treated their  concerns fairly, completely, and they understood why the suggestion was  rejected.  Agencies frequently deferred to the needs of the group after  expressing their individual preferences during the collaborative process.
The opinions expressed by these officials are consistent with our prior  work.  We noted in a recent report that e-Rulemaking initiative officials  have successfully used collaboration strategies to achieve consensus with  partner agencies on funding contributions to the e-Rulemaking initiative  based on agency-specific characteristics.
section_title: Work Conducted Mostly Consistent with Key Practices but Written Agreements Did Not Include Performance Measures
paragraphs: While managing the development of the centralized e-Rulemaking system,  e-Rulemaking officials have, for the most part, followed key practices for  successfully managing a project.  However, there are a few practices that  officials did not incorporate into the management of the e-Rulemaking  initiative, such as including system performance measures in written  agreements with agencies.  The first agencies began migrating to the  centralized e-Rulemaking system in May 2005 and the public is scheduled  to have access to the system in September 2005.  While all agencies will  eventually migrate to the centralized system, the schedule for doing so may  change, due in part to funding issues.  E-Rulemaking officials have also  created a process for approving changes to the system based on concerns  or issues identified by agencies or the public.
subsections: section_title: Officials Mostly Followed Key Practices during System Development
paragraphs: E-Rulemaking officials, for the most part, followed key practices for  successfully managing a project while developing the centralized e- Rulemaking system; however, there are a few practices that they did not  follow.  Table 1 summarizes the key practices they followed and those that  they did not follow when developing the centralized e-Rulemaking system.   This table does not indicate how well they followed each practice, but  rather, indicates if the steps they took to manage the e-Rulemaking  initiative reflected the presence of the practice.
E-Rulemaking officials followed all of the key practices except two.  They  did not document completely decisions related to the approach they used  to identify alternative designs.  Also, EPA, as managing partner, does not  have written agreements that address system performance measures with  agencies participating in the e-Rulemaking initiative.  Although the written  agreements do not include performance measures, e-Rulemaking officials  do have performance goals that they are measuring.  For example, one goal  is to have the centralized e-Rulemaking system available to the public and  agency rule writers and managers 99.99 percent of the time.  Officials also  said they are developing a postimplementation review plan.
EPA, as managing partner, signed written agreements with 15 agencies  during fiscal years 2003 or 2004 that indicated EPA would establish  performance measures, but the agreements did not include any details  about them.  These agreements did include expected outcomes, roles and  responsibilities, and resource commitments.  These 15 agencies—which  committed to providing financial assistance or in-kind resources to the  initiative—included the 5 agencies, or parts of agencies, that are migrating  to the centralized e-Rulemaking system during the first migration phase.   During fiscal year 2005, EPA plans to sign similar agreements with 20  additional agencies that include these three items.  The additional agencies  are those to whom OMB has issued budget guidance for 2005 regarding  providing funding to EPA for the e-Rulemaking initiative.  As additional  agencies become involved in funding the initiative, e-Rulemaking officials  said they plan to sign similar agreements with them.
section_title: Officials Have Process for Approving Changes to the System Based on Agency and Public Input as More Agencies Migrate to It
paragraphs: In May 2005, the agencies included in phase I of the migration schedule— EPA, the Department of Housing and Urban Development, the Animal and  Plant Health Inspection Service within the Department of Agriculture, a  portion of the Department of Homeland Security, and the National Archives  and Records Administration—began migrating to the centralized e- Rulemaking system.  Current plans are for 18 additional departments or  agencies to begin using the system in fiscal year 2006, resources permitting.   The remaining rulemaking departments or agencies are scheduled to begin  using the system at different times during fiscal year 2007 and beyond.   According to e-Rulemaking officials, the schedule may change depending  on funding.   They originally planned to migrate 10 agencies to the new  system during phase I, but had to cut back due to funding shortfalls and  late contributions, specifically in fiscal year 2004.
When an agency is scheduled to migrate to the new system, it will go  through a multistep process tailored to meet its needs.  E-Rulemaking  officials will sign an agreement that outlines the steps the agency will go  through to migrate to the centralized system.  Upon request, prior to  initiating the implementation process, e-Rulemaking officials will meet  with a department or agency to brief them on the initiative and  demonstrate the system.   The implementation steps, in chronological  order, are: 1. 2. site survey (i.e., identify and analyze agency-specific technical and  functional needs); 3. data preparation (i.e., develop agency-specific implementation); 4. agency configuration (i.e., develop technical and functional engineering  requirements to support agency-unique implementation); 5. training (i.e., develop plans and conduct training); 6. usability testing (i.e., allow agency users to become familiar with the e- 7. data migration (i.e., migrate data to the system); 8. moving to production (i.e., agency goes “live” using the system); and 9. postproduction support.
Five of the nine steps are high-level checkpoints at which the agency and e- Rulemaking officials agree that conditions are acceptable to move forward.   These steps are the kick-off meeting, data preparation, training, usability  testing, and moving to production.
The centralized e-Rulemaking system will not remain static as additional  agencies migrate to it, according to e-Rulemaking officials.  They said that  changes to the system will be made when valid concerns or issues are  identified.  Such concerns or issues could be raised by agencies that have  already migrated to the system, agencies that will be migrating to the  system, or public users of the system.  E-Rulemaking officials are already  planning changes to the system before phase II of the migration schedule  begins.  These changes are based on issues identified during beta testing by  the agencies and usability testing by the public.  According to e-Rulemaking  officials, they have created a Change Control Board that will review and  determine which requests for changes to the system will be granted.  This  board will also prioritize the changes that are to be made and obtain an  estimate of the cost to implement them from the contractor assisting in the  design and implementation of the system.
A couple of agencies said they had concerns about whether the centralized  system would include all the capabilities of their current electronic  systems.  For example, one agency said that their system tracks hearings as  well as regulatory dockets.  E-Rulemaking officials told these agencies that  any such capabilities will be incorporated into future versions of the  centralized e-Rulemaking system, provided adequate funding is provided.   Moreover, they noted that because each agency signs an agreement with  the e-Rulemaking Program Office before migrating to the centralized  system, the agency can assure itself that the system has the capabilities it  needs before signing the agreement and moving forward with migration.
section_title: Conclusions
paragraphs: The process e-Rulemaking officials and the e-Rulemaking Initiative  Executive Committee used to decide which of the three designs— centralized, tiered, or distributed—to develop and implement as well as the  basis for that decision, was reasonable and adequately supported.  Using  two cost and risk models and comparing the three designs to industry best  practices was a sound approach to use in order to select which design  should be implemented.  In addition, because there was a lack of published  information about the costs to develop or operate either a paper or  electronic rulemaking system, e-Rulemaking officials used their  professional judgment and own experiences to estimate how much the  centralized e-Rulemaking system could save the federal government.
E-Rulemaking officials established a governance structure to collaborate  with other agencies to obtain input on developing and implementing the  centralized e-Rulemaking system and its collaboration efforts with other  agencies have been extensive and well-received.  Most of the agencies we  contacted were very satisfied with the collaboration efforts and thought  that e-Rulemaking officials listened to their ideas and used them when  developing and implementing the system.  Officials from a few agencies we  interviewed even said that in terms of the e-government initiatives, the e- Rulemaking initiative was one of the better collaborative efforts in which  they have participated.
While managing the development of the centralized e-Rulemaking system,  e-Rulemaking officials followed most of the key practices for successfully  managing a project.  E-Rulemaking officials could, however, improve their  management of the e-Rulemaking initiative by including system  performance measures in written agreements with agencies.  Having such  agreements would provide criteria for determining if the e-Rulemaking  initiative is operating in the most effective, efficient, and economic manner  possible.
section_title: Recommendation
paragraphs: E-Rulemaking officials established a governance structure that allowed it  to successfully collaborate with other agencies about how to develop and  implement the centralized e-Rulemaking system and used most of the key  practices included in this report for managing an initiative.  To learn from  how it managed this initiative and to build on the success of it, we  recommend that the Administrator of EPA, as managing partner of this  initiative, take steps to ensure that the written agreements between EPA  and the participating agencies include performance measures that address  issues such as system performance, maintenance, and cost savings. These  measures are necessary to provide criteria for evaluating the effectiveness  of the e-Rulemaking initiative as well as for determining if the initiative is  operating in the most efficient and economical manner.
section_title: Agency Comments
paragraphs: The Administrator of EPA was provided a draft of this report for his review  and comment.  The EPA Assistant Administrator and Chief Information  Officer provided written comments on the draft in an August 17, 2005,  letter, which is reprinted in appendix III.  The Assistant Administrator  agreed that the report accurately describes the e-Rulemaking Initiative.    She added that e-Rulemaking officials appreciated GAO’s recommendation  to ensure the effectiveness of the initiative and they look forward to  continuing to work with GAO for the continued success of the project.  E- Rulemaking officials said they agree with GAO’s recommendation and they  plan to implement it.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we will not distribute it until 30 days from the date of  this letter.  At that time we will send copies of this report to the Chair and  Ranking Minority Member of the House Committee on Government  Reform, the Administrator of EPA, and the Director of OMB.  We will make  copies available to others on request.  In addition, the report will be  available at no charge on the GAO Web site at http://www.gao.gov.
If you have any questions regarding this report, please contact me at (202)  512-5837 or at williamso@gao.gov. Contact points for our Office of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report were Thomas Beall, Boris  Kachura, Steven Law, Joseph Santiago, Shellee Soliday, and Grant Turner.
section_title: Objectives, Scope, and Methodology
paragraphs: While the Office of Management and Budget (OMB) oversees the e- Rulemaking initiative, it named the Environmental Protection Agency  (EPA) managing partner for the initiative.  As managing partner, EPA heads  the Program Management Office (PMO), which is staffed by officials from  several rulemaking agencies participating in this initiative.  The PMO staff  manages the day-to-day activities of the e-Rulemaking initiative while the e- Rulemaking Initiative Executive Committee makes the final decisions  about the initiative’s strategy, resources, and timetable.  Hereafter, we refer  to PMO officials as e-Rulemaking officials.
Our first objective was to describe the approach EPA used to identify  alternative designs for the e-Rulemaking system and how the decision to  proceed with a single, centralized system design was made.  To address this  objective, we reviewed the Capital Asset Plan and Business Cases  submitted by the Environmental Protection Agency (EPA) and the  Department of Transportation (DOT).  We also reviewed a contractor’s  analysis of the capabilities of current e-Rulemaking systems, a contractor’s  summary of assessments of the three identified alternative designs  including analyses of operational risks and security vulnerabilities, and  documentation related to the basis for e-Rulemaking officials’ and the e- Rulemaking Initiative Executive Committee’s recommendation about  which design alternative to implement.  We also interviewed OMB, e- Rulemaking, and DOT officials involved with developing information about  the design alternatives, the selection of the centralized system design, and  the development of the estimated cost savings associated with the  centralized design alternative.
Our second objective was to describe how EPA collaborated with other  rulemaking agencies to obtain input about the e-Rulemaking system and  those agencies’ views regarding the collaboration.  To address this  objective, we interviewed e-Rulemaking officials and reviewed  documentation related to collaboration efforts such as those describing the  purpose and composition of various committees and work groups and  written agreements between agencies and EPA regarding funding of the  system.  To acquire agency views on the collaboration that has occurred,  we randomly selected 12 of the 27 agencies that serve on the e-Rulemaking  Initiative Advisory Board and interviewed officials from those agencies  about their experiences with the collaboration efforts.  Included in this set  of selected agencies were (1) agencies that serve on the e-Rulemaking  Executive Committee, (2) agencies that currently have an e-Rulemaking  system, and (3) agencies that are heavily involved in rulemaking.  We  analyzed the agency officials’ responses to our questions on collaboration  and identified those views that were most frequently expressed by officials  from multiple agencies as a means of gauging the overall quality of the  collaboration.
In addition to the agencies included in the sample, we discussed  collaboration with two additional agencies that serve on the Advisory  Board.  The first was DOT.  Since we contacted DOT officials in relation to  the first objective, we also discussed collaboration issues with them.  We  contacted the second agency, the National Archives and Records  Administration (NARA), because many officials included in the original  sample suggested that we talk to that agency since an official from NARA  was heavily involved in meeting with all agencies to discuss the  development of the e-Rulemaking system.  We also discussed the methods  used to obtain information from the public about the development of the e- Rulemaking system with PMO officials and reviewed related documents.
Our third objective was to (1) determine whether EPA used key practices  for successfully managing a project when managing the e-Rulemaking  initiative and (2) describe EPA’s future plans for developing and  implementing the centralized e-Rulemaking system.   To determine whether  e-Rulemaking officials followed key practices, we identified key practices  for successfully managing a project by reviewing previous GAO work  related to the e-government initiatives.  The key practices included in prior  GAO reports were based in part on studies done by federal, state, and local  agencies, international agencies, and the private sector and on guidance  provided by the federal government, state, local and international  governments, private research groups, national associations, and  educational institutions.  GAO information technology staff agreed with the  list of key practices we identified.  After developing the list of key  practices, we compared the list to the information we had gathered about  e-Rulemaking officials’ management of the e-Rulemaking initiative.  When  making the comparison, we did not assess how well each key practice was  followed, but rather we determined if the steps they took to manage the e- Rulemaking initiative reflected the presence of the practice.  We did not  attempt to assess the quality or extent of each practice’s implementation.   In order to describe future plans for the initiative, we reviewed documents  that addressed future plans such as the business cases and implementation  schedules and interviewed e-Rulemaking officials.  We did not assess the  quality of these plans.
Although we report cost and related data from contractor assessments of  the three governmentwide e-Rulemaking system design alternatives, we did  not examine the reliability of that data since our first objective was to  describe how e-Rulemaking officials selected one of the three alternatives  rather than to determine if the appropriate alternative was selected.   We  performed our work from December 2003 through June 2005 in accordance  with generally accepted government auditing standards.
section_title: List of GAO Reports Related to E-Rulemaking
paragraphs: Federal Rulemaking: Agencies’ Use of Information Technology to  Facilitate Public Participation. GAO/GGD-00-135R. Washington, D.C.:  June 30, 2000.
Electronic Government: Government Paperwork Elimination Act  Presents Challenges for Agencies. GAO/AIMD-00-282. Washington, D.C.:  September 15, 2000.
Electronic Government: Better Information Needed on Agencies’  Implementation of the Government Paperwork Elimination Act. GAO-01- 1100. Washington, D.C.: September 28, 2001.
Regulatory Management: Communication About Technology-Based  Innovations Can Be Improved. GAO-01-232. Washington, D.C.: February  12, 2001.
Information Technology: OMB Leadership Critical to Making Needed  Enterprise Architecture and E-government Progress. GAO-02-389T.  Washington, D.C.: March 21, 2002.
Electronic Government: Selection and Implementation of the Office of  Management and Budget’s 24 Initiatives. GAO-03-229. Washington, D.C.:  November 22, 2002.
Electronic Government: Success of the Office of Management and  Budget’s 25 Initiatives Depends on Effective Management and Oversight.  GAO-03-495T. March 13, 2003.
Electronic Rulemaking: Efforts to Facilitate Public Participation Can Be  Improved. GAO-03-901. Washington, D.C.: September 17, 2003.
Electronic Government: Potential Exists for Enhancing Collaboration on  Four Initiatives. GAO-04-6. Washington, D.C.: October 10, 2003.
Electronic Government: Initiatives Sponsored by the Office of  Management and Budget Have Made Mixed Progress. GAO-04-561T.  Washington, D.C.: March 24, 2004.
Electronic Government: Federal Agencies Have Made Progress  Implementing the E-Government Act of 2002. GAO-05-12. Washington,  D.C.: December 10, 2004.
Electronic Government: Funding of the Office of Management and  Budget’s Initiatives. GAO-05-420. Washington, D.C.: April 25, 2005.