Electronic Rulemaking: Progress Made in Developing Centralized E-Rulemaking System
Released: 2005-10-11T00:00:00

Letter

Background

For many years, federal agencies have been encouraged to use information technology to improve their communications with the public and to increase participation in the rulemaking process. One of the recommendations of the National Performance Review in September 1993 was to “se information technology and other techniques to increase opportunities for early, frequent, and interactive public participation during the rulemaking process ...” The Paperwork Reduction Act of 1995 states that the Director of OMB should promote the use of IT “to improve the productivity, efficiency, and effectiveness of Federal programs…” Similarly, a December 17, 1999, presidential memorandum on “Electronic Government” noted that “as public awareness and Internet usage increase, the demand for online Government interaction and simplified, standardized ways to access Government information and services becomes increasingly important.”

Between June 2000 and February 2001, as information technology began to play an increasingly important role in agencies’ regulatory management and the facilitation of public participation in federal rulemaking, we reported on a number of associated opportunities and challenges. We pointed out that the use of information technology could reduce regulatory burden, improve the transparency of regulatory processes, and, ultimately, facilitate the accomplishment of regulatory objectives. We reported on innovative uses of information technology that representatives of federal or state agencies and nongovernmental organizations believed could be more widely used by federal regulatory agencies, and we recommended actions that could facilitate innovation, avoid duplication of effort, and potentially result in a broader and more consistent approach across federal agencies.

The President, through the President’s Management Agenda, has advanced greater use of information technology across a range of government functions, including rulemaking. In July 2001, the President identified the expansion of e-government as one of the five priorities of his management agenda. To support this, OMB has developed an implementation strategy and identified 25 e-government initiatives, one of which is e-Rulemaking. In May 2002, the Director of OMB issued a memorandum to the heads of executive departments and agencies advising them of “our intention to consolidate redundant IT systems relating to the President’s on-line rulemaking initiative.” Citing OMB’s authority under the Clinger-Cohen Act of 1996, the Director said OMB had identified “several potentially redundant systems across the federal government that relate to the rulemaking process,” and indicated that consolidation of those systems could save millions of dollars. The administration said e-Rulemaking would “democratize an often closed process.”

In late 2002, the President signed into law the E-Government Act of 2002. The act requires agencies, to the extent practicable, to accept public comments on proposed rules “by electronic means” and to ensure that a publicly accessible federal Web site contains “electronic dockets” for their proposed rules. The act also established an Office of Electronic Government within OMB, headed by an Administrator appointed by the President and required to work with the Administrator of OMB’s Office of Information and Regulatory Affairs in establishing the strategic direction of the e-government program and to oversee its implementation.

Initially, OMB named the Department of Transportation (DOT) as the lead agency, or managing partner, for the e-Rulemaking effort, entrusting DOT with the day-to-day management of the project as well as collaboration with other e-Rulemaking agencies. In a previous report, we noted that “DOT had the most developed electronic docket system of the agencies that we contacted, covering every rulemaking action in the department and including all public comments received regardless of medium.” DOT served as managing partner until an independent consultant’s assessment concluded in August 2002 that the system operated by EPA was the optimal platform to serve as the consolidated electronic docket platform for the entire federal government. As a result of this assessment, OMB transferred managing partner responsibilities from DOT to EPA in late 2002.

The e-Rulemaking initiative provides a single portal for businesses and citizens to access the federal rulemaking process and comment on proposed rules. It is composed of three modules. The first module was completed in January 2003 with the launch of the www.regulations.gov Web site. The second module, which is the focus of this report, will move beyond rule identification and a comment mechanism by establishing a governmentwide electronic docket management system into which all relevant regulatory supporting materials and public comments will be placed. The third and final module will create an electronic regulatory desktop to facilitate the rule development process.

Officials Considered Three E-Rulemaking Designs and Chose a Single, Centralized Design Based on Cost, Deployment Risks, and Security

E-Rulemaking officials and the e-Rulemaking Initiative Executive Committee considered three alternative designs for the federal e- Rulemaking system and ultimately chose to develop and implement a single, centralized design. This decision was based upon studies that examined the costs, deployment risks, and security aspects of the three designs. Throughout the management of the e-Rulemaking initiative, e- Rulemaking officials have estimated that the centralized design would save money; the latest estimate is that it would save approximately $94 million over 3 years. Officials said they used their best professional judgment and information about the costs to develop and operate paper and electronic rulemaking systems.

Centralized, Tiered, and Distributed Designs Considered

E-Rulemaking officials and the e-Rulemaking Initiative Executive Committee considered three designs for the system architecture of the governmentwide e-Rulemaking system. These were a centralized, tiered, or a distributed design. The key characteristics of the governmentwide e- Rulemaking system architecture would: enable agencies to manage content and workflow processes using variable access controls and role definitions; provide a robust and scaleable Web-based solution that supports the capture, conversion, and dissemination of high volumes of information; provide the public and stakeholders with the ability to perform electronic docket searching, viewing, and commenting across multiple agencies; and minimize the total cost of system ownership and management while delivering responsive service to agencies/subagencies, businesses, and citizens.

Anyone with the ability to connect to the Internet with a standard industry browser would be able to access the e-Rulemaking system.

Centralized design. The centralized design uses a minimal number of governmentwide e-Rulemaking system components in the same location to provide consistent access and services for citizens and agency staff across all dockets. Under this design, the features and functions of the governmentwide e-Rulemaking system are provided by component-based application architecture and all standard components are centrally located. These application components would reside on servers in a single location for delivery of services to the public and the agencies. Unlike the tiered and distributed designs, which are discussed in the next sections, the centralized design will not support existing agency docket systems nor will it distribute components of the governmentwide e-Rulemaking system in agency facilities.

Tiered design. The tiered design utilizes a centralized governmentwide e- Rulemaking system to deliver all agency and citizen services, but is different from the centralized design because common hardware and software components installed in the governmentwide e-Rulemaking system are also installed at different agency sites to enhance system performance. For example, components like the database management or document management system may be placed on separate servers in the same or different locations to process information more efficiently to boost system performance, and the responsibility for the data is dispersed across multiple entities or agency sites and would be maintained locally at each site. Because the tiered design is based on the use of common hardware and software components, the governmentwide e-Rulemaking system is not linked to any existing agency specific docket system with its unique hardware and software. As a result, no customized software interfaces are needed.

Distributed design. The distributed design integrates a centralized governmentwide e-Rulemaking system with existing agency-specific electronic docket systems while satisfying all governmentwide e-Rulemaking system requirements. This design links agencies with existing docket systems to the governmentwide e-Rulemaking system using customized software (middleware) to allow interconnectivity between agencies’ systems and the governmentwide e-Rulemaking system. This design provides the public with a means of searching across agency dockets and establishes the governmentwide e-Rulemaking system as an access point to the public and agencies for searching, reviewing, and commenting on dockets that reside on the agencies’ existing docket systems. In addition, agencies with docket systems will continue to perform their own workflow/business/docket life-cycle processing.

Officials Used Contractor’s Analysis When Preparing Business Case Recommending Centralized Design

In 2002, DOT, which was the managing partner for the e-Rulemaking initiative, contracted with a consulting firm to assess the capabilities of seven existing agency e-Rulemaking systems and prepare a business case describing alternative designs for the system and the recommended design. The firm was to provide advice to DOT about the best technical approach for the initiative, along with a full analysis of alternatives that leverage the use of existing technology to meet the vision, goals, and objectives of the initiative. Based on its assessment, the consulting firm’s August 2002 report recommended that EPA’s eDocket system would be the optimal platform for a governmentwide centralized e-Rulemaking system. The business case discussed two other alternative designs as well. DOT submitted this business case to OMB in September 2002.

Based on the recommendation to use EPA’s e-Rulemaking system as the platform for a centralized system, OMB transferred managing partner responsibilities for the e-Rulemaking initiative from DOT to EPA. After EPA became the managing partner in late 2002, EPA submitted three additional business cases to OMB—in December 2002, September 2003, and September 2004—and recommended the centralized design. During this time period, e-Rulemaking officials also hired a contractor to refine the three alternative designs, summarize the costs and risks for each alternative, and recommend one alternative for implementation. E- Rulemaking officials used the contractor’s January 2004 report recommending the centralized alternative when preparing the last of the three business cases that was submitted to OMB.

Three Assessments Favored Centralized Design

E-Rulemaking officials and the e-Rulemaking Initiative Executive Committee based the decision to select the centralized design over the tiered and distributed designs on three assessments that generally found that the centralized design was most cost effective, had the lowest risk for deployment and support instability, was the most secure, and was most likely to deliver the breadth and functionality sought by agencies and the public. E-Rulemaking officials hired a contractor to conduct two analyses based on cost and risk models, and the contractor obtained from a consulting firm a third assessment that was based on industry best practices and experiences in assessing similar architectures.

The first assessment used a cost-analysis modeling tool used by the private sector and government agencies to analyze and estimate the costs associated with software application development. Using this model, the cost estimates to deliver and operate the three designs were $18.7 million for the centralized design, $21.1 million for the tiered design, and $87.2 million for the distributed design (assuming 10 agencies were integrated with the centralized governmentwide e-Rulemaking system). The model predicted it would take 1 year to deliver the centralized design and almost 3 years to deliver the distributed design. Also, according to the model, the complexity of the distributed design would present a high risk of instability to the overall operation and maintenance of the systems.

The second assessment used a model that estimates the total cost of implementing and supporting a specific application within a commercial enterprise and assesses the expected risk of successful deployment and the benefit to the organization of adopting the new capability. Using this model, the cost to deliver and operate the three designs was $20.1 million for the centralized design, $22.8 million for the tiered design, and $94.9 million for the distributed design (assuming 10 agencies were integrated with the centralized governmentwide e-Rulemaking system). The model estimated that there was a 50 to 60 percent greater likelihood of unsuccessful deployment of the distributed design when compared to the other designs. The most significant risk factor accounting for this increased likelihood of failure is system complexity, resulting in major integration, infrastructure, and change management issues that tend to be difficult to resolve.

The third assessment was done by a leading information technology firm with the intent of predicting the cost, risk, security, and supportability of the three designs based on industry best practices and the firm’s experience in assessing similar architectures. The firm recommended the centralized design because it provided for a consistent implementation across the government and was the lowest cost and least risky design. The firm rated the tiered design as “feasible but not recommended” for various reasons including the following. First, the firm said it would be difficult to maintain consistent data quality in a tiered design because the responsibility for data would be spread across multiple entities. Second, consistent security and business continuity policies could not be implemented because each agency would most likely manage its data according to its security policies and business continuity models. Third, data retrieval could be complex because data would be maintained locally at each agency. The firm did not recommend the distributed design for several reasons, including the highly complex implementation and high delivery risk and higher cost for maintenance and support due to the large number of disparate systems involved in the design.

Officials Used Professional Judgment to Develop Estimated Savings

E-Rulemaking officials estimated that the federal government would save approximately $94 million over 3 years by deploying the e-Rulemaking system—$56 million in savings from eliminating duplication of systems and $38 million in savings from annual maintenance fees. Officials said that they developed this estimate prior to completing the three assessments previously described and that at the time they developed this estimate, there was a lack of published information about how much it cost to develop or operate paper or electronic rulemaking systems. They primarily used their professional judgment, information about costs for developing and operating EPA’s paper and electronic rulemaking systems, and information provided by OMB about the number of rulemaking entities to develop the estimate. The estimate assumes, among other things, that all rulemaking entities would either develop an electronic rulemaking system or, if they already had an electronic rulemaking system, they would continue to operate those systems.

Agencies Generally Expressed Satisfaction with E-Rulemaking Officials’ Efforts to Solicit Their Input

E-Rulemaking officials extensively collaborated with rulemaking agencies and used several methods to solicit the participation of those agencies during the e-Rulemaking effort, notably the use of interagency working groups. Most officials at these other rulemaking agencies said they had adequate opportunity to collaborate on the initiative and described the process as effective. They provided us with examples that supported their opinions.

Officials’ Collaborative Efforts Included Three Levels of Panels with Broad Representation

E-Rulemaking officials have successfully used a variety of collaboration tools to encourage participation and arrive at consensus among the participating agencies, particularly the use of interagency working groups. Two core working groups facilitate discussion and decision making for the project: (1) the e-Rulemaking Advisory Board, composed primarily of technical and policy staff from the participating agencies; and (2) the e- Rulemaking Initiative Executive Committee, composed of upper-level agency managers such as Chief Information Officers and individuals at the Assistant Secretary level, which was added after the project had begun to obtain upper management support from participating agencies—especially those that already have an electronic system in place. Additionally, e-Rulemaking officials have convened specialized working groups to tackle unique areas of concern, such as system design, project funding, or legal issues. These specialized working groups are open forums designed to foster consensus on decisions made on the e-Rulemaking initiative.

Under e-Rulemaking officials’ guidance, specialized working groups are charged with developing specific proposals for the project and transmitting these proposals to the Advisory Board for discussion. The Advisory Board may consider revisions to the proposal before reaching a decision that represents a consensus of the participants. This decision is then sent to the Executive Committee, which makes a final decision on the proposals and recommends a course of action to OMB. Agency officials we spoke with noted that representatives of all agencies are free to participate on the Executive Committee, the Advisory Board, or any working group.

The three levels of panels are supplemented by a variety of other collaborative tools and methods, including: one-on-one meetings with agency officials, surveys, e-mail communication, teleconferencing, and an online library of documents related to the initiative. EPA also provided public notice of its work in the Federal Register and held public forums on the East and West coasts to obtain the views of businesses, citizens, and interest groups regarding the design of the e-Rulemaking system.

Officials Praised for Their Collaboration, Responsiveness to Unique Agency Needs, and Explanations for Suggestions That Were Rejected

The tenor of our discussions with officials of 14 of the 27 agencies serving on the Advisory Board was that they were satisfied with the level of collaboration. Participating agencies indicated that they had adequate opportunity to provide input and described the collaboration of e- Rulemaking officials as effective. Officials from a few agencies even said that in terms of the e-government initiatives, the e-Rulemaking initiative was one of the better collaborative efforts in which they have participated.

Agencies often praised e-Rulemaking officials’ concern for the unique needs of their agency. For example, one agency communicated its concern that it was not clear how the e-Rulemaking system would accommodate interim rules. While most rules are not effective until after the rulemaking process is concluded, some interim rules are effective immediately, for example, due to emergency conditions or other concerns related to health and safety. After receiving this comment, e-Rulemaking officials recognized the need to be clear about how interim rules would be processed and how comments on such rules would be addressed. The system was revised in accordance with the agency’s suggestions.

In another instance, one agency disagreed with the implementation schedule it was given for its transition to the e-Rulemaking system. The agency’s officials anticipated that the transition date would overlap with a critical stage in their annual rulemaking cycle and expressed concern that any errors in processing and adopting rules during that time would be particularly harmful. E-Rulemaking officials readily agreed to change the schedule to account for this timing problem.

Even when an agency’s suggestion was not incorporated into the system design, they acknowledged that e-Rulemaking officials treated their concerns fairly, completely, and they understood why the suggestion was rejected. Agencies frequently deferred to the needs of the group after expressing their individual preferences during the collaborative process.

The opinions expressed by these officials are consistent with our prior work. We noted in a recent report that e-Rulemaking initiative officials have successfully used collaboration strategies to achieve consensus with partner agencies on funding contributions to the e-Rulemaking initiative based on agency-specific characteristics.

Work Conducted Mostly Consistent with Key Practices but Written Agreements Did Not Include Performance Measures

While managing the development of the centralized e-Rulemaking system, e-Rulemaking officials have, for the most part, followed key practices for successfully managing a project. However, there are a few practices that officials did not incorporate into the management of the e-Rulemaking initiative, such as including system performance measures in written agreements with agencies. The first agencies began migrating to the centralized e-Rulemaking system in May 2005 and the public is scheduled to have access to the system in September 2005. While all agencies will eventually migrate to the centralized system, the schedule for doing so may change, due in part to funding issues. E-Rulemaking officials have also created a process for approving changes to the system based on concerns or issues identified by agencies or the public.

Officials Mostly Followed Key Practices during System Development

E-Rulemaking officials, for the most part, followed key practices for successfully managing a project while developing the centralized e- Rulemaking system; however, there are a few practices that they did not follow. Table 1 summarizes the key practices they followed and those that they did not follow when developing the centralized e-Rulemaking system. This table does not indicate how well they followed each practice, but rather, indicates if the steps they took to manage the e-Rulemaking initiative reflected the presence of the practice.

E-Rulemaking officials followed all of the key practices except two. They did not document completely decisions related to the approach they used to identify alternative designs. Also, EPA, as managing partner, does not have written agreements that address system performance measures with agencies participating in the e-Rulemaking initiative. Although the written agreements do not include performance measures, e-Rulemaking officials do have performance goals that they are measuring. For example, one goal is to have the centralized e-Rulemaking system available to the public and agency rule writers and managers 99.99 percent of the time. Officials also said they are developing a postimplementation review plan.

EPA, as managing partner, signed written agreements with 15 agencies during fiscal years 2003 or 2004 that indicated EPA would establish performance measures, but the agreements did not include any details about them. These agreements did include expected outcomes, roles and responsibilities, and resource commitments. These 15 agencies—which committed to providing financial assistance or in-kind resources to the initiative—included the 5 agencies, or parts of agencies, that are migrating to the centralized e-Rulemaking system during the first migration phase. During fiscal year 2005, EPA plans to sign similar agreements with 20 additional agencies that include these three items. The additional agencies are those to whom OMB has issued budget guidance for 2005 regarding providing funding to EPA for the e-Rulemaking initiative. As additional agencies become involved in funding the initiative, e-Rulemaking officials said they plan to sign similar agreements with them.

Officials Have Process for Approving Changes to the System Based on Agency and Public Input as More Agencies Migrate to It

In May 2005, the agencies included in phase I of the migration schedule— EPA, the Department of Housing and Urban Development, the Animal and Plant Health Inspection Service within the Department of Agriculture, a portion of the Department of Homeland Security, and the National Archives and Records Administration—began migrating to the centralized e- Rulemaking system. Current plans are for 18 additional departments or agencies to begin using the system in fiscal year 2006, resources permitting. The remaining rulemaking departments or agencies are scheduled to begin using the system at different times during fiscal year 2007 and beyond. According to e-Rulemaking officials, the schedule may change depending on funding. They originally planned to migrate 10 agencies to the new system during phase I, but had to cut back due to funding shortfalls and late contributions, specifically in fiscal year 2004.

When an agency is scheduled to migrate to the new system, it will go through a multistep process tailored to meet its needs. E-Rulemaking officials will sign an agreement that outlines the steps the agency will go through to migrate to the centralized system. Upon request, prior to initiating the implementation process, e-Rulemaking officials will meet with a department or agency to brief them on the initiative and demonstrate the system. The implementation steps, in chronological order, are: 1. 2. site survey (i.e., identify and analyze agency-specific technical and functional needs); 3. data preparation (i.e., develop agency-specific implementation); 4. agency configuration (i.e., develop technical and functional engineering requirements to support agency-unique implementation); 5. training (i.e., develop plans and conduct training); 6. usability testing (i.e., allow agency users to become familiar with the e- 7. data migration (i.e., migrate data to the system); 8. moving to production (i.e., agency goes “live” using the system); and 9. postproduction support.

Five of the nine steps are high-level checkpoints at which the agency and e- Rulemaking officials agree that conditions are acceptable to move forward. These steps are the kick-off meeting, data preparation, training, usability testing, and moving to production.

The centralized e-Rulemaking system will not remain static as additional agencies migrate to it, according to e-Rulemaking officials. They said that changes to the system will be made when valid concerns or issues are identified. Such concerns or issues could be raised by agencies that have already migrated to the system, agencies that will be migrating to the system, or public users of the system. E-Rulemaking officials are already planning changes to the system before phase II of the migration schedule begins. These changes are based on issues identified during beta testing by the agencies and usability testing by the public. According to e-Rulemaking officials, they have created a Change Control Board that will review and determine which requests for changes to the system will be granted. This board will also prioritize the changes that are to be made and obtain an estimate of the cost to implement them from the contractor assisting in the design and implementation of the system.

A couple of agencies said they had concerns about whether the centralized system would include all the capabilities of their current electronic systems. For example, one agency said that their system tracks hearings as well as regulatory dockets. E-Rulemaking officials told these agencies that any such capabilities will be incorporated into future versions of the centralized e-Rulemaking system, provided adequate funding is provided. Moreover, they noted that because each agency signs an agreement with the e-Rulemaking Program Office before migrating to the centralized system, the agency can assure itself that the system has the capabilities it needs before signing the agreement and moving forward with migration.

Conclusions

The process e-Rulemaking officials and the e-Rulemaking Initiative Executive Committee used to decide which of the three designs— centralized, tiered, or distributed—to develop and implement as well as the basis for that decision, was reasonable and adequately supported. Using two cost and risk models and comparing the three designs to industry best practices was a sound approach to use in order to select which design should be implemented. In addition, because there was a lack of published information about the costs to develop or operate either a paper or electronic rulemaking system, e-Rulemaking officials used their professional judgment and own experiences to estimate how much the centralized e-Rulemaking system could save the federal government.

E-Rulemaking officials established a governance structure to collaborate with other agencies to obtain input on developing and implementing the centralized e-Rulemaking system and its collaboration efforts with other agencies have been extensive and well-received. Most of the agencies we contacted were very satisfied with the collaboration efforts and thought that e-Rulemaking officials listened to their ideas and used them when developing and implementing the system. Officials from a few agencies we interviewed even said that in terms of the e-government initiatives, the e- Rulemaking initiative was one of the better collaborative efforts in which they have participated.

While managing the development of the centralized e-Rulemaking system, e-Rulemaking officials followed most of the key practices for successfully managing a project. E-Rulemaking officials could, however, improve their management of the e-Rulemaking initiative by including system performance measures in written agreements with agencies. Having such agreements would provide criteria for determining if the e-Rulemaking initiative is operating in the most effective, efficient, and economic manner possible.

Recommendation

E-Rulemaking officials established a governance structure that allowed it to successfully collaborate with other agencies about how to develop and implement the centralized e-Rulemaking system and used most of the key practices included in this report for managing an initiative. To learn from how it managed this initiative and to build on the success of it, we recommend that the Administrator of EPA, as managing partner of this initiative, take steps to ensure that the written agreements between EPA and the participating agencies include performance measures that address issues such as system performance, maintenance, and cost savings. These measures are necessary to provide criteria for evaluating the effectiveness of the e-Rulemaking initiative as well as for determining if the initiative is operating in the most efficient and economical manner.

Agency Comments

The Administrator of EPA was provided a draft of this report for his review and comment. The EPA Assistant Administrator and Chief Information Officer provided written comments on the draft in an August 17, 2005, letter, which is reprinted in appendix III. The Assistant Administrator agreed that the report accurately describes the e-Rulemaking Initiative. She added that e-Rulemaking officials appreciated GAO’s recommendation to ensure the effectiveness of the initiative and they look forward to continuing to work with GAO for the continued success of the project. E- Rulemaking officials said they agree with GAO’s recommendation and they plan to implement it.

As agreed with your office, unless you publicly announce the contents of this report earlier, we will not distribute it until 30 days from the date of this letter. At that time we will send copies of this report to the Chair and Ranking Minority Member of the House Committee on Government Reform, the Administrator of EPA, and the Director of OMB. We will make copies available to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you have any questions regarding this report, please contact me at (202) 512-5837 or at williamso@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report were Thomas Beall, Boris Kachura, Steven Law, Joseph Santiago, Shellee Soliday, and Grant Turner.

Objectives, Scope, and Methodology

While the Office of Management and Budget (OMB) oversees the e- Rulemaking initiative, it named the Environmental Protection Agency (EPA) managing partner for the initiative. As managing partner, EPA heads the Program Management Office (PMO), which is staffed by officials from several rulemaking agencies participating in this initiative. The PMO staff manages the day-to-day activities of the e-Rulemaking initiative while the e- Rulemaking Initiative Executive Committee makes the final decisions about the initiative’s strategy, resources, and timetable. Hereafter, we refer to PMO officials as e-Rulemaking officials.

Our first objective was to describe the approach EPA used to identify alternative designs for the e-Rulemaking system and how the decision to proceed with a single, centralized system design was made. To address this objective, we reviewed the Capital Asset Plan and Business Cases submitted by the Environmental Protection Agency (EPA) and the Department of Transportation (DOT). We also reviewed a contractor’s analysis of the capabilities of current e-Rulemaking systems, a contractor’s summary of assessments of the three identified alternative designs including analyses of operational risks and security vulnerabilities, and documentation related to the basis for e-Rulemaking officials’ and the e- Rulemaking Initiative Executive Committee’s recommendation about which design alternative to implement. We also interviewed OMB, e- Rulemaking, and DOT officials involved with developing information about the design alternatives, the selection of the centralized system design, and the development of the estimated cost savings associated with the centralized design alternative.

Our second objective was to describe how EPA collaborated with other rulemaking agencies to obtain input about the e-Rulemaking system and those agencies’ views regarding the collaboration. To address this objective, we interviewed e-Rulemaking officials and reviewed documentation related to collaboration efforts such as those describing the purpose and composition of various committees and work groups and written agreements between agencies and EPA regarding funding of the system. To acquire agency views on the collaboration that has occurred, we randomly selected 12 of the 27 agencies that serve on the e-Rulemaking Initiative Advisory Board and interviewed officials from those agencies about their experiences with the collaboration efforts. Included in this set of selected agencies were (1) agencies that serve on the e-Rulemaking Executive Committee, (2) agencies that currently have an e-Rulemaking system, and (3) agencies that are heavily involved in rulemaking. We analyzed the agency officials’ responses to our questions on collaboration and identified those views that were most frequently expressed by officials from multiple agencies as a means of gauging the overall quality of the collaboration.

In addition to the agencies included in the sample, we discussed collaboration with two additional agencies that serve on the Advisory Board. The first was DOT. Since we contacted DOT officials in relation to the first objective, we also discussed collaboration issues with them. We contacted the second agency, the National Archives and Records Administration (NARA), because many officials included in the original sample suggested that we talk to that agency since an official from NARA was heavily involved in meeting with all agencies to discuss the development of the e-Rulemaking system. We also discussed the methods used to obtain information from the public about the development of the e- Rulemaking system with PMO officials and reviewed related documents.

Our third objective was to (1) determine whether EPA used key practices for successfully managing a project when managing the e-Rulemaking initiative and (2) describe EPA’s future plans for developing and implementing the centralized e-Rulemaking system. To determine whether e-Rulemaking officials followed key practices, we identified key practices for successfully managing a project by reviewing previous GAO work related to the e-government initiatives. The key practices included in prior GAO reports were based in part on studies done by federal, state, and local agencies, international agencies, and the private sector and on guidance provided by the federal government, state, local and international governments, private research groups, national associations, and educational institutions. GAO information technology staff agreed with the list of key practices we identified. After developing the list of key practices, we compared the list to the information we had gathered about e-Rulemaking officials’ management of the e-Rulemaking initiative. When making the comparison, we did not assess how well each key practice was followed, but rather we determined if the steps they took to manage the e- Rulemaking initiative reflected the presence of the practice. We did not attempt to assess the quality or extent of each practice’s implementation. In order to describe future plans for the initiative, we reviewed documents that addressed future plans such as the business cases and implementation schedules and interviewed e-Rulemaking officials. We did not assess the quality of these plans.

Although we report cost and related data from contractor assessments of the three governmentwide e-Rulemaking system design alternatives, we did not examine the reliability of that data since our first objective was to describe how e-Rulemaking officials selected one of the three alternatives rather than to determine if the appropriate alternative was selected. We performed our work from December 2003 through June 2005 in accordance with generally accepted government auditing standards.

List of GAO Reports Related to E-Rulemaking

Federal Rulemaking: Agencies’ Use of Information Technology to Facilitate Public Participation. GAO/GGD-00-135R. Washington, D.C.: June 30, 2000.

Electronic Government: Government Paperwork Elimination Act Presents Challenges for Agencies. GAO/AIMD-00-282. Washington, D.C.: September 15, 2000.

Electronic Government: Better Information Needed on Agencies’ Implementation of the Government Paperwork Elimination Act. GAO-01- 1100. Washington, D.C.: September 28, 2001.

Regulatory Management: Communication About Technology-Based Innovations Can Be Improved. GAO-01-232. Washington, D.C.: February 12, 2001.

Information Technology: OMB Leadership Critical to Making Needed Enterprise Architecture and E-government Progress. GAO-02-389T. Washington, D.C.: March 21, 2002.

Electronic Government: Selection and Implementation of the Office of Management and Budget’s 24 Initiatives. GAO-03-229. Washington, D.C.: November 22, 2002.

Electronic Government: Success of the Office of Management and Budget’s 25 Initiatives Depends on Effective Management and Oversight. GAO-03-495T. March 13, 2003.

Electronic Rulemaking: Efforts to Facilitate Public Participation Can Be Improved. GAO-03-901. Washington, D.C.: September 17, 2003.

Electronic Government: Potential Exists for Enhancing Collaboration on Four Initiatives. GAO-04-6. Washington, D.C.: October 10, 2003.

Electronic Government: Initiatives Sponsored by the Office of Management and Budget Have Made Mixed Progress. GAO-04-561T. Washington, D.C.: March 24, 2004.

Electronic Government: Federal Agencies Have Made Progress Implementing the E-Government Act of 2002. GAO-05-12. Washington, D.C.: December 10, 2004.

Electronic Government: Funding of the Office of Management and Budget’s Initiatives. GAO-05-420. Washington, D.C.: April 25, 2005.