Information Technology: Leadership Remains Key to Agencies Making Progress on Enterprise Architecture Efforts
Released: 2003-12-18T00:00:00

Letter

Background

The concept of an architecture to describe an enterprise first emerged in the mid-1980s, and over the years, various frameworks for defining the content of enterprise architectures have been published. Our work in the early 1990s identified architectures as critical success factors in allowing organizations to effectively apply IT to meet mission goals. Since then, we have worked with the Congress, OMB, and the federal Chief Information Officers (CIO) Council to promote the importance of architectures and assist agencies in developing, maintaining, and using them. In our reviews of selected agency IT management practices and major systems modernization programs, we have continued to identify the lack of an architecture as a major management weakness, and we have made recommendations to address this important area.

A Brief Description

In simple terms, an enterprise can be viewed as any purposeful activity, and an architecture can be characterized as the structure (or structural description) of any activity. Building on this, we can view enterprise architectures as systematically derived and captured structural descriptions—in useful models, diagrams, and narrative—of the mode of operation for a given enterprise, which can be either a single organization or a functional or mission area that transcends more than one organizational boundary (e.g., financial management, homeland security).

The architecture describes the enterprise’s operations in both logical terms (such as interrelated business processes and business rules, information needs and flows, and work locations and users) and technical terms (such as hardware, software, data, communications, and security attributes and performance standards). It provides these perspectives both for the enterprise’s current (or “as-is”) environment and for its target (or “to-be”) environment, as well as a transition plan for moving from the “as-is” to the “to-be” environment.

Importance of Enterprise Architectures

The importance of enterprise architectures is a basic tenet of IT management, and their effective use is a recognized hallmark of successful public and private organizations. For over a decade, we have promoted the use of architectures, recognizing them as a crucial means to a challenging goal: that is, agency operational structures that are optimally defined, in terms of both business and technology. The alternative, as our work has shown, is perpetuation of the kinds of operational environments that saddle most agencies today, in which the lack of integration among business operations and the IT resources that support them leads to systems that are duplicative, not well integrated, and unnecessarily costly to maintain and interface.

Managed properly, an enterprise architecture can clarify and help optimize the interdependencies and relationships among an organization’s business operations and the underlying IT infrastructure and applications that support these operations. Employed in concert with other important IT management controls (such as portfolio-based capital planning and investment control practices), architectures can greatly increase the chances that organizations’ operational and IT environments will be configured so as to optimize mission performance. Enterprise architectures are integral to managing large-scale programs as well as initiatives that span several agencies, such as those currently being undertaken to support the electronic government (e-government) efforts led by OMB.

Brief History of Enterprise Architecture Frameworks and Management Guidance

During the mid-1980s, John Zachman, widely recognized as a leader in the field of enterprise architecture, identified the need to use a logical construction blueprint (i.e., an architecture) for defining and controlling the integration of systems and their components. Accordingly, Zachman developed a structure or framework for defining and capturing an architecture, which provides for six “windows” from which to view the enterprise. Zachman also proposed six abstractions or models associated with each of these perspectives. Zachman’s framework provides a way to identify and describe an entity’s existing and planned component parts, and the relationships between those parts, before the entity begins the costly and time-consuming efforts associated with developing or transforming itself.

Since Zachman introduced his framework, a number of frameworks have emerged within the federal government, beginning with the publication of the National Institute of Standards and Technology (NIST) framework in 1989. Since that time, other federal entities have issued enterprise architecture frameworks, including the Department of Defense (DOD) and the Department of the Treasury. In September 1999, the federal CIO Council published the Federal Enterprise Architecture Framework (FEAF), which was intended to provide federal agencies with a common construct for their architectures, thereby facilitating the coordination of common business processes, technology insertion, information flows, and system investments among federal agencies. The FEAF describes an approach, including models and definitions, for developing and documenting architecture descriptions for multiorganizational functional segments of the federal government.

More recently, OMB established the Federal Enterprise Architecture Program Management Office to develop a Federal Enterprise Architecture (FEA) according to a collection of five “reference models,” which are intended to facilitate governmentwide improvement through cross-agency analysis and the identification of duplicative investments, gaps, and opportunities for collaboration, interoperability, and integration within and across government agencies. The FEA reference models are summarized in table 1.

Although these post-Zachman frameworks differ in their nomenclatures and modeling approaches, each consistently provides for defining an enterprise’s operations in both logical terms and technical terms, provides for defining these perspectives for the enterprise’s current and target environments, and calls for a transition plan between the two.

Several laws and regulations have established requirements and guidance, respectively, for agencies’ management of architectures, beginning with the Clinger-Cohen Act in 1996, which directs the CIOs of major departments and agencies to develop, maintain, and facilitate the implementation of IT architectures as a means of integrating agency goals and business processes with IT. OMB Circular A-130, which implements the Clinger- Cohen Act, requires that agencies document and submit their initial enterprise architectures to OMB and that agencies submit updates to OMB when significant changes to their enterprise architectures occur. The circular also directs the OMB Director to use various kinds of reviews to evaluate the adequacy and efficiency of each agency’s compliance with the circular.

OMB was given explicit responsibility for overseeing government enterprise architectures by the E-Government Act of 2002, which established the Office of Electronic Government within OMB. This act gives OMB the responsibility for facilitating the development of enterprise architectures within and across agencies and supporting improvements in government operations through the use of IT.

Prior Work Indicates Opportunities for Improving Enterprise Architectures

We began reviewing federal agencies’ use of architectures in 1994, initially focusing on those agencies that were pursuing major systems modernization programs that were high risk. These included the National Weather Service systems modernization, the Federal Aviation Administration air traffic control modernization, and the Internal Revenue Service (IRS) tax systems modernization. Generally, we reported that these agencies’ enterprise architectures were incomplete, and we made recommendations that they develop and implement complete enterprise architectures to guide their modernization efforts.

Since then, we have reviewed architecture management at other federal agencies, including the Department of Education, the former Customs Service, the former Immigration and Naturalization Service, and the Centers for Medicare and Medicaid Services. We have also reviewed the use of enterprise architectures for critical agency functional areas, such as the integration and sharing of terrorist watch lists across key federal departments, and the logistics management area within DOD. These reviews have continued to identify the absence of complete and enforced enterprise architectures, which in turn has led to agency business operations, systems, and data that are not integrated (“stovepiped”), duplicative, and incompatible. These conditions have either prevented agencies from sharing data or forced them to depend on expensive, custom-developed interface systems to do so.

Our Enterprise Architecture Management Maturity Framework

In 2002, we published Version 1.0 of our Enterprise Architecture Management Maturity Framework (EAMMF) to provide federal agencies with a common benchmarking tool for planning and measuring their enterprise architecture efforts, as well as to provide OMB with a means for doing the same governmentwide. This framework is an extension of A Practical Guide to Federal Enterprise Architecture, Version 1.0, published by the CIO Council. The framework arranges core elements from the practical guide into a matrix of five hierarchical stages and four critical success attributes; that is, each core element appears at a particular stage of maturity, and it is also associated with one of the critical success attributes. In April 2003, we published Version 1.1 of this framework, which reflects changes and additions that are based on comments we received on the initial version.

EAMMF Version 1.0

The EAMMF is made up of five stages of maturity, each of which includes an associated set of elements, along with all of the elements of the previous stages. Table 2 shows these stages, followed by the description of each as contained in Version 1.0 of our framework.

Stage 1: Creating EA awareness. Agencies at this stage are characterized either by no plans to develop and use an enterprise architecture, or by plans and actions that do not yet demonstrate an awareness of the value of having and using one. Although Stage 1 agencies may have initiated some enterprise architecture core elements, these agencies’ efforts are ad hoc and unstructured, and they do not provide the management foundation that is necessary for successful enterprise architecture development.

Stage 2: Building the EA management foundation. The focus at Stage 2 is on assignment of roles and responsibilities and establishment of plans for developing enterprise architecture products. Specifically, a Stage 2 agency has designated a chief architect and established and staffed a program office that is responsible for enterprise architecture development. Further, a steering committee or group that has responsibility for directing and overseeing the development has been established, and the membership of the steering committee includes business and IT representatives. At Stage 2, the agency either has plans for developing or has begun development of at least some of the necessary enterprise architecture products. This stage also requires the agency to have selected both a framework that will be the basis for the nature and content of the specific products it plans to develop and an automated tool to help in the development.

Stage 3: Developing architecture products. At Stage 3, an agency focuses on actual development of enterprise architecture products. The agency has defined the scope of its enterprise architecture as encompassing the entire enterprise, whether an organization or a functional area, and it has a written and approved policy demonstrating institutional commitment. Although the products may not yet be complete, they are intended to describe the agency in terms of business, data, applications, and technology. Further, the products are to describe the current and future states and the sequencing plan for transitioning from current to future state. As the architecture products are being developed, they are to be subject to configuration control.

Stage 4: Completing EA products. An agency at Stage 4 has complete and approved enterprise architecture products that it can use to help select and control its portfolio of IT investments. The complete products describe the organization in terms of business, data, applications, and technology. Also, the products are complete in that they describe the agency’s current and future states and the transition plan for sequencing from the current state to the future state. Further, the agency’s CIO has approved the enterprise architecture, and the agency has a written policy requiring that IT investments comply with the enterprise architecture.

Stage 5: Leveraging the EA to manage change. At Stage 5, an agency is able to evolve the enterprise architecture products according to a written and approved policy for maintaining the architecture. Also at this stage, the steering committee, investment review board, or agency head approves the enterprise architecture. Finally, the agency has incorporated the enterprise architecture into its corporate decision making, and it has established and is using metrics to measure the effectiveness of its enterprise architecture.

In addition to the maturity stages, each core element is also associated with attributes that are critical to the successful performance of any management function (see table 3). The critical success attributes are identical in Versions 1.0 and 1.1 of our framework.

Attribute 1: Demonstrates commitment. Because the enterprise architecture is a corporate asset for systematically managing institutional change, the support and sponsorship of the head of the enterprise are essential to the success of the architecture effort. An approved enterprise policy statement provides such support and sponsorship, promoting institutional “buy in” and encouraging resource commitment from participating components. Equally important in demonstrating commitment is vesting ownership of the architecture with an executive body that collectively owns the enterprise.

Attribute 2: Provides capability to meet commitment. The success of the enterprise architecture effort depends largely on the organization’s capacity to develop, maintain, and implement the enterprise architecture. Consistent with any large IT project, these capabilities include providing adequate resources (i.e., people, processes, and technology); defining clear roles and responsibilities; and defining and implementing organizational structures and process management controls that promote accountability and effective project execution.

Attribute 3: Demonstrates satisfaction of commitment. Satisfaction of the organization’s commitment to develop, maintain, and implement an enterprise architecture is demonstrated by the production of artifacts (e.g., the plans and products). Such artifacts demonstrate “follow through”— actual enterprise architecture production. Satisfaction of commitment is further demonstrated by senior leadership approval of enterprise architecture documents and artifacts; such approval communicates institutional endorsement and ownership of the architecture and the change that it is intended to drive.

Attribute 4: Verifies satisfaction of commitment. This attribute focuses on measuring and disclosing the extent to which efforts to develop, maintain, and implement the enterprise architecture have fulfilled stated goals or commitments. Measuring such performance allows for tracking progress that has been made toward stated goals, allows the appropriate actions to be taken when performance deviates significantly from goals, and creates incentives to influence both institutional and individual behaviors.

Collectively, these attributes form the basis by which an organization can institutionalize management of any given function or program, such as enterprise architecture management. Within each stage, each critical success attribute includes between one and four core elements, which are descriptions of a practice or condition that is needed for effective enterprise architecture management. On the basis of the implicit dependencies among the core elements, the EAMMF associates each element with one of five hierarchical management stages, referred to as maturity stages. Each stage reflects the collection of enterprise architecture management practices and conditions (i.e., core elements) that are being undertaken by an enterprise at a given maturity level.

Figure 1 is a summary of Version 1.0 of the framework, showing the key elements associated with the stages and attributes previously described.

Version 1.1 of this framework was released in April 2003. Like the initial version, Version 1.1 is based on the CIO Council guidance and augmented by our research experience in reviewing architecture programs. Changes and additions to the framework were also based on comments received on the initial version. As a comparison between the two frameworks shows, a number of new elements have been added to Version 1.1. Figure 2 shows a summary of the new framework, Version 1.1.

The stages and attributes remain the same as with Version 1.0, although the descriptions of the stages are updated in Version 1.1 to reflect the new elements in the framework, as follows: Stage 1: Creating EA awareness. As with Version 1.0, at Stage 1, either an organization does not have plans to develop and use an architecture, or it has plans that do not demonstrate an awareness of the value of having and using an architecture. Although Stage 1 agencies may have initiated some enterprise architecture activity, these agencies’ efforts are ad hoc and unstructured, lack institutional leadership and direction, and do not provide the management foundation that is necessary for successful enterprise architecture development as defined in Stage 2.

Stage 2: Building the EA management foundation. An organization at Stage 2 recognizes that the enterprise architecture is a corporate asset by vesting accountability for it in an executive body that represents the entire enterprise. At this stage, an organization assigns enterprise architecture management roles and responsibilities and establishes plans for developing enterprise architecture products and for measuring program progress and product quality. An organization at this stage also commits the necessary resources for developing an architecture—people, processes, and tools. Specifically, a Stage 2 organization has designated a chief architect and established and staffed a program office that is responsible for enterprise architecture development and maintenance. Further, it has established a committee or group that has responsibility for enterprise architecture governance (i.e., directing, overseeing, and approving architecture development and maintenance). This committee or group membership has enterprisewide representation. At Stage 2, the organization either has plans for developing or has started developing at least some enterprise architecture products, and it has fostered an enterprisewide awareness of the value of enterprise architecture and its intended use in managing its IT investments. The organization has also selected a framework and a methodology that will be the basis for developing the enterprise architecture products and has selected a tool for automating these activities.

Stage 3: Developing the EA. An organization at Stage 3 focuses on developing architecture products according to the selected framework, methodology, tool, and established management plans. Roles and responsibilities assigned in the previous stage are in place, and resources are being applied to develop actual enterprise architecture products. At this stage, the scope of the architecture has been defined to encompass the entire enterprise, whether an organization or a functional area. Although the products may not be complete, they are intended to describe the organization in terms of business, performance, information/data, service/application, and technology (including security explicitly in each), as provided for in the framework, methodology, tool, and management plans. Further, the products are to describe the current (“as-is”) and future (“to-be”) states and the plan for transitioning from the current to the future state (the sequencing plan). As the products are developed and evolve, they are subject to configuration management. Further, through the established enterprise architecture management foundation, the organization is tracking and measuring its progress against plans; identifying and addressing variances, as appropriate; and then reporting on its progress.

Stage 4: Completing the EA. An organization at Stage 4 has completed its enterprise architecture products, meaning that the products have been approved by the enterprise architecture steering committee (established in Stage 2) or an investment review board, and by the CIO. The completed products collectively describe the enterprise in terms of business, performance, information/data, service/application, and technology for both its current and future operating states, and the products include a sequencing plan for transitioning from the current to the future state. Further, an independent agent has assessed the quality (i.e., completeness and accuracy) of the enterprise architecture products. Additionally, evolution of the approved products is governed by a written enterprise architecture maintenance policy that is approved by the head of the organization.

Stage 5: Leveraging the EA to manage change. An organization at Stage 5 has secured senior leadership approval of the enterprise architecture products and a written institutional policy stating that IT investments must comply with the architecture, unless granted an explicit compliance waiver. Further, decision makers are using the architecture to identify and address ongoing and proposed IT investments that are conflicting, overlapping, not strategically linked, or redundant. As a result, Stage 5 entities avoid unwarranted overlap across investments and ensure maximum systems interoperability, which in turn ensures the selection and funding of IT investments with manageable risks and returns. Also, at Stage 5, the organization tracks and measures enterprise architecture benefits or return on investment, and adjustments are continuously made to both the enterprise architecture management process and the enterprise architecture products.

Significant Differences between EAMMF Versions 1.0 and 1.1

Overall, Version 1.1 is more demanding (i.e., sets a higher standard) than Version 1.0 because Version 1.1 adds important content, clarifies existing content, and links the EAMMF framework to related IT management guidance, such as our IT investment management framework. Key differences in Version 1.1 of the framework appear first in Stage 2 and affect later stages either explicitly or implicitly. That is, some planning elements associated with Stage 2 now propagate explicitly through later stages as plans are executed and architecture products are developed, completed, and implemented. For example: Version 1.1 includes “performance” among the models that are needed to describe the “as-is” and “to-be” environments; these models are introduced into the planning elements in Stage 2 and built upon as plans are executed: that is, as architecture products are developed and completed in Stages 3 and 4, respectively.

Version 1.1 explicitly recognizes the need to address security in the descriptions of the “as-is” and “to-be” environments; this element is introduced in Stage 2 and reiterated in Stages 3 and 4.

Version 1.1 introduces the need to plan for metrics in Stage 2 and to measure different aspects of enterprise architecture development, quality, and use in Stages 3, 4, and 5.

Other differences introduced in Version 1.1 affect later stages implicitly, since each stage includes all elements of previous stages. For example, in Stage 2, an element has been added that recognizes the need for adequate resources (people, processes, and technology). This element appears in Stage 2 explicitly, but it is included in later stages implicitly. Stage 4 now includes an element requiring that enterprise architecture products and management processes undergo independent verification and validation; this element continues in Stage 5.

In addition, two core elements, both in Stage 2, have been altered from Version 1.0, as follows: Enterprise architecture maintenance, in addition to development, is now included among the responsibilities of the program office.

The use of an enterprise architecture methodology is added to the use of a framework and automated tool in developing the architecture.

Last, the sequence of two elements (the policies on maintenance and on IT investment compliance with the architecture) is reversed in Version 1.1. That is, maintenance policy is now associated with Stage 4 and investment compliance with Stage 5. This reordering reflects greater alignment of these elements with the definitions of their respective framework stages.

Finally, several new elements were added to Stage 5 that provide for maximizing the value and use of the enterprise architecture by keeping it current and using it to manage change (including the existence of a process to formally manage enterprise architecture change, the enterprise architecture being an integral component of the IT investment management process, the periodic updating of enterprise architecture products, and the compliance of IT investments with the enterprise architecture). These and the other changes are summarized in table 4.

Our 2001 Survey Showed the Immature State of Federal Agency Architecture Management

We first surveyed enterprise architecture management maturity across the federal government in 2001, and we reported in February 2002 that about 52 percent of federal agencies reported having at least the management foundation that is needed to begin successfully developing, implementing, and maintaining an enterprise architecture, and that about 48 percent of agencies had not yet advanced to that basic stage of maturity. At the other extreme, about 4 percent of federal agencies’ enterprise architecture efforts had matured to the point that they could be considered effective, with one agency attaining the highest stage of maturity. This overall state of affairs was consistent for the three agency types that we surveyed: cabinet- level departments (e.g., Treasury); department component agencies (e.g., IRS, which is a component of Treasury); and independent agencies (e.g., Social Security Administration).

We also reported that the state of architecture management across the federal government was attributable to four management challenges that agencies reported facing as they attempt to develop and use architectures. These challenges were (1) overcoming limited executive understanding, (2) inadequate funding, (3) insufficient skilled staff, and (4) organizational parochialism. Additionally, we recognized OMB’s efforts to promote and oversee agencies’ enterprise architecture efforts. Nevertheless, we determined that OMB’s leadership and oversight could be improved by, for example, using a more structured means of measuring agencies’ progress and by addressing the above management challenges. To this end, our February 2002 report provided OMB with the necessary baseline data, improvement framework, and several recommendations. OMB generally agreed with our findings and conclusions in that report and stated that it would consider our recommendations.

Agencies Are Making Limited Architecture Management Progress; Most Programs Remain Immature

Our 2003 survey results indicate that while some individual agencies have made progress in improving their enterprise architecture management maturity, progress for the federal government as a whole has not occurred. Specifically, while about one-fourth of all agencies improved their enterprise architecture management maturity stage relative to Version 1.0 of our framework, about one-fourth of all agencies decreased in maturity and about one-half of all agencies remained at the same stage. Furthermore, the more demanding standard established by our framework Version 1.1 caused a decline in agency maturity levels, demonstrating that improvements are needed before agencies’ enterprise architecture management practices can be considered effective. The average maturity stage for the 96 responses included in our survey was 1.76 when measured against Version 1.0 of our framework and 1.33 when compared with Version 1.1 of our framework.

Appendix IV provides a list of these individual agencies and their maturity stages.

Agencies’ Architecture Management Progress Is Limited

Overall, little substantial change was revealed in agencies’ overall enterprise architecture maturity when their efforts were compared with Version 1.0 of our framework. Of the 93 agencies included in both our 2001 and 2003 surveys, 22 agencies (24 percent) increased their respective EAMMF maturity stages, 24 agencies (26 percent) decreased their stages, and 47 agencies (51 percent) remained the same. (See fig. 3.)

At the department level, 4 departments increased their maturity stage, 4 decreased, and 6 stayed at the same stage. The Department of Homeland Security—which began operations as a department in March 2003— debuted at Stage 3. Although progress for agencies in the aggregate continued to be limited, departments as a group made the most progress: the average maturity for the 14 departments that responded to both the 2001 and 2003 surveys increased from 1.93 to 2.00 against Version 1.0 of the framework. In contrast, component agencies showed a slight decline in maturity against Version 1.0. Specifically, of the component agencies that responded to both surveys, 9 increased their maturity stage, 15 decreased in maturity, and 31 stayed the same, with the average maturity stage decreasing from 1.69 to 1.62. For independent agencies that responded to both surveys, 9 increased their maturity stage, 5 decreased in maturity, and 10 stayed at the same stage. On average, independent agencies showed an increase in maturity, from 1.75 to 1.96 against Version 1.0.

Figure 4 summarizes the maturity status of departments, components, independent agencies, and all agencies, according to Version 1.0 of our framework, and compares our 2001 and 2003 survey results.

Most agencies that made progress from 2001 to 2003 moved from a lower maturity stage to Stage 2 or 3 (as shown in fig. 4, most agencies were clustered in Stages 1 and 2, so this is not unexpected). In particular, of the 22 agencies that increased their maturity stage, 6 increased from Stage 1 to Stage 2, and 12 increased from Stage 1 or 2 to Stage 3. Most agencies that regressed fell to Stage 1 from Stages 2 and 3. Specifically, of the 24 agencies that decreased their maturity stage, 16 decreased to Stage 1 from Stage 2 or 3. Figure 5 shows the number of agencies whose maturity levels improved and declined between 2001 and 2003 as measured against Version 1.0 of our maturity framework.

Agencies’ progress since our first survey is similarly limited when we consider the total number of core elements satisfied. The 93 agencies that responded to both the 2001 and 2003 surveys satisfied an average of about 11 of the 19 elements in Version 1.0 in both 2001 and 2003. As a whole, the 93 agencies satisfied about 57 percent of all possible framework elements in 2001 and about 60 percent of all possible framework elements in 2003.

From 2001 to 2003, agencies showed improvements in satisfying certain core elements, but these improvements were offset by declines in agency satisfaction of other core elements. Examples of core elements where agency satisfaction significantly improved are as follows: “Metrics exist for measuring EA benefits” (about a 38 percent increase), “Chief architect exists” (about a 23 percent improvement), and “EA products are under configuration management” (about an 18 percent increase).

Examples of core elements where agency satisfaction significantly declined are as follows: “EA products describe ‘as-is’ environment, ‘to-be’ environment, and sequencing plan” (about a 39 percent decrease); “EA products describe enterprise’s business—and the data, applications, and technology that support it” (about a 36 percent decrease); “Either EA steering committee, investment review board, or agency head has approved EA” (about a 25 percent decrease); and “Program office responsible for EA development exists” (about a 23 percent decrease).

Figures 6 to 10 show the number of agencies that satisfied the framework elements in each stage of Version 1.0 in 2001 and in 2003.

Appendixes V, VI, and VII provide detailed tables showing each of the 93 agencies’ status regarding the elements of the framework.

For the 22 agencies that advanced one or more maturity stages from 2001 to 2003, fulfillment of no single core element resulted in these advancements. That is, for the 22 agencies, increases in maturity stages are attributable to the fulfillment of 7 core elements spanning three stages of maturity. Table 5 shows those newly satisfied core elements that accounted for increases in maturity stage.

As with increases in agency maturity levels, no single core element accounted for the decreases in agency maturity between our 2001 and 2003 surveys. However, as shown in table 6, the Stage 2 framework element requiring a program office was the most significant newly unsatisfied element for the 24 agencies that decreased maturity levels.

One factor accounting for decreases in maturity is improved accuracy in agencies’ responses to our survey. Improved accuracy is a function of (1) improved agency familiarity with and understanding of enterprise architecture management and our framework since our last survey and (2) the requirement in our 2003 survey for documentation to support certain survey responses.

Current State of Architecture Management across Federal Agencies Is Not Mature

When compared with Version 1.1 of our framework, the state of enterprise architecture management across the federal government is not mature. In particular, about 21 percent of federal agencies (20 of 96) have the Stage 2 management foundation that is needed to begin successfully developing, implementing, and maintaining an enterprise architecture, and about 79 percent of agencies (76 of 96) have not yet advanced to this basic stage of maturity. One agency, the Executive Office of the President, provided responses placing it at a stage of enterprise architecture management maturity that can be considered mature and effective. This overall state of federal government maturity is consistent for each of the three groups that make up the 96 agencies surveyed: departments, component agencies, and independent agencies. Figure 11 summarizes the maturity status of departments, component agencies, independent agencies, and all agencies according to Version 1.1 of our framework.

No single core element that was added to our framework contributed significantly to this situation, but the “methodology” subelement of the Stage 2 element “EA is being developed with a framework, methodology, and automated tool” was the most significant factor keeping agencies from achieving Stage 2. Specifically, the absence of a “methodology” kept 7 agencies from attaining Stage 2 status.

Nevertheless, certain core elements of Version 1.1 of our framework were frequently not satisfied by agencies. Of the 31 core elements in Version 1.1, 17 were not satisfied by over 50 percent of agencies. Furthermore, 8 elements associated with maturity Stages 4 and 5 were not satisfied by over 80 percent of agencies. Figures 12 to 16 show how departments, component agencies, and independent agencies were rated against each of the Version 1.1 core elements.

Although significant gaps existed across federal agencies in meeting the core elements of Version 1.1 of the framework, at least 80 percent of agencies reported performing 8 core elements that were related to Stages 2 and 3 of our framework. The most often satisfied elements included the following Stage 2 elements: “EA plans call for describing both the ‘as-is’ and the ‘to-be’ environments of the enterprise, as well as a sequencing plan for transitioning from the ‘as-is’ to the ‘to-be’”(about 94 percent); “EA plans call for describing both the ‘as-is’ and the ‘to-be’ environments in terms of business, performance, information/data, application/service, and technology” (about 90 percent); and “EA plans call for business, performance, information/data, application/service, and technology descriptions to address security” (about 86 percent).

The most often satisfied elements also included the Stage 3 element: “EA products describe or will describe both the ‘as-is’ and the ‘to-be’ environments of the enterprise, as well as a sequencing plan for transitioning from the ‘as-is’ to the ‘to-be’” (about 88 percent).

In addition, although only one agency has achieved Stage 5, most agencies reported satisfying the Stage 5 core elements requiring that IT investments comply with their enterprise architecture (about 80 percent) and that enterprise architecture is an integral component of IT investment management process (about 69 percent).

Furthermore, 96 percent of agencies in Stages 1 through 4 are performing at least 1 core element above their current maturity stage, which means that agencies as a whole are, to varying degrees, performing above their assigned maturity stages. Specifically, of the 76 agencies at Stage 1, about 95 percent are performing at least 1 core element in a higher maturity stage. About 35 percent of agencies need to satisfy only 1 additional core element to advance to at least the next maturity stage. Two of these agencies, Commerce and the U.S. Mint, could advance two stages by satisfying just 1 additional core element. Commerce, currently a Stage 1 agency, could advance to Stage 3 by satisfying the framework element “Program office responsible for development and maintenance exists.” The Mint, also currently a Stage 1 agency, could advance to Stage 3 by satisfying the framework element “Adequate resources exist.”

Departments, component agencies, and independent agencies had varying degrees of success satisfying certain core elements within individual stages. In general, departments had more success satisfying lower stage elements than did components and independent agencies. In Stage 2, for example, while 69 percent of departments reported using a framework, methodology, and automated tool to develop their enterprise architecture, only 29 percent of components and 50 percent of independent agencies reported the same. Additionally, in Stage 3, while 81 percent of departments reported that progress against plans is measured and reported, only 25 percent of components and 25 percent of independent agencies reported the same. One possible reason for this situation, which is discussed later in this report, is that OMB’s oversight of agency enterprise architecture efforts focuses on departments and major independent agencies—not on component agencies.

Although, as a whole, departments satisfied more lower level framework elements than did component agencies and independent agencies, departments generally still need to satisfy several lower level framework elements to achieve a Stage 3 maturity level. On average, each department needs to satisfy 2 core elements to satisfy all Stage 2 and 3 framework elements.

The maturity stage of a department generally was not indicative of the maturity of its component agencies. For example, the Departments of Health and Human Services and Transportation reached Stage 2, while their component agencies averaged Stage 1. DOD’s Global Information Grid (GIG) architecture was at Stage 3 and its Business Enterprise Architecture was at Stage 1, while DOD components averaged slightly over 1. Conversely, the Departments of Commerce, Justice, and the Treasury were at Stage 1, with their component agencies averaging higher maturity levels. Component agencies of Commerce showed a slightly higher maturity level than did component agencies of other departments. Although the average maturity level of the 56 department component agencies we surveyed was 1.23, the five Commerce component agencies showed an average maturity level of 1.80, largely owing to the maturity levels for the Bureau of the Census (Stage 3), the U.S. Patent and Trademark Office (Stage 2), and the National Oceanic and Atmospheric Administration (Stage 2). The Department of Agriculture’s maturity level (Stage 1) was the same as the average maturity level of its component agencies. Figure 16 summarizes the average maturity level for departments and their respective component agencies.

Eight Agencies Are Either at or Close to Achieving Stage 5 Maturity

The results of our survey and analysis of survey responses against Version 1.1 of our maturity framework show that the Executive Office of the President (EOP) is the sole Stage 5 agency. However, 7 other agencies are close to becoming models of enterprise architecture management. For example, the DOD GIG architecture and IRS, both of which attained Stage 3 of Version 1.1, need to satisfy only 3 more elements to become Stage 5 agencies. To achieve Stage 5, the GIG architecture needs to satisfy the Stage 4 element “EA products describe both the ‘as-is’ and the ‘to-be’ environments of the enterprise, as well as a sequencing plan for transitioning from the ‘as-is’ to the ‘to-be’ ” and the Stage 5 elements “Return on EA investment is measured and reported” and “Organization head has approved current version of EA.” IRS could become a Stage 5 agency by satisfying the Stage 4 elements “Business, performance, information/data, application/service, and technology descriptions address security” and “EA products and management processes undergo independent verification and validation” and the Stage 5 element “Return on EA investment is measured and reported.” Table 7 shows the agencies that need to satisfy 5 or fewer elements to achieve Stage 5 under Version 1.1.

OMB Has Taken Action to Promote Enterprise Architecture, but Improvements Have Not Occurred and Management Challenges Remain

OMB has taken a number of steps to promote, standardize, and improve enterprise architecture use across the government. For example, OMB now requires agencies to submit enterprise architectures for review. It also leads various CIO Council initiatives to develop the FEA, including associated models, and to facilitate cross-agency efforts and major initiatives such as e-government. However, despite OMB’s actions, the same management challenges facing agencies 2 years ago have increased in prevalence, and agencies report mixed results from OMB’s efforts to address these challenges. The persistence of these challenges can be attributed, at least in part, to the office not implementing our prior recommendations aimed at addressing them and improving its enterprise architecture oversight.

OMB Has Taken Steps to Promote Agency Enterprise Architecture

OMB recognizes the importance of enterprise architectures and has supported their use since the passage of the Clinger-Cohen Act of 1996, with particular emphasis and attention in the last 2 years. For example, in collaboration with others and us, OMB issued guidance on the purpose and use of enterprise architectures shortly after passage of the act. It has also incorporated enterprise architecture considerations into its oversight processes and issued guidance directing that agency IT investments be based on agency IT enterprise architectures. More recently, it has launched efforts to promote the development and use of enterprise architectures through the budget process and various CIO Council initiatives.

As a means of promoting agencies’ enterprise architecture use, OMB has also included requirements for having and using enterprise architectures as part of the budget process, which began with the fiscal year 2002 budget cycle and, according to OMB officials, has continued through the current cycle (fiscal year 2005). More specifically: For the fiscal year 2002 budget cycle, OMB required agency budget submissions to provide investment plans in several areas, including enterprise architectures.

For fiscal year 2003, OMB required departments and major agencies that are CIO Council members to address how IT investment decision making incorporated architecture alignment and, for agencies that do not have architectures, to provide a plan for developing one. OMB also assessed the status of major department and agency architectures against the CIO Council’s Practical Guide for Federal Enterprise Architecture and reported the assessment results in the President’s fiscal year 2003 budget. However, this assessment covered only 23 of the 96 agencies included in this survey, and assessment results were not reported in a way to permit a clear understanding of the agencies’ enterprise architecture management status or to facilitate year-to-year progress determinations. For example, for the Environmental Protection Agency (EPA), the assessment resulted in the following report: “EPA has the fundamental elements of an EA documented.”

As part of the fiscal year 2004 budget cycle, OMB again assessed major department and agency architectures and reported the assessment results in the President’s fiscal year 2004 budget. However, the scope of the assessment again was not as comprehensive and meaningful as our survey results and covered only 22 of the 96 agencies included in this survey. For example, for the Department of Agriculture, OMB reported, “USDA’s EA is continuing to focus on the business, data, application, and technology layers of the EA. USDA is also working to integrate the EA efforts throughout the department.” Also for the fiscal year 2004 cycle, the office evaluated major IT investment business cases for consistency with agency architectures and with the FEA business reference model.

OMB has also worked through the CIO Council, which is co-chaired by OMB’s Deputy Director of Management, to improve enterprise architecture management and use. Specifically, the CIO Council established the Architecture and Infrastructure Committee to, for example, develop simpler and more consistent enterprise architecture terminology and facilitate cross-agency enterprise architecture efforts. This committee has three subcommittees that, since being chartered in October 2002, have organized, appointed leaders, established membership, and begun implementing plans. The name and objective of each subcommittee are provided below.

The Enterprise Architecture Governance Subcommittee was established to provide policy guidance and advice and assistance in the definition, design, and implementation of enterprise architecture discipline and practice throughout the federal government. It is expected to support the alignment of the FEA with agency enterprise architectures and to serve as the core federal group providing advocacy for enterprise architecture integration of business and technology architectures across state, local, and international boundaries.

The Emerging Technology Subcommittee was created to identify technologies with the potential to improve the value and quality of the FEA.

The Component Subcommittee is expected to foster the identification, maturation, use, and reuse of component-based architectures and architectural components in the federal government.

OMB Is Developing a Federal Enterprise Architecture

OMB’s development of the FEA is intended to facilitate governmentwide improvement through cross-agency analysis and the identification of duplicative investments, gaps, and opportunities for collaboration, interoperability, and integration within and across government agencies. According to OMB, the result will be a more citizen-centered, customer- focused government that maximizes technology investments to better achieve mission outcomes. As previously mentioned, the FEA will be composed of five reference models: Business reference model. The business reference model serves as the foundation for the FEA. It is intended to describe the federal government’s businesses, independent of the agencies that perform them. The model consists of four business areas: (1) services for citizens, (2) mode of delivery, (3) support delivery of services, and (4) management of government resources.

These four business areas are decomposed into 39 lines of business, which are made up of 153 subfunctions. Examples of lines of business under the services for citizens business area are homeland security, law enforcement, and economic development. Each of these lines of business includes a number of subfunctions. For example, for the homeland security line of business, a subfunction is border and transportation security; for law enforcement, a subfunction is citizen protection; and for economic development, a subfunction is financial sector oversight.

Version 1.0 of the model was released to agencies in July 2002 and was used in the fiscal year 2004 budget process. According to OMB, Version 1.0 of the model revealed that many federal agencies were involved in each line of business, and that agencies’ proposed fiscal year 2004 IT investments offered multibillion-dollar consolidation opportunities. In June 2003, Version 2.0 was released, which, according to OMB, reflects changes to align the model with other governmentwide management frameworks (e.g., budget function codes) and improvement initiatives (e.g., the President’s Budget Performance Integration Initiative) and addresses comments from agencies. OMB expects agencies to use the model, as part of their capital planning and investment control processes, to help identify opportunities to consolidate IT investments across the federal government.

Service component reference model. The service component reference model is intended to identify and classify IT service (i.e., application) components that support federal agencies and promote the reuse of components across agencies. The model is organized as a hierarchy, beginning with seven service domains, as shown in table 8.

These service domains are decomposed into 29 service types, which are further broken down into 168 components. For example, the customer services domain is made up of 3 service types: customer relationship management, customer preferences, and customer-initiated assistance. Components of the customer relationship management service type include call center management and customer analytics, components of the customer preferences service type include personalization and subscriptions, and components of the customer-initiated assistance service type include on-line help and on-line tutorials.

Version 1.0 of the service component reference model was released in June 2003. The model is intended to help agencies and OMB identify, among other things, agencies that are building or have already built similar service components that can be reused.

Technical reference model. The technical reference model is intended to describe the standards, specifications, and technologies that collectively support the secure delivery, exchange, and construction of service components. The model is made up of the following four core service areas: Service access and delivery: the collection of standards and specifications that support external access, exchange, and delivery of service components.

Service platform and infrastructure: the delivery platforms and infrastructure that support the construction, maintenance, and availability of a service component or capabilities.

Component framework: the underlying foundation, technologies, standards, and specifications by which service components are built, exchanged, and deployed.

Service interface and integration: the collection of technologies, methodologies, standards, and specifications that govern how agencies will interface internally and externally with a service component.

Each of these service areas is made up of service categories, which identify lower levels of technologies, standards, and specifications; service standards, which define the standards and technologies that support the service category; and the service specification, which details the standard specification or the provider of the specification. For example, within the first core service area (service access and delivery), an example of a service category is access channels, and service standards are Web browsers and wireless personal digital assistants. Examples of service specifications for the Web browser service standard are Internet Explorer and Netscape Navigator.

Version 1.0 of the technical reference model was released in January 2003, followed by Version 1.1, reflecting minor revisions that were based, in part, on agencies’ reviews, in August 2003. The model is intended to help agencies in defining their target technical architectures.

Performance reference model. The performance reference model is intended to describe a set of performance measures for the federal government (i.e., outcome and output measures for each line of business and subfunction identified in the business reference model). Thus, the model is expected to support the measurement of cross-agency initiatives. Version 1.0 of the model was released in September 2003.

Data and information reference model. The data and information reference model is intended to describe the type of data and information that support program and business line operations and the relationships among these types. Thus, the model is to help describe the types of interactions and information exchanges that occur between the government and its customers. OMB plans to release Version 1.0 of the model in October 2003.

For the fiscal year 2005 budget cycle, OMB officials told us that they will use the FEA performance, service component, and technical reference models to evaluate agencies’ major IT investments.

Agency responses to our survey indicated high levels of understanding and support for OMB’s FEA work. For example, about 80 percent of agencies responded that they understand the goals and objectives of the FEA (about 8 percent did not) and that they support those goals and objectives (about 6 percent did not), and about 72 percent of agencies responded that their agency’s architecture is traceable to the FEA (about 6 percent were not). Additionally, about 67 percent responded that they understand the approach to developing the FEA (about 13 percent did not), and about 63 percent stated that they support this approach (about 10 percent did not). About 61 percent of agencies responded that their enterprise architecture would change as a result of the FEA (about 8 percent would not). (See table 9.)

Prevalence of Long-standing Enterprise Architecture Management Challenges Is Growing

Despite OMB’s architecture-related activities, agencies continue to face the same management challenges that we identified 2 years ago—that is, obtaining top management support and commitment, overcoming parochialism, and having the requisite resources (financial and human capital) to get the job done. Moreover, the percentage of agencies identifying these management challenges has grown. For example, getting top management to understand the purpose, content, and value of architectures was seen as a challenge by about 50 percent of agencies—up from 39 percent in our last survey. As our framework recognizes, obtaining executive understanding and support is essential to having an effective enterprise architecture program. Without it, agencies will have increased difficulty in addressing other challenges, such as overcoming parochialism among organizational components and obtaining requisite resources (funding and human capital). Our survey results bear this out—at the same time that the percentage of agencies identifying top management understanding and support as a challenge rose, the percentage of agencies identifying these other challenges almost all rose. For example, the percentage that identified parochialism as a challenge grew from 39 to 47 percent. Also, while 50 percent of agencies continued to report funding as a significant challenge, the percentage of agencies that reported obtaining skilled staff as a challenge grew from 32 to 49 percent. (See table 10.)

Agencies also reported mixed levels of satisfaction with OMB’s efforts to address these management challenges. Specifically, just over half of agencies were satisfied with OMB’s efforts to foster top management understanding and to overcome agency component organization parochialism (58 and 53 percent, respectively). Moreover, fewer than half of agencies (40 percent) were satisfied with OMB’s actions to address their enterprise architecture funding and staffing challenges. (See table 11.)

OMB Has Yet to Implement Our Recommendations for Addressing Architecture Management Challenges and Improving Oversight

Our February 2002 report concluded that OMB needed to advance the level of enterprise architecture management maturity by exercising improved oversight and identifying governmentwide solutions to common enterprise architecture management challenges facing agencies. Specifically, we recommended that the OMB Director, in collaboration with the federal CIO Council, use the maturity framework and agency baseline information provided in our February 2002 report as the basis for helping agencies to advance the state of their respective enterprise architecture development, implementation, and maintenance efforts, and for measuring agency progress. We further recommended that in doing so, the director require each of the 116 agencies surveyed in our 2002 report to (1) submit to OMB an annual update of the agency’s satisfaction of each of the core elements contained in the maturity framework and (2) have this update verified by the agency’s inspector general or comparable audit function before it is submitted to OMB.

Additionally, we recommended in our 2002 report that the OMB Director, in collaboration with the CIO Council, develop and implement a plan to address the governmentwide impediments to greater agency use of enterprise architectures. We recommended that, at a minimum, this plan should include the two primary challenges identified in the 2002 report— that is, agency executive management understanding of enterprise architectures and the availability of enterprise architecture human capital expertise. Finally, we recommended that the director report annually to the Senate Committee on Governmental Affairs and the House Committee on Government Reform on the results of OMB’s annual update of the state and progress of federal agencies’ enterprise architecture efforts.

OMB officials generally agreed with the findings and conclusions of our 2002 report and stated that they would consider using our framework. However, after 18 months, office officials told us that they are still considering using our framework as a basis for evaluating agencies’ progress in developing and implementing their architectures, but had not committed to doing so because they were still reviewing the options for evaluating agencies’ progress in developing and implementing their enterprise architectures using our framework and other potential tools. Additionally, the office did not report any plans to address governmentwide impediments to greater agency use of architectures. Further, OMB reported that it has and plans to continue to provide information to the Congress on the state of agency enterprise architecture efforts and on progress in implementing the FEA.

Conclusions

Overall, the federal government’s state of enterprise architecture management remains less than satisfactory, with little progress being made over the last 2 years. As a result, most federal agencies continue to run the serious risk of investing in IT solutions that will not overcome, but rather will perpetuate, long-standing incompatibilities and duplication within agency operational and systems environments. OMB has taken steps to promote the development and use of enterprise architectures; however, these steps have yet to produce desired results. It is thus important for OMB to take additional actions, such as those that we have previously recommended and OMB has yet to implement. To do less risks continued exposure of agency IT investments to the unnecessary risk of being duplicative, incompatible, and needlessly costly.

Recommendations for Executive Action

We reiterate the recommendations we made in our February 2002 report on the governmentwide status of enterprise architecture use, with the modification that OMB use Version 1.1 of our framework and the baseline data from our 2003 survey included in this report, rather than Version 1.0 of our framework and our 2001 survey data.

Additionally, we recommend that the OMB Director, in developing and implementing the plan we previously recommended to address governmentwide impediments to greater agency use of enterprise architectures, ensure that the plan provides for identifying agencies that have effectively overcome enterprise architecture management challenges and sharing those and other lessons learned and best practices. Also, we recommend that the director, in annually reporting to the Senate Committee on Governmental Affairs and the House Committee on Government Reform, as we previously recommended, include in the report what steps have been taken to implement our recommendations, including reasons for not adopting our maturity framework.

Agency Comments

In oral comments on a draft of this report, officials from OMB’s Office of Information and Regulatory Affairs and the Federal Enterprise Architecture Program Management Office stated that they generally agreed with our findings and recommendations. They also stated that they agreed with the need for agency assessments using Version 1.1 of our framework, and that these assessments should be independently verified. They added that fully implementing our recommendations would require sustained management attention.

As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to interested congressional committees, the OMB Director, and agencies that participated in our survey. We will also provide copies to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. If you have any questions concerning this information, please contact me at (202) 512-3439 or by e-mail at hiter@gao.gov. Key contributors to this report are listed in appendix IX.

Agencies Indicated Architecture Benefits and Experiences with the Use of Tools, Frameworks, Methodologies, and Contractors

In response to our 2003 survey, agencies reported additional information related to the implementation of their enterprise architectures. This information includes architecture benefits and architecture tool, framework, methodology, and contractor experiences.

Agencies Reported Various Architecture Benefits

Office of Management and Budget (OMB) policy, Chief Information Officer (CIO) Council guidance, and our research and reviews of agencies’ management of information technology (IT) have identified multiple benefits of effectively using enterprise architectures, including avoiding duplication between IT systems, promoting integration of systems, reducing system-related costs, and optimizing agency mission performance. Agency responses to our 2001 survey affirmed these and offered additional benefits, such as lower system-related costs and benefits related to enhanced productivity and improved efficiency.

Agencies responding to our 2003 survey reported similar benefits. For example, benefits related to improved systems interoperability were cited by 53 percent of agencies, while improved organization and change management were cited by 51 percent of agencies. Also, enhanced productivity and lower system-related costs were cited by 41 percent and 39 percent, respectively. Table 12 shows the benefits that were most frequently identified by survey respondents. One new benefit cited by 56 percent of agencies was the use of “enterprise licenses.” Such licenses take advantage of the economies of scale associated with purchasing a large number of commercial product licenses.

Agencies Reported Using a Variety of Enterprise Architecture Tools with Varying Degrees of Satisfaction

An automated enterprise architecture tool serves as the repository of architecture artifacts, which are work products that are produced and used to capture and convey architectural information. An agency’s choice of tool should be based on a number of considerations, including agency needs and the size and complexity of the architecture.

Agencies reported using various automated tools to develop and maintain their enterprise architectures. The most commonly identified architecture products were Microsoft Office (72 agencies), System Architect (31 agencies), the Enterprise Architecture Management System (18 agencies), Rational Rose (17 agencies), Metis (11 agencies), and Framework (7 agencies). Forty-one agencies reported using “other” architecture products. Figure 17 indicates the proportion of agencies that indicated using each architecture tool.

Agencies reported different levels of satisfaction with the enterprise architecture tools they are using. As shown in table 13, about 68 percent of agencies using System Architect were satisfied, about 73 percent of agencies using Metis were satisfied, and about 61 percent of agencies using Microsoft’s Office Suite were satisfied. In contrast, about 17 percent of agencies using the EA Management System were satisfied (about 67 percent of agencies using EA Management System responded that it was too early to comment on satisfaction levels), and about 41 and 43 percent of agencies using Rational Rose and Framework, respectively, were satisfied. With respect to agencies’ dissatisfaction with their tools, about 3 percent of agencies using System Architect were dissatisfied, and about 13 percent of agencies using Microsoft’s Office Suite were dissatisfied. Also, about 11 percent of agencies using the EA Management System were dissatisfied, and about 12 and about 29 percent of agencies using Rational Rose and Framework, respectively, were dissatisfied with those tools. No agencies using Metis were dissatisfied.

Agencies Reported Using a Variety of Enterprise Architecture Frameworks with Varying Levels of Satisfaction

An enterprise architecture framework (or model) provides a formal structure for representing the architecture and serves as the basis for the nature and content of the specific products that the agency plans to develop, use, and maintain. As such, a framework helps to ensure the consistent representation of information from across the organization and supports orderly capture and maintenance of architecture content.

Agencies reported using various frameworks. The most frequently cited frameworks in our survey responses were the Federal Enterprise Architecture Framework (FEAF) (61 agencies), the Federal Enterprise Architecture Program Management Office (FEAPMO) Reference Models (56 agencies), and the Zachman Framework (36 agencies). Figure 18 indicates the proportion of agencies that indicated using each framework. Other frameworks used included the Treasury Enterprise Architecture Framework (TEAF); the National Institute of Standards and Technology Framework (NIST framework); the Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) Framework; and the Department of Defense Architecture Framework (DoDAF).

Agencies reported different levels of satisfaction with the enterprise architecture frameworks they are using, as shown in table 14. The levels of satisfaction ranged from 81 percent, reported by agencies using the Zachman Framework, to 45 percent, reported by agencies using the NIST framework. As table 14 shows, few agencies reported being dissatisfied out of 209 responses.

Less Than Half of Federal Agencies Had Selected an Enterprise Architecture Methodology

An enterprise architecture methodology provides a common set of procedures for developing architecture products and, if implemented properly, helps to ensure consistency in the procedures used across the organization for developing and maintaining the enterprise architecture. An organization’s methodology or methodologies should govern how the architecture products will be developed, maintained, and validated. Methodologies need to be documented, understood, and consistently applied. They should prescribe the standards, steps, tools, techniques, and measures to be used to provide reasonable assurance that expected product quality is attained.

Less than half (41 percent) of the federal agencies that we surveyed had selected a methodology. About 55 percent (23 of 42) of the methodologies that agencies reported using were Spewak’s enterprise architecture planning methodology or a variation. Four of the remaining 19 methodologies were developed by META Group, and 2 were developed by Gartner, Inc. Two agencies cited James Martin’s Information Strategy Planning, and 2 agencies cited the Department of Commerce’s Enterprise Architecture Methodology. The remaining 21 percent (9 of 42) were unique methodologies.

Most Agencies Reported Using Contractor Support for Developing Enterprise Architectures

Agencies reported heavy use of contractor support for developing their respective architectures. Most agencies (72 of 92 agencies that responded to this question—78 percent) stated that their architectures were developed in-house with contractor support. Ten agencies (11 percent) reported that contractors developed their enterprise architectures. Ten agencies (11 percent) reported that they developed their enterprise architectures in-house without any contractor support. Table 15 describes the level of contractor use, by agency type.

Cost Data Related to Implementing Enterprise Architectures Varied, with Personnel Accounting for Most Costs

Agency-reported data revealed a wide variance in the cost of developing, completing, and maintaining enterprise architectures. Agencies generally reported that their architecture development costs could be allocated to several categories, with the majority of costs attributable to agency and contractor personnel.

Departments’ Investment in Architecture Varied More Widely than Component and Independent Agencies’

As we have previously reported, the scope and nature of the enterprise and the extent of enterprise transformation and modernization envisioned will dictate the depth and detail of the architecture to be developed and maintained. Restated, the architecture should be tailored to the individual enterprise and that enterprise’s intended use of the architecture. Accordingly, the level of resources that an agency invests in its architecture is likely to vary. Agency responses to our survey showed this to be the case.

Agencies that reported cost data reported $599 million being spent to date on the development of architectures, with individual agency development costs to date ranging from $5,000 to $248 million. Departments’ architecture development costs varied more than component and independent agencies’ costs, while component agencies reported spending the most to date, with independent agencies spending the least. Agencies reported estimated costs to complete architecture development ranging from $3,000 to $319 million, and annual estimated maintenance costs ranging from $1,000 to $36 million. Figures 19 through 27 depict the variability of cost data reported by departments, component agencies, and independent agencies.

Agency and Contractor Personnel Accounted for the Majority of Architecture Development Costs

Of the $599 million reported in architecture development costs, agencies allocated $511 million to the following seven cost categories that we identified in our questionnaire: agency personnel, contractor personnel, tools, methodologies, independent validation and verification, training, and other. For those agencies that reported and allocated costs, the majority of these costs were for agency and contractor personnel—$116.7 million (23 percent) were attributed to agency personnel and $188.9 million (37 percent) were attributed to contractor personnel. About $193.3 million (38 percent) were attributed to “other” costs, $7.1 million (1 percent) to architecture tools, and $3.9 million (eight-tenths of 1 percent) to independent validation and verification contract personnel. Further, $1.0 million (two-tenths of 1 percent) of costs were attributed to methodologies and another $1.0 million (two-tenths of 1 percent) to training. Figure 28 shows the architecture development costs by category.

Table 16 shows enterprise architecture development, completion, and maintenance costs for each agency that provided cost data.

Objectives, Scope, and Methodology

Our objectives were to determine (1) what progress federal agencies have made in effectively developing, implementing, and maintaining their enterprise architectures and (2) the actions of the Office of Management and Budget (OMB) to advance the state of enterprise architecture development and use across the federal government.

To address our objectives, we obtained and reviewed relevant guidance on enterprise architectures, such as OMB Circular A-130 and guidance published by the federal Chief Information Officers (CIO) Council, including the Federal Enterprise Architecture Framework Version 1.1 and the Practical Guide. We also researched our past reports and guidance on the management and use of enterprise architectures, including the results of our 2001 governmentwide enterprise architecture survey and our enterprise architecture management maturity framework.

Next, we used the CIO Council’s Practical Guide and our enterprise architecture management maturity framework to develop two data collection instruments—one for federal departments and one for agencies that are either components within a department or are independent (see app. VIII). We pretested our survey instruments at one federal department and one component agency.

To ensure consistency and comparability with our 2001 governmentwide enterprise architecture survey, we based our survey population on the same 116 agencies, with appropriate additions and deletions. These agencies consisted of all cabinet-level departments, major component agencies within departments, and other independent agencies. We modified our 2001 survey population to reflect the federal government’s reorganization of March 1, 2003, in which the Department of Homeland Security (DHS) and its directorates (i.e., component agencies) became operational, resulting in the addition of 5 agencies. At the same time, the establishment of DHS resulted in 4 agencies that were included in our 2001 survey being eliminated from our survey population because they were absorbed into DHS directorates. We also eliminated the U.S. Marine Corps as a separate agency within our population so that the Department of the Navy, at its request, could provide a single response for the Navy and the Marine Corps. Table 17 lists additions to and deletions from our 2001 survey population and provides explanations for each change.

For each of the 116 agencies, we identified the CIO or comparable official and notified them of our work and distributed the appropriate survey instrument to designated officials via e-mail. We also discussed the purpose and content of the survey instrument with agency officials when requested. After receiving our survey, officials from DHS and the Departments of the Interior and Veterans Affairs told us that their respective architectures cover their component agencies and, thus, a single response would be provided. (When departments opted to provide a departmental response inclusive of component agencies, our analysis pertains to the department as a whole. Conversely, when departments and their component agencies reported separately, our departmental analysis is exclusive of component agencies.) Additionally, officials from the Department of Agriculture’s Farm Service Agency, Natural Resources Conservation Service, and Rural Utilities Service told us they would provide a response that reflects the Service Center Modernization Initiative, which encompasses those three component agencies. We agreed with these proposed approaches. Both the Department of Defense’s Business Enterprise Architecture and Agriculture’s previously mentioned Service Center Modernization Initiative provided responses that were not solicited in our survey population, which we included in our analysis and in this report. Tables 18 and 19 show the consolidated, omitted, and additional responses that led to the difference between our survey population of 116 agencies and the 96 respondents included in this report, including an explanation for each adjustment.

The timing of the 96 responses varied, ranging from April 1 to July 9, 2003, and thus the determinations in this report regarding the state of enterprise architecture development and use and progress at specific agencies and groups of agencies are linked to particular points in time. Appendixes V, VI, and VII, which contain the results of our analysis of each agency’s response to our survey, identify the date that each agency responded. To verify the accuracy of agencies’ responses to our survey regarding enterprise architecture management policies, organizations, and responsibilities, we required agencies to submit documentation or additional information for survey questions related to certain framework criteria. Specifically, we requested agencies to submit documentation or additional information for questions 6 to 11, 18, 20 to 24, 26, and 35 to 39. Although our survey requested that agencies provide data about the status of various enterprise architecture products, we did not independently verify the data that agencies provided about the comprehensiveness or completeness of their architecture products. Additionally, we contacted agency officials when necessary to clarify their responses.

To determine the progress of federal agencies’ enterprise architecture efforts, we analyzed agency survey responses using Version 1.0 of our maturity framework and compared them with the results of our 2001 survey, which were also based on Version 1.0. We also analyzed survey responses using Version 1.1 of our maturity framework to establish a new baseline against which future progress can be measured. When an agency’s response and our subsequent analysis indicated that it did not meet a core element as defined in the framework, we assigned that agency to the next lowest stage of framework maturity (i.e., to achieve a given stage of maturity, an agency must meet all core elements at that stage). For example, if an agency satisfied all Stage 2 and Stage 4 elements, but did not satisfy one Stage 3 element, that agency is considered to be a Stage 2 agency.

When determining agency maturity levels, we did not consider whether agency enterprise architecture plans or products included “performance” because explicitly including enterprise performance data is a relatively new concept, and there was a minimal amount of federal guidance related to enterprise performance data available to agencies at the time our surveys were distributed.

Tables 20 to 23 show the relationship between the survey questions and the framework elements for Version 1.0 of the framework, as well as identify where documentation was required to support answers.

Tables 24 to 27 show the relationship between the survey questions and the framework elements for Version 1.1 of the framework.

After compiling agency responses and determining agencies’ respective maturity stages, we analyzed responses across different slices of our respondent population to determine patterns and issues.

Finally, to determine OMB’s actions to oversee agency enterprise architecture management efforts, we analyzed relevant policy and budget guidance, obtained information about OMB’s roles in the CIO Council and efforts to develop and use the Federal Enterprise Architecture (FEA) (including OMB’s use of the FEA in the budget process), and interviewed OMB officials about ongoing and planned management actions. We also analyzed agency responses to survey questions regarding OMB’s enterprise architecture-related oversight and guidance.

We conducted our work in the Washington, D.C., metropolitan area, from September 2002 to November 2003, in accordance with generally accepted government auditing standards.

Maturity Stages of Departments and Agencies According to EA Maturity Framework, Versions 1.0 and 1.1

The following table presents three assessments of the maturity stage of each listed organization on the basis of the following: (1) responses to our 2001 survey evaluated against Version 1.0 of our framework, (2) responses to our 2003 survey evaluated against Version 1.0 of our framework, and (3) responses to our 2003 survey evaluated against Version 1.1 of our framework.

Detailed Comparison of Individual Department Responses against Our EA Management Maturity Framework

Department of Agriculture

The Department of Agriculture provided its 2001 survey responses on July 9, 2001, and its 2003 responses on May 12, 2003.

Department of Commerce

The Department of Commerce provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 18, 2003.

Department of Defense

Global Information Grid

The Department of Defense provided its 2001 survey responses on July 25, 2001, and its 2003 response for its Global Information Grid on June 5, 2003.

Business Enterprise Architecture

The Department of Defense provided its 2003 response for its Business Enterprise Architecture on May 30, 2003. The department did not provide a similar response to our 2001 survey.

Department of Education

The Department of Education provided its 2001 survey responses on July 23, 2001, and its 2003 responses on April 28, 2003.

Department of Energy

The Department of Energy provided its 2001 survey responses on June 28, 2001, and its 2003 responses on April 23, 2003.

Department of Health and Human Services

The Department of Health and Human Services provided its 2001 survey responses on August 14, 2001, and its 2003 responses on May 12, 2003.

Department of Homeland Security

The Department of Homeland Security was not involved in our 2001 survey because it was established on March 1, 2003. It provided its 2003 responses on June 10, 2003.

Department of Housing and Urban Development

The Department of Housing and Urban Development provided its 2001 survey responses on June 28, 2001, and its 2003 responses on April 21, 2003.

Department of the Interior

The Department of the Interior provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 21, 2003.

Department of Justice

The Department of Justice provided its 2001 survey responses on July 10, 2001, and its 2003 responses on May 20, 2003.

Department of Labor

The Department of Labor provided its 2001 survey responses on July 2, 2001, and its 2003 responses on April 17, 2003.

Department of State

The Department of State provided its 2001 survey responses on July 13, 2001, and its 2003 responses on May 12, 2003.

Department of Transportation

The Department of Transportation provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 24, 2003.

Department of the Treasury

The Department of the Treasury provided its 2001 survey responses on June 28, 2001, and its 2003 responses on April 26, 2003.

Department of Veterans Affairs

The Department of Veterans Affairs provided its 2001 survey responses on August 17, 2001, and its 2003 responses on April 21, 2003.

Detailed Comparison of Individual Component Responses against Our EA Management Maturity Framework

Department of Agriculture

Agricultural Marketing Service

The Agricultural Marketing Service provided its 2001 survey responses on July 9, 2001, and its 2003 responses on May 13, 2003.

Agricultural Research Service

The Agricultural Research Service provided its 2001 survey responses on July 13, 2001, and its 2003 responses on April 18, 2003.

Animal and Plant Health Inspection Service

The Animal and Plant Health Inspection Service provided its 2001 survey responses on June 26, 2001, and its 2003 responses on April 21, 2003.

Cooperative State Research, Education, and Extension Service

The Cooperative State Research, Education, and Extension Service provided its 2001 survey responses on July 9, 2001, and its 2003 responses on April 16, 2003.

Food and Nutrition Service

The Food and Nutrition Service provided its 2001 survey responses on July 17, 2001, and its 2003 responses on April 24, 2003.

Food Safety and Inspection Service

The Food Safety and Inspection Service provided its 2001 survey responses on July 9, 2001, and its 2003 responses on June 10, 2003.

July 12, 2001, and its 2003 responses on May 5, 2003.

Forest Service

The Forest Service provided its 2001 survey responses on August 3, 2001, and its 2003 responses on April 21, 2003.

Risk Management Agency

The Risk Management Agency provided its 2001 survey responses on July 27, 2001, and its 2003 responses on May 6, 2003.

Service Center Modernization Initiative

The Service Center Modernization Initiative provided its responses on May 16, 2003.

Department of Commerce

Bureau of the Census

The Bureau of the Census provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 21, 2003.

Economic Development Administration

The Economic Development Administration provided its 2001 survey responses on July 10, 2001, and its 2003 responses on April 28, 2003.

International Trade Administration

The International Trade Administration provided its 2001 survey responses on June 26, 2001, and its 2003 responses on April 29, 2003.

National Oceanic and Atmospheric Administration

The National Oceanic and Atmospheric Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 21, 2003.

U.S. Patent and Trademark Office

The U.S. Patent and Trademark Office provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 21, 2003.

Department of Defense

Ballistic Missile Defense Organization

The Ballistic Missile Defense Organization provided its 2001 survey responses on July 25, 2001, and its 2003 responses on June 10, 2003.

Defense Advanced Research Projects Agency

The Defense Advanced Research Projects Agency provided its 2001 survey responses on July 25, 2001, and its 2003 responses on April 7, 2003.

Defense Commissary Agency

The Defense Commissary Agency provided its 2001 survey responses on July 25, 2001, and its 2003 responses on June 9, 2003.

Defense Contract Audit Agency

The Defense Contract Audit Agency provided its 2001 survey responses on July 25, 2001, and its 2003 responses on May 30, 2003.

Defense Contract Management Agency

The Defense Contract Management Agency provided its 2001 survey responses on July 3, 2001, and its 2003 responses on May 30, 2003.

Defense Information Systems Agency

The Defense Information Systems Agency provided its 2001 survey responses on July 11, 2001, and its 2003 responses on June 10, 2003.

July 25, 2001, and its 2003 responses on June 20, 2003.

Defense Logistics Agency

The Defense Logistics Agency provided its 2001 survey responses on July 25, 2001, and its 2003 responses on May 22, 2003.

Defense Security Cooperation Agency

The Defense Security Cooperation Agency provided its 2001 survey responses on July 25, 2001, and its 2003 responses on June 19, 2003.

Defense Security Service

The Defense Security Service provided its 2001 survey responses on July 25, 2001, and its 2003 responses on June 9, 2003.

Defense Threat Reduction Agency

The Defense Threat Reduction Agency provided its 2001 survey responses on July 25, 2001, and its 2003 responses on May 29, 2003.

July 27, 2001, and its 2003 responses on June 2, 2003.

Department of the Army

The Department of the Army provided its 2001 survey responses on July 25, 2001, and its 2003 responses on June 2, 2003.

Department of the Navy

The Department of the Navy provided its 2001 survey responses on July 25, 2001, and its 2003 responses on June 9, 2003.

National Imagery and Mapping Agency

The National Imagery and Mapping Agency provided its 2001 survey responses on July 25, 2001, and its 2003 responses on June 6, 2003.

Department of Health and Human Services

Administration for Children and Families

The Administration for Children and Families provided its 2001 survey responses on June 29, 2001, and its 2003 responses on May 12, 2003.

Agency for Healthcare Research and Quality

The Agency for Healthcare Research and Quality provided its 2001 survey responses on July 12, 2001, and its 2003 responses on May 12, 2003.

Centers for Disease Control and Prevention

The Centers for Disease Control and Prevention provided its 2001 survey responses on July 23, 2001, and its 2003 responses on May 12, 2003.

Centers for Medicare and Medicaid Services

The Centers for Medicare and Medicaid Services provided its 2001 survey responses on June 29, 2001, and its 2003 responses on May 12, 2003.

Food and Drug Administration

The Food and Drug Administration provided its 2001 survey responses on July 13, 2001, and its 2003 responses on May 12, 2003.

Health Resources and Services Administration

The Health Resources and Services Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on May 12, 2003.

Indian Health Service

The Indian Health Service provided its 2001 survey responses on June 29, 2001, and its 2003 responses on May 12, 2003.

Program Support Center

The Program Support Center provided its 2001 survey responses on June 29, 2001, and its 2003 responses on May 12, 2003.

Department of Justice

Bureau of Alcohol, Tobacco, Firearms and Explosives

The Bureau of Alcohol, Tobacco, Firearms and Explosives provided its 2001 survey responses on July 16, 2001, and its 2003 responses on April 21, 2003.

Drug Enforcement Administration

The Drug Enforcement Administration provided its 2001 survey responses on July 18, 2001, and its 2003 responses on May 20, 2003.

Federal Bureau of Investigation

The Federal Bureau of Investigation provided its 2001 survey responses on July 18, 2001, and its 2003 responses on May 28, 2003.

Federal Bureau of Prisons

The Federal Bureau of Prisons provided its 2001 survey responses on July 18, 2001, and its 2003 responses on May 22, 2003.

U.S. Marshals Service

The U.S. Marshals Service provided its 2001 survey responses on June 29, 2001, and its 2003 responses on May 19, 2003.

Department of Transportation

Federal Aviation Administration

The Federal Aviation Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 29, 2003.

Federal Highway Administration

The Federal Highway Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 1, 2003.

Federal Motor Carrier Safety Administration

The Federal Motor Carrier Safety Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 24, 2003.

Federal Railroad Administration

The Federal Railroad Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 24, 2003.

Federal Transit Administration

The Federal Transit Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 24, 2003.

National Highway Traffic Safety Administration

The National Highway Traffic Safety Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 24, 2003.

Department of the Treasury

Bureau of Engraving and Printing

The Bureau of Engraving and Printing provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 21, 2003.

Bureau of the Public Debt

The Bureau of the Public Debt provided its 2001 survey responses on July 5, 2001, and its 2003 responses on April 21, 2003.

June 28, 2001, and its 2003 responses on April 16, 2003.

Financial Management Service

The Financial Management Service provided its 2001 survey responses on June 28, 2001, and its 2003 responses on May 19, 2003.

Internal Revenue Service

The Internal Revenue Service provided its 2001 survey responses on July 20, 2001, and its 2003 responses on April 21, 2003.

Office of Thrift Supervision

The Office of Thrift Supervision provided its 2001 survey responses on June 29, 2001, and its 2003 responses on June 9, 2003.

U.S. Mint

The U.S. Mint provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 21, 2003.

Detailed Comparison of Independent Agency Responses against Our EA Management Maturity Framework

Agency for International Development

The Agency for International Development provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 22, 2003.

Central Intelligence Agency

The Central Intelligence Agency provided its 2001 survey responses on August 6, 2001, and its 2003 responses on May 30, 2003.

Corporation for National and Community Service

The Corporation for National and Community Service provided its 2001 survey responses on July 20, 2001, and its 2003 responses on April 22, 2003.

Environmental Protection Agency

The Environmental Protection Agency provided its 2001 survey responses on June 28, 2001, and its 2003 responses on May 15, 2003.

Equal Employment Opportunity Commission

The Equal Employment Opportunity Commission provided its 2001 survey responses on August 1, 2001, and its 2003 responses on May 2, 2003.

Executive Office of the President

The Executive Office of the President provided its 2001 survey responses on October 1, 2001, and its 2003 responses on June 6, 2003.

Export-Import Bank

The Export-Import Bank provided its 2001 survey responses on September 20, 2001, and its 2003 responses on June 11, 2003.

Federal Deposit Insurance Corporation

The Federal Deposit Insurance Corporation provided its 2001 survey responses on July 20, 2001, and its 2003 responses on April 18, 2003.

Federal Energy Regulatory Commission

The Federal Energy Regulatory Commission provided its 2001 survey responses on August 27, 2001, and its 2003 responses on May 12, 2003.

Federal Reserve System

The Federal Reserve System provided its 2001 survey responses on August 23, 2001, and its 2003 responses on April 23, 2003.

Federal Retirement Thrift Investment Board

The Federal Retirement Thrift Investment Board provided its 2001 survey responses on July 20, 2001, and its 2003 responses on July 9, 2003.

General Services Administration

The General Services Administration provided its 2001 survey responses on July 2, 2001, and its 2003 responses on April 23, 2003.

National Aeronautics and Space Administration

The National Aeronautics and Space Administration provided its 2001 survey responses on July 25, 2001, and its 2003 responses on April 21, 2003.

National Credit Union Administration

The National Credit Union Administration provided its 2001 survey responses on July 18, 2001, and its 2003 responses on April 10, 2003.

National Labor Relations Board

The National Labor Relations Board provided its 2001 survey responses on August 9, 2001, and its 2003 responses on June 9, 2003.

Nuclear Regulatory Commission

The Nuclear Regulatory Commission provided its 2001 survey responses on July 23, 2001, and its 2003 responses on April 21, 2003.

Office of Personnel Management

The Office of Personnel Management provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 28, 2003.

Peace Corps

The Peace Corps provided its 2001 survey responses on July 20, 2001, and its 2003 responses on May 15, 2003.

Railroad Retirement Board

The Railroad Retirement Board provided its 2001 survey responses on July 11, 2001, and its 2003 responses on April 18, 2003.

Securities and Exchange Commission

The Securities and Exchange Commission provided its 2001 survey responses on July 19, 2001, and its 2003 responses on April 22, 2003.

Small Business Administration

The Small Business Administration provided its 2001 survey responses on June 29, 2001, and its 2003 responses on April 22, 2003. 2001, and its 2003 responses on April 21, 2003.

Social Security Administration

The Social Security Administration provided its 2001 survey responses on July 3, 2001, and its 2003 responses on April 21, 2003.

U.S. Postal Service

The U.S. Postal Service provided its 2001 survey responses on August 13, 2001, and its 2003 responses on April 21, 2003.

2003 Survey of Enterprise Architecture Efforts

To assess agency enterprise architecture management maturity levels, we developed two similar surveys, one addressed to departments and the other to component and independent agencies. These two surveys were largely identical, with the following differences: Throughout, questions referred to “departments” in the department survey and to “agencies” in the agency survey.

Two questions on the department survey (questions 39 and 40) and three questions on the agency survey (questions 39 to 41) were addressed specifically to departments and agencies, respectively.

The last five questions on the two surveys were numbered differently, since they followed the department- and agency-specific questions described above. Questions 41 to 45 on the department survey were numbered 42 to 46 on the agency survey. (Note, however, that these five questions were not used in the decision criteria described in app. III.)

The following reproduced survey combines the two surveys into one display by using the phrase “agency/department” in places where one or the other term had been used in the separate surveys. It also displays both the two department questions and the three agency questions that were addressed specifically as described above.

We are also asking that you provide the name and government, GAO is conducting a survey of federal telephone number of a contact for your departments’ and agencies’ enterprise architecture (EA) agency/department who can answer any questions we efforts to gauge progress towards meeting Clinger-Cohen may have about your survey responses.

Act and OMB requirements and to identify successes that can be shared with other federal agencies. There are two versions of this survey. One version is being sent to federal agencies and a different version is being sent to cabinet-level departments.

Enterprise architectures are well defined and enforced blueprints (i.e., descriptions) for operational and technological change. Such architectures provide a clear ( ) and comprehensive picture of an entity, whether it is an organization (e.g., federal department, agency, or bureau) ( )

If you have any questions, please contact: environment; and (3) a capital investment roadmap for transitioning from the current to the target environment (i.e. sequencing plan).

We are requesting departments and agencies to provide information from readily available data. We are not asking that extensive analyses be performed in order to respond to these questions. Please complete this survey and return it to GAO no later than April 21, 2003.

You may return your completed survey and any supporting materials by E-mail, fax, or Federal Express.

If you return your survey by E-mail, the address is: pettisb@gao.gov.

If you return your survey by fax, the fax number is: (202) 512-6450 - Attn: Scott Pettis.

If you return your survey by Federal Express, the address is: Scott Pettis, Senior IT Analyst, 441 G St. NW, Rm. 4Y12, Washington, DC 20548. 1. Which of the following best describes your agency/department’s status with respect to enterprise architecture? (Check one box.) 1. We have developed an enterprise architecture Skip to question 3. 2. We do not have an enterprise architecture, but are in the process of developing one Skip to question 3. 3. We do not have an enterprise architecture, but plan to develop one Skip to question 3. 4. We do not plan to develop an enterprise architecture Answer question 2. 2. Please explain why your agency/department does not plan to develop an enterprise architecture. (Enter your response in the box below.)

If you were directed to answer question 2, you have completed the survey.

Please return it as soon as possible. Thank you.

YOU SHOULD ANSWER THE FOLLOWING QUESTIONS IF YOUR AGENCY/DEPARTMENT HAS AN ENTERPRISE ARCHITECTURE, IS IN THE PROCESS OF DEVELOPING ONE, OR PLANS TO DEVELOP ONE. 3. Which of the following best describes the scope of your agency/department’s completed, in-process, or planned enterprise architecture(s). (Check all that apply and provide additional information if necessary.) 1. Agency/department wide, organization based (i.e., all mission and business functions) 2. Agency/department wide, function based (e.g., financial management, logistics management, grant management, etc.) 3. Non-agency/department wide organization based 4. Non-agency/department wide function based If you checked box 3 or 4 above because your architecture is not agency/department wide, please list the organizations or functions covered by your enterprise architecture, and explain the basis for the defined scope. (Enter your response in the box below.) 4. Does (or will) this particular enterprise architecture include the following? (Check one box for each row.) (1) (2) (3)

An explicit discussion of security in the “as is” business operations,

performance measurement, information/data, services/applications, and technology descriptions of the agency/department.

A description of the agency/department’s future or “to-be” environment, An explicit discussion of security in the “to be” business operations, performance measurement, information/data, services/applications, and technology descriptions of the agency/department.

A description of the sequencing plan for moving from the “as is” to the “to be” environment.

If you answered “No” to any of the items in question 4, please explain why. (Enter your response in the box below.)

Is your agency/department’s enterprise architecture published? (Check one box and provide additional information if necessary.) 1. Yes Please provide a list naming each enterprise architecture product/artifact with a brief description of each product/artifact. 7. Does your agency/department have a written and approved policy for the development, maintenance, and use of enterprise architecture? (Check one box for each row. If policy is written but not approved, please check “No”.) (1) (2)

Development of the enterprise architecture

Maintenance of the enterprise architecture Use of the enterprise architecture If you checked “yes” for development, maintenance, or use, please provide a copy of the written and approved 8. Has your agency/department established committees or groups that represent the agency/department and have responsibility for the following? (Check one box for each row.) (1) (2)

Direction of the enterprise architecture

Oversight of the enterprise architecture Approval authority for the enterprise architecture Other aspects of the enterprise architecture (Describe)

If yes, please provide a copy of the charter or comparable documentation. 9. Has your agency/department established an official program office with responsibility for the following? (Check one box for each row.) (1) (2)

Development of the enterprise architecture

Maintenance of the enterprise architecture If yes, please provide a copy of the charter or comparable documentation. 10. Does your agency/department have an individual designated as the chief architect? (Check one box and provide additional information if necessary.) 1. Yes Please provide this individual’s name and phone number: ( )

Does this individual report to the chief information officer? 2. No What position does the chief architect report to?

2. No Skip to question 12. 11. Is your agency/department’s chief architect responsible for each of the following? (Check one box for each row.) (1) (2)

Heading the enterprise architecture program office

Directing development of the enterprise architecture Directing maintenance of the enterprise architecture Please provide a position description or comparable document describing the chief architect’s responsibilities. 12. Please provide the costs of developing and maintaining your enterprise architecture by the following major cost elements: (If you are in the process of developing your Enterprise Architecture, please enter data in all three columns.) if any, to Complete (to date)

Other (describe) $ . Please quantify your agency/department’s requested and approved enterprise architecture resources.

Personnel (FTEs)

If any gap exists between requested and approved resources for Fiscal Year 2001, 2002, or 2003, please answer question 14. Otherwise, proceed to question 15. 14. How much of an impact, if any, has the gap between enterprise architecture resources requested and resources finally approved had on your agency/department’s enterprise architecture program? (Check one and provide additional information if necessary.) 1. Very adverse impact 2. Somewhat adverse impact 3. Moderate adverse impact 4. Slight adverse impact 5. No adverse impact Please provide any additional details about the impact of any gap noted above. (Enter your response in the box below.) 15. Which of the following automated tools are being used for this enterprise architecture? For each tool being used, how satisfied or dissatisfied are you with it? (Check yes or no in each row. If yes, check additional box.)

If tool is being used, are you . . . used? (1) (2) (3) (4) (5) (6)

Enterprise Architecture

(EAMS)

Framework by Ptech Inc.

JCAPS by Logicon Inc.

Yes 16. Which of the following model(s) or framework(s) (i.e., a formal structure for representing the enterprise architecture) is your agency/department using to develop this enterprise architecture? For each model or framework being used, how satisfied or dissatisfied are you with it? (Check yes or no in each row. If yes, check additional box.)

If model or framework is being used, are you . . . being used? (1) (2) (3) (4) (5) (6)

Command, Control,

(C4ISR)

Framework (DoDAF)

Framework (FEAF) (NIST)

Framework (TEAF)

Yes 17. Which of the following best describes how your agency/department’s enterprise architecture was or is being developed? (Check one box and provide additional information if necessary.) 1. Developed in-house using contractor(s) support 2. Developed in-house without any contractor(s) support 3. Developed by contractor(s) Please provide the contractor’s name(s): 18. Is your agency/department using an enterprise architecture development methodology or methodologies (i.e., a common set of procedures, such as Spewak’s Enterprise Architecture Planning methodology, for developing enterprise architecture products)? (Check one box and provide additional information if necessary.) 1. Yes Provide the following information about the enterprise architecture methodology or methodologies your agency/department is using: 19. To what extent was or is your agency/department’s “business” side involved in developing the enterprise architecture? (Check one.) 1. Very great extent 4. Some or little extent 5. No extent 20. Was the current version (i.e., latest major release) of your agency/department’s enterprise architecture submitted and approved by the following entities: (Check one box in each row under submitted and approved. If the enterprise architecture was submitted but not approved, please check “No”. If no, indicate whether action is planned.)

If no, is action planned? information officer? committee?

Approved by a committee or enterprise? investment review board?

Approved by the head of your agency/department?

Approved by other official or committee? Please specify: Submitted to OMB?

Please provide documentation for each approval indicated above. 21. Do your agency/department’s enterprise architecture products undergo independent verification and validation (IV&V)? (Check one box and provide additional information if necessary.) 1. Yes If IV&V is contractor-provided, please provide a copy of the contractor’s statement of work. 22. Do your agency/department’s enterprise architecture management processes undergo independent verification and validation (IV&V)? (Check one box and provide additional information if necessary.) 1. Yes If IV&V is contractor-provided, please provide a copy of the contractor’s statement of work. 2. No 23. Does your agency/department periodically update its enterprise architecture products? (Check one box and provide additional information if necessary.) 1. Yes If yes, please provide Date of last update: 24. Is your agency/department’s enterprise architecture under configuration management (i.e., a process for establishing and maintaining the integrity of work products)? (Check one box and provide additional information if necessary.) 1. Yes If yes, please provide Date of current version: 25. Does a process exist for formally managing changes to your agency/department’s enterprise architecture? (Check 26. Does your agency/department have a written and approved policy that requires that IT investments comply with the enterprise architecture? (Check one box and provide additional information if necessary. If policy is written but not approved, please check “No”.) 1. Yes Please provide a copy of the written policy. Continue with question 27 2. No Skip to question 28 27. Does your agency/department permit waivers to its requirement that IT investments comply with the enterprise architecture? (Check one.) 1. Yes, only if the request provides a written justification 2. Yes, a waiver can be granted based on an informal request 3. No, the agency/department does not provide for waivers to this policy 28. Is your agency/department’s enterprise architecture an integral component of your agency/department’s IT investment management process? (Check one.) 29. To what extent does your agency/department’s IT investments comply with the enterprise architecture? (Check one.) 1. Very great extent 4. Some or little extent 5. No extent 30. Was your agency/department’s decision to develop an enterprise architecture based on: 1) a business case that provided economic justification (i.e., benefits in excess of costs); 2) the need to comply with the Clinger-Cohen Act and/or OMB requirements; 3) the need to respond to the President’s Management Agenda; and/or, 4) some other factor(s) that was considered? (Check all that apply.) 1. A business case that anticipated a positive return 2. The need to comply with Clinger-Cohen and/or OMB requirements 3. The need to respond to the President’s Management Agenda 4. Other factor(s) - Please specify in the box below: 31. What benefits, if any, can be attributed to your agency/department’s use of an enterprise architecture? If the benefit can be attributed to the use of an enterprise architecture, to what extent, if at all, has the benefit been attained thus (Check yes or no in each row. If yes, indicate extent benefit attained.)

Architecture? (1) (2) (3) (4) (5) (6)

Other (describe)

Yes No 32. To what extent, if at all, did the following challenges affect the development of your agency/department’s enterprise architecture? (Check one box in each row.) (1) (2) (3) (4) (5)

Top management’s understanding of the importance

From this point, the agency and department surveys differ.

federal department such as the Department of the

Treasury or USDA? (Check one.)

bureaus) enterprise architecture development, maintenance, or use? (Check one box and provide 1. Yes Continue with question 40 additional information if necessary.)

2. No Skip to question 42 1. Yes Please provide a copy of the policy or guidance with your response.

40. To what extent, if at all, has your agency’s

department provided oversight of your enterprise architecture efforts? (Check one.)

1. Very great extent

(e.g., oversight and approval processes)?

2. Great extent

a.

4. Some or little extent

b. c.

41. Was your agency’s enterprise architecture approved

by your department’s chief information officer? (Check one.)

From this point, the questions are again the same for each survey, except for their numbering: on the department survey, each question number was one less than the numbering shown in the following (the numbering shown corresponds to that on the agency survey).

Note that none of the questions that follow were used in the decision criteria that determined the maturity stage assigned to any respondent (see appendix III for these criteria).

42. Overall, how satisfied or dissatisfied is your agency/department with OMB’s direction and guidance to your agency/department regarding development, maintenance, and implementation of your enterprise architecture? (Question responses will be aggregated and not directly attributable to any agency/department.) (Check one and provide additional information if necessary.) 3. Neither satisfied nor dissatisfied If you indicated that your agency/department is other than “Very satisfied” or “Satisfied,” please describe why and what improvements are needed. 43. How satisfied is your agency/department with OMB’s efforts to address the following enterprise architecture management challenges GAO reported in its February 2002 report (GAO-02-6)? (Question responses will be aggregated and not directly attributable to any agency/department.) (Check one box in each row and provide additional information if necessary.) (1) (2) (3) (4) (5)

Fostering top management understanding

If you indicated that your agency/department is other than “Very satisfied” or “Satisfied,” to any of the above, please describe why and what improvements are needed. 44. Do you agree or disagree with the following statements as they apply to OMB’s Federal Enterprise Architecture (FEA)? (Question responses will be aggregated and not directly attributable to any agency/department.) (Check one box in each row.) (1) (2) (3) (4) (5)

My agency/department’s enterprise architecture will

change as a result of the FEA If you indicated other than “Strongly agree” or “Agree,” to any of the above, please describe why and what improvements are needed. 45. In your agency/department’s opinion, what impact has the FEA had (or will the FEA have) on your agency/department’s enterprise architecture? (Question responses will be aggregated and not directly attributable to any agency/department.) (Check one.) 1. Very positive impact 2. Generally positive impact 3. Neither positive nor negative impact 4. Generally negative impact 5. Very negative impact 6. No basis to judge 46. Please provide any additional comments on your agency/department’s enterprise architecture program in the box Thank you for your assistance.

Please return your survey and any requested supporting materials to the E-mail address or fax number indicated on page 1.

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the person named above, Barbara S. Collier, William B. Cook, Neal J. Doherty, Michael Holland, Catherine M. Hurley, Stuart M. Kaufman, Scott Pettis, and David B. Shumate made key contributions to this report.

GAO’s Mission

The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs