Customs Service Modernization: Architecture Must Be Complete and Enforced to Effectively Build and Maintain Systems
Published: 1998-05-05T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the Customs Service's enterprise information systems architecture, focusing on determining whether: (1) the architecture is complete; and (2) Customs has processes and procedures to enforce compliance with the architecture.
section_title: What GAO Found
paragraphs: GAO noted that: (1) Customs does not yet have a complete enterprise information systems architecture to guide and constrain the millions of dollars it spends annually to develop and acquire new information systems and evolve existing ones; (2) for five of its six business areas Custom's architecture does not: (a) describe all the agency's business functions; (b) define the information needed to perform the functions; and (c) completely identify the users and locations of the functions; (3) while the architecture and related documentation describe business functions, and users and work locations for the sixth business area, they do not identify all the information needs and flows for all the trade functions; (4) also, Customs has named certain technical standards, products, and services that it will use in building systems to support all its business areas; (5) however, Customs has not chosen these based on a complete description of its business needs; (6) the limitations in Customs' architecture are rooted in its decision to focus on defining the technical characteristics of its systems environment; (7) Customs' view does not include the logical characteristics of its enterprise system environment, which would enable it to define and implement systems that optimally support the agency's mission needs; (8) Customs plans to develop the architecture in accordance with Department of the Treasury architectural guidance; (9) specifically, Customs plans to define its functional, information, and work needs and their interrelationships across its six business areas and, in light of these needs and interrelationships, reevaluate the technical characteristics it has selected for its systems environment; (10) until Customs defines the logical characteristics of its business environment and uses them to establish technical standards and approaches, it does not have adequate assurance that the systems it plans to build and operationally deploy will effectively support the agency's business needs; (11) Customs also has not developed and implemented effective procedures to enforce its architecture once it is completed; (12) Customs officials stated that a newly established investment management process will be used to enforce architectural compliance; (13) this process, however, does not require that system investments be architecturally compliant or that architectural deviations be justified and documented; and (14) as a result, Customs risks incurring the same problems as other federal agencies that have not effectively defined and enforced an architecture.

section_title: Letter
paragraphs: To build effective and efficient systems that support an agency’s mission, a systems blueprint—commonly called a systems architecture—must be used to guide and constrain the systems’ development and evolution. An effective systems architecture should be derived by systematically and thoroughly analyzing and defining the agency’s target operating environment—including its business functions, information needs and flows across functions, and system characteristics required to optimally support these information needs and flows. Furthermore, once an architecture is complete, it must be rigorously enforced to preclude suboptimizing an agency’s enormous investment in information technology.
This letter responds to your request that we review the Customs Service’s enterprise information systems architecture. As agreed with your offices, our objectives were to determine whether (1) the architecture is complete and (2) Customs has processes and procedures to enforce compliance with the architecture. To do so, we compared Customs’ architecture and associated processes to applicable Department of Treasury and related federal architecture requirements. We performed our work at Customs’ headquarters in Washington, D.C., and the Newington Data Center in Newington, Virginia, from July 1997 through January 1998 in accordance with generally accepted government auditing standards. Details of our scope and methodology are contained in appendix I.
We requested comments on a draft of this report from the Acting Commissioner of Customs or his designee. Customs provided comments that are discussed in the “Agency Comments and Our Evaluation” section and are reprinted in appendix II.
subsections: section_title: Background
paragraphs: The mission of the Customs Service is to ensure that all goods and persons entering and exiting the United States do so in compliance with all U.S. laws and regulations. It does this by (1) enforcing the laws governing the flow of goods and persons across the borders of the United States and (2) assessing and collecting duties, taxes, and fees on imported merchandise. During fiscal year 1997, Customs collected $22.1 billion in revenue at more than 300 ports of entry, and it processed nearly 450 million passengers who entered the United States during the year.
To accomplish its mission, Customs is organized into six business areas—trade compliance, outbound, passenger, finance, human resources, and investigations. Each business area is described below.
The trade compliance business area includes enforcement of laws and regulations associated with the importation of goods into the United States. To enforce compliance with the trade laws and regulations, Customs (1) works with the trade community to promote understanding of applicable laws and regulations, (2) selectively examines cargo to ensure that only eligible goods enter the country, (3) reviews documentation associated with cargo entries to ensure that it is properly valued and classified, (4) collects billions of dollars annually in duties, taxes, and fees associated with imported cargo, (5) assesses fines and penalties for noncompliance with trade laws and regulation, and (6) manages the collection of these moneys to ensure that all trade-related debts due to Customs are paid and properly accounted for.
The outbound business area includes Customs operations related to the enforcement of laws and regulations associated with the movement of merchandise and conveyances from the United States. To enforce compliance with these laws and regulations, Customs (1) selectively inspects cargo at U.S. ports to guard against the exportation of illegal goods, such as protected technologies, stolen vehicles, and illegal currency, (2) collects, disseminates, and uses intelligence to identify high-risk cargo and passengers, (3) seizes and accounts for illegal cargo, (4) assesses and collects fines and penalties associated with the exportation of illegal cargo, and (5) physically examines baggage and cargo at airport facilities for explosive and nuclear materials. In addition, the outbound business includes collecting and disseminating trade data within the federal government. Accurate trade data are crucial to establishing accurate trade statistics on which to base trade policy decisions and negotiate trade agreements with other countries. By the year 2000, Customs estimates that exports will be valued at $1.2 trillion, as compared to $696 million in 1994.
The passenger business area includes processing all passengers and crew of arriving and departing (1) air and sea conveyances and (2) noncommercial land vehicles and pedestrians. In fiscal year 1997, Customs processed nearly 450 million travelers, and by the year 2000, expects almost 500 million passengers to arrive in the United States annually. Many of Customs’ passenger activities focus on illegal immigration and drug smuggling and are coordinated with other federal agencies, such as the Immigration and Naturalization Service and the Department of Agriculture’s Animal and Plant Health Inspection Service. Activities include targeting high-risk passengers, which requires timely and accurate information, and physically inspecting selected passengers, baggage, and vehicles to determine compliance with laws and regulations.
The finance business area includes asset and revenue management activities. Asset management consists of activities to formulate Customs’ budget; properly allocate and distribute funds; and acquire, manage, and account for personnel, goods, and services. Revenue management encompasses all Customs activities to identify and establish amounts owed Customs, collect these amounts, and accurately report the status of revenue from all sources. Sources of revenue include duties, fees, taxes, other user fees, and forfeited currency and property. The revenue management activities interrelate closely with the revenue collection activities in the trade compliance, outbound, and passenger business areas.
The human resources business area is responsible for filling positions, providing employee benefits and services, training employees, facilitating workforce effectiveness, and processing personnel actions for Customs’ 18,000 employees and managers.
The investigations business area includes activities to detect and eliminate narcotics and money laundering operations. Customs works with other agencies and foreign governments to reduce drug-related activity by interdicting (seizing and destroying) narcotics, investigating organizations involved in drug smuggling, and deterring smuggling efforts through various other methods. Customs also develops and provides information to the trade and carrier communities to assist them in their efforts to prevent smuggling organizations from using cargo containers and commercial conveyances to introduce narcotics into the United States.
To carry out its responsibilities, Customs relies on information systems and processes to assist its staff in (1) documenting, inspecting, and accounting for the movement and disposition of imported goods and (2) collecting and accounting for the related revenues. Customs’ Office of Information and Technology (OIT) fiscal year 1998 budget is about $147 million for information management and technology activities. Customs expects its reliance on information systems to increase as a result of its burgeoning workload. For 1995 through 2001, Customs estimates that the annual volume of import trade between the United States and other countries will increase from $761 billion to $1.1 trillion. This will result in Customs processing an estimated increase of 7.5 million commercial entries— from 13.1 million to 20.6 million annually—during the same period. Recent trade agreements, such as the North American Free Trade Agreement (NAFTA), have also increased the number and complexity of trade provisions that Customs must enforce.
Customs recognizes that its ability to process the growing volume of imports while improving compliance with trade laws depends heavily on successfully modernizing its trade compliance process and its supporting automated systems. To speed the processing of imports and improve compliance with trade laws, the Congress enacted legislation that eliminated certain legislatively mandated paper requirements and required Customs to establish the National Customs Automation Program (NCAP).
The legislation also specified certain functions that NCAP must provide, including giving members of the trade community the capability to electronically file import entries at remote locations and enabling Customs to electronically process “drawback” claims. In response to the legislation, Customs began in 1994 to reorganize the agency, streamline operations, and modernize the information systems that support operations.
subsections: section_title: Information Systems Architecture: A Brief Description
paragraphs: As computer-based systems have become larger and more complex over the last decade, the importance of and reliance on information systems architectures have grown steadily. These comprehensive “construction plans” systematically detail the full breadth and depth of an organization’s mission-based “modus operandi” in (1) logical terms, such as defining business functions and providing high-level descriptions of information systems and their interrelationships, and (2) technical terms, such as specifying hardware, software, data, communications, security, and performance characteristics. Without an architecture to guide and constrain a modernization program, there is no systematic way to preclude either inconsistent system design and development decisions or the resulting suboptimal performance and added cost associated with incompatible systems.
The Congress and the Office of Management and Budget (OMB) have recognized the importance of agency information systems architectures. The 1996 Clinger-Cohen Act, for example, requires Chief Information Officers (CIO) to develop, maintain, and facilitate integrated system architectures. In addition, OMB has issued guidance that among other things, requires agency’s information systems investments to be consistent with federal, agency, and bureau architectures. OMB has also issued guidance on the development and implementation of agency information technology architectures.
Treasury has also issued to its bureaus, including Customs, guidance on developing an information systems architecture. This guidance, known as Treasury Information Systems Architecture Framework (TISAF), is also included in OMB’s guidance. According to Treasury, TISAF is intended to help reduce the cost, complexity, and risk associated with information technology development and operations. In July 1997, Treasury issued additional guidance to complement TISAF. This guidance, which was finalized in September 1997, provides “how to” processes for developing an information systems architecture in accordance with TISAF.
section_title: Customs’ Current Systems Development and Maintenance Efforts
paragraphs: Customs has several efforts underway to develop and acquire new information systems and evolve (i.e., maintain) existing ones to support its six business areas. Customs’ fiscal year 1998 budget for information management and technology activities is about $147 million.
Customs’ major information technology effort is its Automated Commercial Environment (ACE) system. In 1994, Customs began to develop ACE to replace its existing automated import system, the Automated Commercial System. ACE is intended to provide an integrated, automated information system for collecting, disseminating, and analyzing import-related data and ensuring the proper collection and allocation of revenues, totaling about $19 billion annually. According to Customs, ACE is planned to automate critical functions that the Congress specified when it established NCAP.
Customs reported that it spent $47.8 million on ACE as of the end of fiscal year 1997. In November 1997, Customs estimated it would cost $1.05 billion to develop, operate, and maintain ACE over the 15 years from fiscal years 1994 through 2008. Customs plans to deploy ACE to all 342 ports that handle commercial cargo imports.
Customs plans to develop and deploy ACE in multiple phases. According to Customs, the first phase, known as NCAP, is to be an ACE prototype. Customs currently plans to deploy NCAP in four releases. The first is scheduled to be deployed for field evaluation at three locations beginning in May 1998, and the fourth is scheduled for October 1999. Customs, however, has not adhered to previous NCAP deployment schedules.
Specifically, implementation of the NCAP prototype slipped from January 1997 to August 1997 and then again to a series of four releases beginning in October 1997, with the fourth release starting in June 1998.
Customs also has several other efforts underway to modify or enhance existing information systems that support its six business areas. For example, in fiscal year 1998, Customs plans to spend about $3.7 million to enhance its Automated Export System (AES), which supports the outbound business area and is designed to improve Customs’ collection and reporting of export statistics and to enforce export regulations. In addition, Customs plans to spend another $4.6 million to modify its administrative systems supporting its finance and human resource business areas. Examples of other systems that Customs plans to modify or enhance are the Automated Commercial System, the Treasury Enforcement and Communication System, and the Seized Asset and Case Tracking System.
section_title: Recent Reviews of Customs Have Disclosed Systems Problems
paragraphs: In May 1996, we reported that Customs was not prepared to select an architecture and develop ACE because it was not effectively applying critical management practices that help organizations mitigate the risks associated with modernizing automated systems and better position themselves for success. Specifically, Customs (1) lacked clear accountability for ensuring successful implementation of NCAP requirements, (2) selected an information systems architecture for ACE and other systems without first analyzing its business requirements, (3) lacked policies and procedures to manage ACE and other systems as investments, and (4) did not ensure that systems under development adhere to Customs’ own system development policies.
As a result of our recommendations, Customs took the following actions.
Assigned day-to-day responsibility for implementing NCAP to the Assistant Commissioner, Office of Information and Technology. Initiated an effort, with contractor assistance, to develop an enterprise information systems architecture.
Designated an information technology investment review board (IRB) and hired a contractor to develop investment management policies and procedures. The contractor completed its work in mid-1997 and the agency is in the process of implementing and institutionalizing these information technology investment management processes and procedures.
Revised its Systems Development Life Cycle (SDLC), conducted ACE cost-benefit analyses, instituted SDLC compliance reviews, and prepared a variety of ACE-related project plans. Customs also developed processes to ensure that SDLC compliance is an ongoing activity.
In May 1997, we reported that significant weaknesses continue to be identified during audits of Customs’ financial statements that hinder Customs’ ability to provide reasonable assurance that sensitive data maintained in automated systems, such as critical information used to monitor Customs’ law enforcement operations, are adequately protected from unauthorized access and modification. Since then, Treasury’s Inspector General has reported that Customs’ computer systems continue to be vulnerable to unauthorized access. Specifically, the Inspector General reported that security weaknesses could allow for unauthorized modification and deletion of application and systems software and data in Customs computer systems that support trade, financial management, and law enforcement activities.
section_title: Customs’ Architecture Is Incomplete but Plans for Completing It Are Underway
paragraphs: Treasury and Customs officials recognize that Customs’ systems architecture is not complete and plan to complete it. For five of its six business areas (outbound, passenger, finance, human resources, and investigations), Custom’s architecture does not (1) describe all the agency’s business functions, (2) outline the information needed to perform the functions, and (3) completely identify the users and locations of the functions. Further, while the architecture and related documentation describe business functions and users and locations for one business area (trade compliance), they do not identify the information needs and flows for all the functions. Nonetheless, Customs has defined many characteristics of its information systems’ hardware, software, communications, data management, and security components. Because these characteristics are not based on a complete understanding of its enterprisewide functional and information needs, Customs does not have adequate assurance that its information systems will optimally support its ability to (1) fully collect and accurately account for billions of dollars in annual federal revenue and (2) allow for the expeditious movement of legal goods and passengers across our nation’s borders while preventing and detecting the movement of illegal goods and passengers.
subsections: section_title: A Framework for Effectively Developing a Complete Systems Architecture
paragraphs: Reflecting the general consensus in the industry that large, complex systems development and acquisition efforts should be guided by explicit architectures, we issued a report in 1992 defining a comprehensive framework for designing and developing systems architectures. This framework divides systems architectures into a logical component and a technical component.
The logical component ensures that the systems meet the business needs of the organization. It provides a high-level description of the organization’s mission and target concept of operations; the business functions being performed and the relationships among functions; the information needed to perform the functions; the users and locations of the functions and information; and the information systems needed to support the agency’s business needs. An essential element of the logical architecture is the definition of the component interdependencies (e.g., information flows and interfaces).
The technical component ensures that systems are interoperable, function together efficiently, and are cost-effective over their life cycles (including maintenance costs). The technical component details specific information technology and communications standards and approaches that will be used to build systems, including those that address critical hardware, software, communications, data management, security, and performance characteristics.
TISAF, Treasury’s departmentwide architecture framework, is generally consistent with our framework. According to TISAF, a complete architecture has the following four components, each representing a different perspective or view of the agency:
Functional: A representation of what the organization does (i.e., its mission and business processes) and how the organization can use information systems to support its business operations.
Work: A description of where and by whom information systems are to be used throughout the agency.
Information: A description of what information is needed to support business operations.
Infrastructure: A description of the hardware and “services” (e.g., software and telecommunications) needed to implement information systems across the agency.
TISAF’s functional, work, and information components together form the logical view of the architecture, while its infrastructure represents the technical view of the architecture.
To develop and evolve systems that effectively support business functions, a top-down process must be followed. The logical architecture (e.g., business functions and information flows) is defined first and then used to specify supporting systems (e.g., interfaces, standards, and protocols).
Treasury endorses this top-down approach. Treasury officials responsible for developing and implementing TISAF stated that development of the architecture begins with defining and describing the agency’s major business functions. Once this is accomplished, the agency can identify the relationships among the functions, the information needed to perform the functions, the users and locations of the functions, and the existing and needed applications and related information technology required to execute and support the business functions. According to Treasury guidance, the architecture’s infrastructure component (i.e., its systems specifications and standards) should be derived from the other three components. In addition, the guidance states that each element of the architecture must be integrated and traceable, and the relationships between them must be explicit.
section_title: Incomplete Enterprise Architecture Increases the Risk of Building Systems That Do Not Effectively Support Business Needs
paragraphs: Customs does not have a complete systems architecture to effectively and efficiently guide and constrain the millions of dollars it invests each year in developing, acquiring, and maintaining the information systems that support its six business areas. In summary, for five of Customs’ six business areas (outbound, passenger, finance, human resources, and investigations), the architecture neither defines all critical business functions nor identifies all information needs (including information security) and information flows within and among the business areas. For the sixth business area (trade compliance), Customs has defined all the business functions and users and work locations and some, but not all, of the information and data needs and flows.
With respect to the business functions, Customs’ architecture provides descriptions of only 29 of 79 collective functions in its six business areas. The architecture does not describe the other 50 functions in sufficient detail to understand what they are, how they relate, who will perform them, where they will be performed, what information they will produce or consume, and how the information should be handled (i.e., captured, stored, processed, managed, distributed, and protected). Table 1 summarizes by business area the number of functions defined in the architecture.
Examples of undefined functions in the outbound, passenger, investigations, and human resources business areas are as follows:
Outbound: The architecture names “examine cargo” and “seize and process cargo” as 2 of the 13 functions in this business area. However, the architecture does not describe how to examine cargo, what cargo to examine, when to examine cargo, what information/data is needed to examine cargo, how the results of the cargo examination are used and by whom, or how cargo examination data should be protected. Similarly, the architecture does not describe when cargo will be seized and by whom, what criteria are used to seize cargo, how cargo will be seized and accounted for, or what information is required to account for the seized cargo (e.g., date of seizure, company name, and commodity).
Passenger: The architecture names “identify compliance target” and “process non-compliant passengers/conveyances” as 2 of the 13 functions in this business area. However, the architecture does not describe how targets are identified, who identifies targets, how target information is disseminated, what information is collected to determine compliance, or how target information needs to be protected. Likewise, the architecture does not define compliant passenger/conveyance, how passengers are processed and by whom, or where passengers/conveyances are processed.
Investigations: The architecture names “perform interdiction” as 1 of the 10 functions in this business area. However, the architecture does not describe how an interdiction is conducted, who conducts interdictions, what criteria are used to identify potential passengers or cargo to interdict, what happens to the seized persons or cargo, or how interdiction information needs to be protected.
Human Resources: The architecture names “manage internal service programs” as 1 of the 22 functions in this business area. However, the architecture does not describe what services are provided and by whom, who is eligible to receive the services, or where the potential recipients are located.
Within the trade compliance business area, even though Customs’ architecture does not define 10 of 15 trade compliance functions, Customs has described these 10 business functions, the relationships among them, and the work to be performed within each function (including who will perform the work and where it will be performed) in documents other than the architecture. Further, Customs has specified the data needed to support some, but not all, of the trade compliance functions. For example, Customs identified key information sources (such as cargo manifests and summary declarations) associated with NCAP, the ACE prototype that covers a subset of trade compliance activities, and specific data elements associated with each information source.
Customs, however, has not defined the information/data needs, including security, and information/data flows among its six business areas. With respect to information security in particular, Customs’ architecture does not (1) specify functional requirements for enterprisewide security, (2) include a security concept of operations that describes how Customs will operate (e.g., what controls will be used) to satisfy these requirements, or (3) include a security subarchitecture that specifies how these controls will be implemented, certified, and accredited and how the controls’ operational effectiveness will be validated. Given that computer security continues to be a long-standing problem at Customs, this issue is particularly troubling. In our audits of Customs’ fiscal year 1992 and 1993 principal financial statements, we stated that Customs’ controls to prevent and detect unauthorized access and intentional or inadvertent unauthorized modifications to critical and sensitive data and computer programs were ineffective, thereby jeopardizing the security and reliability of the operations central to Customs’ mission. While Customs has since taken meaningful steps toward correcting these access problems, they still remain. According to the Treasury Inspector General’s report on Customs’ fiscal years 1997 and 1996 financial statements, computer security weaknesses continue to exist that could allow for unauthorized modification and deletion of application and systems software and data in Customs’ systems supporting the trade, financial management, and law enforcement activities.
Until Customs addresses these weaknesses, it will not know the full extent of inter- and intra-business area functional and informational needs and dependencies and thus cannot develop, acquire, and maintain supporting information systems that optimally support the agency’s operations and activities. Moreover, until these interdependencies among and within business areas have been fully analyzed and defined and an approach for securing the associated information has been established, the opportunities for incompatibilities and duplications among systems and the information they process and share increase, as do the opportunities for unauthorized access and modification of data. Such opportunities jeopardize, in turn, the completeness, consistency, and integrity of the data Customs uses and publishes. Given the importance of reliable data to Customs’ (1) billion dollar revenue collection mission, (2) trade statistics used in developing trade policy and negotiating trade agreements, and (3) efforts to prevent and detect the illegal movement of goods and services across our nation’s borders, such risks must be effectively addressed through an enterprise systems architecture.
With respect to the infrastructure or technical component of Customs’ architecture, Customs has specified much of the information that Treasury guidance states should be included in this component (e.g., standards for system and application software, communication interfaces, and hardware). However, as noted previously, this component is not based on a complete analysis of Customs’ functional and information needs. For example, the architecture does not address information security requirements, yet its infrastructure specifies network encryption and remote access server products. Because it specified these products without knowing the business needs they support, Customs does not have adequate assurance that these products are needed or that they satisfy its true business needs, minimally or optimally. That is, the list of products cited may be either unnecessary or insufficient to support its real business needs.
Experience has shown that attempting to define and build major systems without first completing a systems architecture unnecessarily increases the cost and complexity of these systems. For example, we reported that FAA’s lack of a complete architecture resulted in incompatibilities among its air traffic control systems that (1) required higher- than-need-be system development, integration, and maintenance costs and (2) reduced overall system performance. Without having architecturally defined requirements and standards governing information and data structures and communications, FAA was forced to spend over $38 million to acquire a system dedicated to overcoming incompatibilities between systems. According to a Customs’ contractor, Customs is also experiencing such inefficiencies and unnecessary costs because it lacks an architecture. Specifically, this contractor reported that in the absence of an enterprise infrastructure, Customs’ departments have developed and implemented incompatible systems, which has increased modernization risks and implementation costs.
section_title: Customs Did Not Take Full Advantage of Contractor’s Efforts to Define Architecture
paragraphs: Customs awarded a contract in January 1997 to develop, among other things, a “technology architecture.” However, Customs did not properly define the scope of this architecture, limiting it to deliverables associated with the infrastructure component without first completing the other components. Customs officials stated that they contracted for the infrastructure without first completing the higher levels of the architecture because they considered the infrastructure component to be the most important and urgently needed part of the architecture.
This “bottom up” approach is fundamentally inconsistent with government and industry architectural frameworks and guidance, including Treasury’s, and has historically resulted in systems that do not effectively support business operations and waste time and money. For example, after the Internal Revenue Service (IRS) spent over $3 billion attempting to modernize its tax systems without a defined logical architecture, it could not demonstrate benefits commensurate with costs and was forced to significantly restructure the effort. Unless it completes its architecture before attempting to develop operational systems like ACE, Customs runs the risk of repeating failures like those that IRS experienced.
Customs’ CIO officials have since acknowledged the need for a complete systems architecture and its value in information technology investment management. Accordingly, Customs is developing a statement of work for a TISAF-compliant architecture. With the help of a contractor, Customs plans to use whatever data each business area may have already developed relative to functional, work, and information needs as a starting point in completing an enterprise architecture. More specifically, by October 1998, Customs plans to identify the functional, work, and information components for each of the six business areas and identify the relationships and interdependencies across the business areas. Customs also plans to reevaluate its enterprise infrastructure.
section_title: Customs’ Plans for Enforcing Its Architecture Will Not Ensure Compliance
paragraphs: If an architecture is to be implemented effectively, institutional processes must be established to (1) require system compliance with the architecture, (2) assess and enforce such compliance, and (3) waive this requirement only on the basis of careful, thorough, and documented analysis showing that such deviation is warranted.
According to Customs officials, architectural compliance will be assessed and enforced as Customs implements its recently defined investment management process. Under this process, Customs’ investment review board (IRB) uses four criteria in scoring competing investment options and allocating funding among them. The four criteria are risk (e.g., technical, schedule, and cost); strategic alignment (e.g., cross-functional benefits, linkage to Customs’ business plan, and compliance with legislative mandates); mission effectiveness (e.g., contributions to service delivery); and cost/benefit ratio (e.g., tangible and intangible benefits, and costs).
Customs is in the process of implementing its investment management process for the fiscal year 1999 budget cycle.
According to Customs’ investment management process, investment compliance with the architecture is considered, but not required, under the technical risk criterion. As a result, the process does not preclude funding projects that do not comply with the enterprise architecture and does not require that deviations from the architecture be rigorously justified. According to Customs officials, while architectural compliance is not an explicit criterion in the process, it will be considered and documented as part of the IRB funding decisions.
Without an effective, well-defined process for enforcing the architecture, Customs runs the risk that unjustified deviations from the architecture will occur, resulting in systems that do not meet business needs, are incompatible, perform poorly, and cost more to develop, integrate, and maintain than they should. For example, we reported that FAA’s lack of an enforced systems architecture for its air traffic control operations resulted in the use of expensive interfaces to translate different data communication protocols, thus complicating and slowing communications, and the proliferation of multiple application programming languages, which increased software maintenance costs and precluded sharing software components among systems.
section_title: Conclusions
paragraphs: Customs’ incomplete enterprise information systems architecture and limitations in its plans for enforcing compliance with an architecture once one is completed impair the agency’s ability to effectively and efficiently develop or acquire operational systems, such as ACE, and to maintain existing systems. Until Customs (1) performs the thorough analysis and careful decision-making associated with developing all architectural components for interdependent business areas and (2) ensures that these results are rigorously enforced for its information system development, acquisition, and maintenance efforts, it runs the risk of wasting scarce time and money building and maintaining systems that do not effectively and efficiently support its business operations.
section_title: Recommendations
paragraphs: To ensure that the Customs Service develops and effectively enforces a complete enterprise information systems architecture, we recommend that the Commissioner of Customs direct the Customs CIO, in consultation with the Treasury CIO, to follow through on plans to complete the enterprise information systems architecture. At a minimum, the architecture should (1) describe Customs’ target business operations, (2) fully define Customs’ interrelated business functions to support these target operations, (3) clearly describe information needs (including security) and flows among these functions, (4) identify the systems that will provide these functions and support these information needs and flows, and (5) use this information to specify the technical standards and related characteristics that these systems should possess to ensure that they interoperate, function together efficiently, and are cost-effective to maintain.
We also recommend that the Commissioner direct the Deputy Commissioner, as Chairman of the IRB, to establish compliance with the architecture as an explicit requirement of Customs’ investment management process except in cases where careful, thorough, and documented analysis supports a waiver to this requirement.
section_title: Agency Comments and Our Evaluation
paragraphs: In commenting on a draft of this report, Customs agreed with our conclusions and recommendations and stated that it will (1) develop an enterprise systems architecture in accordance with TISAF and in close cooperation with Treasury during fiscal year 1998 and (2) strengthen enforcement of the architecture by being explicit that projects must comply with the architecture and requiring exceptions to be well justified. Additionally, Customs committed to not making major system investments prior to developing a TISAF-compliant architecture.
Customs raised several additional matters related to systems architecture, none of which affect our conclusions and recommendations and thus are not discussed here. Customs’ comments and our responses are reprinted in appendix II.
We are sending copies of this report to the Ranking Minority Members of the Subcommittee on Treasury and General Government, Senate Committee on Appropriations, and Subcommittee on Treasury, Postal Service, and General Government, House Committee on Appropriations. We are also sending copies to the Secretary of the Treasury, the Commissioner of Customs, and the Director of the Office of Management and Budget. Copies will also be made available to others upon request. If you have any questions about this letter, please contact me at (202) 512-6240 or by e-mail at brockj.aimd@gao.gov. Major contributors to this report are listed in appendix III.
section_title: Objectives, Scope, and Methodology
paragraphs: To accomplish the first objective, we reviewed published architectural guidance, including the Treasury Information Systems Architecture Framework (TISAF), to identify key requirements. We also interviewed officials from Treasury’s Office of the Deputy Assistant Secretary for Information Systems and Chief Information Officer (the organization responsible for developing, implementing, and maintaining TISAF) to seek clarification and explanation of TISAF requirements. Further, we asked Customs to give us its enterprise information systems architecture and a mapping of all architectural documents to TISAF’s four architectural components—functional, work, information, and infrastructure. In response, Customs provided the documents listed in table I.1.
Customs subsequently provided two additional architecture documents that it did not map to any TISAF component. The two additional documents were the ACE Technical Architecture and the Enterprise IT Architecture Strategy-Executive Overview.
We then analyzed the architecture documents Customs provided to identify any variances with the TISAF requirements for each architectural component. We also interviewed Customs and supporting contractor officials to (1) seek clarification and explanation of the content of the architecture documents, (2) identify instances where the architectural documents did not satisfy TISAF requirements, and (3) solicit from Customs any additional evidence related to meeting TISAF requirements.
To address the second objective, we reviewed Customs’ policies and procedures governing information technology investment management to determine architecture enforcement processes and interviewed Customs officials to determine organizational roles and responsibilities related to architecture development and enforcement. We also discussed with Customs officials any plans for changing the agency’s processes and organizational responsibilities for developing and enforcing the architecture.
section_title: Comments From the U.S. Customs Service
paragraphs: The following are GAO’s comments on the U.S. Customs Service’s letter dated March 31, 1998.
subsections: section_title: GAO Comments
paragraphs: 1. Our report neither states nor implies that Customs is unable to ensure the proper collection and allocation of revenues totaling about $19 billion annually. Rather, the report states that one of ACE’s key functions is to ensure the proper collection and allocation of revenues totaling about $19 billion annually. 2. Customs states that it began developing its enterprise systems architecture prior to Treasury’s publication of TISAF and is working with Treasury to develop a TISAF-compliant architecture. While these statements are true, they do not address our point that Customs’ architecture is insufficiently complete to be useful in guiding and constraining major systems investments. In order to optimize systems investments, the architecture must specify the six elements cited in our report. Furthermore, each element of the architecture must be built upon the preceding ones. Customs’ architecture does not include these elements for all business areas and, as we point out in our report, the systems and standards selected were not based on a complete analysis of Customs’ functional and information needs.
We do not agree with Customs’ statement that an architecture is never completed. An architecture must be complete (i.e., include the six elements described in our report) to be useful in building or buying systems. This does not mean that a completed architecture cannot be modified to reflect changes in organizational missions and business functions or advancements in information technology products. This process of thoughtful and disciplined change—maintenance—is performed routinely on all information system components (e.g., architectures, documentation, software, and hardware). 3. While we agree that architectural models used in industry and government vary, all models consistently require the top-down, structured approach described in our report. Customs has not followed this approach and, therefore, does not have adequate assurance that its infrastructure (i.e., technical architecture) will meet its business requirements.
Customs states that it has been cautioned against defining an architecture in too much detail lest the business process changes before system development can proceed, but it does not clearly define what it means by too much detail. Customs’ architecture neither defines all critical business functions nor identifies all information needs and flows within and among the business areas for five of its six business areas. As a result, rather than being overly detailed, it lacks the basic, required elements. 4. While the Treasury Inspector General (IG) has given Customs an unqualified opinion in fiscal year 1997, the IG also reported that Customs lacks adequate assurance that all revenue due is collected and compliance with other trade laws is achieved. Despite the progress that has been made, this lack of assurance has been a persistent issue since we reported on our audit on Customs’ financial statements for fiscal year 1992.5. Customs states that we have inaccurately characterized the completeness of its architecture for the finance business area because certain finance business functions have been defined in various other analyses, reports, and strategies. This assertion reflects a misunderstanding of the purpose and value of a systems architecture. Our report concludes that Customs’ architecture for its finance business area (as well as all but one other business area) is substantially incomplete because it does not (1) describe all the agency’s business functions, (2) outline the information needed to perform the functions, or (3) completely identify the users and locations of the functions. Even if other documents contain fragments of the missing information for one business area, which we did not attempt to verify, this does not mitigate the need for a single, comprehensive, maintainable, and enforceable statement of architectural requirements and standards.
section_title: Major Contributors to This Report
subsections: section_title: Accounting and Information Management Division, Washington, D.C.
paragraphs: Rona Stillman, Chief Scientist for Computers and Telecommunications Linda Koontz, Associate Director Randolph Hite, Senior Assistant Director Deborah A. Davis, Assistant Director Madhav Panwar, Senior Technical Advisor Mark Bird, Assistant Director The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.