USDA Telecommunications: Better Management and Network Planning Could Save Millions
Released: 1995-09-25T00:00:00

Letter

Introduction

USDA affects the lives of all Americans and millions of people around the world. Created 133 years ago to conduct research and disseminate information, USDA’s role has been expanded to include, among other things, providing billions of dollars annually to support farm incomes; developing agricultural markets abroad to boost domestic farm production and exports; ensuring a safe food supply; managing and conserving the nation’s forests, water, and farmland; and providing education and supplemental resources to the needy to improve diet and nutrition. USDA’s challenge is to meet its responsibilities as it also adapts to a rapidly changing global marketplace.

During 1994, USDA delivered services through 43 agencies and a network of more than 14,000 field offices. Pursuant to Public Law 103-354, the Secretary of Agriculture reorganized the Department by reducing the number of component agencies from 43 to 29. The Secretary has also announced plans to reduce the number of county field offices by about 1,200 over the next 3 years. To carry out its missions, the Department and its component agencies reported budget outlays of about $61 billion in fiscal year 1994, according to the President’s fiscal year 1996 budget request.

Telecommunications: A Vital but Costly Resource

Like other federal agencies, USDA’s agencies rely on telecommunications networks and systems to accomplish missions and serve customers. The Department and its agencies deliver USDA services through thousands of field offices in states, cities, and counties. These offices acquire and use various types of telecommunications services and equipment to meet mission needs.

Because telecommunications plays a vital role at USDA, it is imperative for the Department to plan and manage all its telecommunications resources effectively and prudently. According to the Department’s January 1993 Information Resources Management (IRM) Strategic Plan, telecommunications systems that provide quick and reliable voice and data communication throughout the Department are critical to USDA’s success in carrying out its many missions and necessary for building a network infrastructure capable of sharing information whenever and wherever it is needed. The effective and prudent use of telecommunications technology is also critical to the success of USDA’s efforts to streamline and consolidate its field office structure and reduce operational costs.

USDA reports show that it spends about $100 million annually for telecommunications. This includes about $37 million for FTS 2000 services in fiscal year 1994. USDA is required to use FTS 2000 network services for basic long-distance communications (i.e., the inter-Local Access and Transport Area (LATA) transport of voice and data communications traffic). Under the federal government’s FTS 2000 contract, USDA agencies and offices use basic switched service for voice, packet switched service for data, video transmission service, and other types of services to support their communications needs.

In addition to FTS 2000, USDA estimates that during fiscal year 1994 it spent another $50 million on local telecommunications and other services obtained from about 1,500 telephone companies. USDA agencies and offices use these services to meet their local telephone and data communications needs within LATAs. Other telecommunications services obtained from commercial carriers that are not available under the FTS 2000 contract, such as satellite communications, are also included in these costs. USDA also estimates that between $10 million and $30 million is spent annually on telecommunications equipment, such as electronic switches and telephone plant wiring, and support services, such as maintenance for acquired telecommunications equipment.

The Federal Information Resources Management Regulation and USDA’s Telecommunications Policy (DR-3300-1) require that USDA’s agencies maximize use of all government telecommunications resources to achieve optimum service at the lowest possible cost. In addition, Section 215 of the Department of Agriculture Reorganization Act of 1994, requires USDA to reduce expenses by jointly using resources at field offices where two or more agencies reside. This includes sharing telecommunications services and equipment. Also, section 216 of this act requires that whenever USDA procures or uses information technology it should do so in a manner that promotes computer information sharing among its agencies.

OIRM Is Responsible for Ensuring USDA’s Telecommunications Are Managed and Planned Cost-Effectively

The senior USDA IRM official—the Assistant Secretary for Administration—has delegated responsibility for managing all aspects of the Department’s telecommunications program to the OIRM Director. According to federal regulations, this responsibility includes the following to ensure that telecommunications resources are maximized at the lowest possible cost: develop departmental telecommunications guidelines and regulations necessary to implement approved principles, policies, and objectives, review and evaluate telecommunications activities for conformance with all applicable federal and USDA telecommunications policies, plans, procedures, and guidelines, develop and implement a telecommunications planning system that integrates short- and long-term objectives and coordinates agency and staff office initiatives in support of these objectives, and monitor agencies’ network systems acquisition and development efforts to ensure effective and economic use of resources and compatibility among systems of various agencies.

At USDA, component agencies manage the acquisition and use of telecommunications services and equipment on a day-to-day basis. Because of this, OIRM is principally responsible for providing departmentwide telecommunications policy and direction and monitoring the agencies’ activities to ensure their compliance. For example, in December 1993, OIRM’s Telecommunications Policy Division consolidated all existing telecommunications policy into a comprehensive directive—Departmental Directive 3300-1—which is USDA’s current policy in this area. Also, in September 1993, OIRM and the Office of Assistant Secretary for Administration developed USDA’s first departmentwide Strategic Telecommunications Plan. According to USDA policy, this Plan shall serve as guidance to the agencies for developing their respective agency telecommunications plans.

With respect to monitoring telecommunications, USDA established a IRM Review Program, as required by federal law, to periodically review component agencies’ information and telecommunications management activities. According to the Federal Information Resources Management Regulation, such a program is intended to, among other things, (1) ensure agencies comply with governmentwide and departmentwide telecommunications policies, regulations, rules, standards, and guidelines, (2) ensure agencies efficiently acquire and effectively use resources, and (3) determine whether agencies’ controls over and reviews of their telecommunications resources provide effective management oversight. To do this, USDA policy requires OIRM to conduct periodic reviews at each of USDA’s agencies.

In addition, OIRM established its Agency Liaison Officer (ALO) Program in late 1992 to, among other things, strengthen coordination of the agencies’ telecommunications projects and planning to ensure that there are not unnecessary barriers to information exchange. In doing so, OIRM obtained additional staff and made them responsible for (1) analyzing IRM programs to ensure that they are consistent with Department goals and objectives and (2) maintaining an understanding of an agency’s plans for information and telecommunications technology investments to ensure there is adequate departmentwide coordination. Moreover, under its technical approval authority, OIRM reviews and approves component agency requests for procurements of telecommunications resources.

Telecommunications Management Practices Differ Among USDA Agencies

Telecommunications management practices vary widely across USDA agencies because the agencies independently plan, acquire, operate, and manage telecommunications resources—equipment and services—in accordance with their own organizational and mission needs. In this regard, commercial telecommunications services that USDA agencies obtain from over 1,500 vendors across the nation are acquired and managed locally, regionally, or centrally depending on the agency. For example, the Consolidated Farm Service Agency (CFSA) has nearly 3,000 county offices that individually acquire commercial telecommunications services from private vendors. This contrasts with the Forest Service, whose 9 regional offices acquire commercial telecommunications services for about 725 local offices, and the Agriculture Marketing Service, whose headquarters office acquires commercial telecommunications services centrally for its field offices.

With some exceptions, bills for commercial telephone calls, leased equipment, and other services for USDA’s component agencies are paid centrally by USDA’s National Finance Center (NFC) in New Orleans, Louisiana. NFC is reimbursed for these costs by the agencies after the bills are paid.

USDA’s component agencies also manage FTS 2000 services differently. For example, some agencies, such as CFSA, have a few Designated Agency Representatives (DARs) responsible for centrally acquiring FTS 2000 services for the entire agency. However, others, such as the Forest Service and the Rural Economic and Community Development (RECD) agency, have numerous DARs that order FTS 2000 services for offices in specific geographical areas. Bills for all FTS 2000 services acquired and used by USDA’s component agencies are paid directly to the General Services Administration.

Just as USDA component agencies acquire and manage telecommunications resources differently, they also plan and develop telecommunications networks separately in support of their agency-specific missions. These networks include telecommunications systems that support local office communications, regional communications between agency offices, and nationwide networks.

Information Sharing: a Long-standing Problem at USDA

Historically, USDA agencies have had difficulty sharing information electronically because they independently acquired information technology and networks that were not intended to address the organizational sharing needs of the Department. As far back as October 1989, we reported that while many USDA agencies shared responsibility for policy issues, such as food safety or water quality, they often were incapable of sharing information electronically due to their stovepipe systems. Because of this, we noted that USDA managers had difficulty carrying out programs to effectively address issues that cut across traditional agency boundaries. For example, nine separate USDA agencies shared responsibility for water quality. However, agencies could not easily share information across the separate network systems these agencies had installed. Therefore, USDA’s water quality programs suffered because critically important information, necessary to effectively carry out these programs, often remained inaccessible outside an agency and was under utilized throughout the Department.

In late 1993, USDA surveyed its employees and received over 8,000 suggestions for operating more efficiently. Many respondents said the information sharing problems adversely affected program delivery and was a significant problem for the Department. Specifically, many respondents reported that USDA’s information systems and networks have too often developed along program and agency lines, causing information “islands” to develop across the Department.

Objectives, Scope, and Methodology

At the request of the Chairman, Senate Committee on Agriculture, Nutrition, and Forestry and the Ranking Minority Member, Subcommittee on Government Management, Information and Technology, House Committee on Government Reform and Oversight, we reviewed the effectiveness of USDA’s management and planning of telecommunications. Our objectives were to determine whether USDA is (1) managing existing telecommunications resources cost-effectively and consolidating services to maximize savings and (2) effectively planning future communications networks to meet the Department’s information sharing needs.

To determine whether USDA is cost-effectively managing telecommunications resources, we reviewed federal laws, regulations, and guidance as well as USDA policies and guidance for establishing telecommunications management controls. We interviewed OIRM management, agency managers, and field personnel to discuss USDA’s telecommunications policy and guidance. We also discussed OIRM’s IRM review program, the ALO Program, and the technical approval process to obtain USDA officials views on the effectiveness of these programs. We evaluated reports documenting IRM reviews completed by OIRM since 1990 and assessed the completeness and effectiveness of these reviews.

In addition, we interviewed senior-level representatives from 10 USDA agencies that account for about 70 percent of USDA’s telecommunications costs to identify management practices they adopted for telecommunications. Specifically, we discussed their management controls for establishing telecommunications inventories, monitoring acquisitions, and reviewing and verifying bills. We reviewed users’ internal policies and guidelines to determine the type and extent of management controls that these agencies have instituted over the use of telecommunications resources.

We visited three locations where USDA installed consolidated telecommunications systems, obtained their telephone bills from NFC, and reviewed them to assess whether telecommunications resources were managed cost-effectively. We conducted our review of telephone bills for USDA agencies at these locations because we had observed telecommunications activities at each of these sites. We also discussed the bill payment process with officials from NFC and obtained additional information from commercial vendors on these bills. In particular, when reviewing bills, we determined whether agencies (1) used FTS 2000 services as required by GSA to make long-distance calls and for other available services and (2) obtained the most cost-effective services available. In addition, we obtained and reviewed telephone bills for USDA’s Rural Development Agency regional offices that closed during the past year to determine whether telephone services had been properly disconnected at these sites.

To determine whether USDA is planning its future communications networks to effectively support its information sharing needs, we reviewed agency plans to develop new network systems and discussed these planned systems with agency management and OIRM officials. We also reviewed USDA’s strategic telecommunications plan to assess whether it provides guidance to the agencies on what departmentwide information sharing needs must be met and how to go about doing this. To evaluate the effectiveness of USDA’s strategic plan in defining the Department’s telecommunications requirements, we interviewed agency IRM and program officials and reviewed OIRM files and other supporting documentation. In addition, we visited field offices engaged in ongoing network development projects to assess project planning and management and determine the effectiveness of project results.

We also interviewed OIRM officials responsible for oversight of agencies’ telecommunications plans and acquisitions to ascertain how these officials review agencies’ plans to ensure that they, along with the subsequent acquisitions, meet departmentwide goals and objectives. In addition, we reviewed OIRM documentation of its oversight activities to determine to what extent agencies’ telecommunications projects are coordinated across the department.

We performed our audit work from March 1994 through July 1995, in accordance with generally accepted government auditing standards. Our work was primarily done at USDA headquarters in Washington, D.C.; USDA’s NFC in New Orleans, Louisiana; and USDA’s Telecommunications Services Division in Fort Collins, Colorado. We also visited component agency offices where telecommunications and network planning activities are administered. They included state offices of USDA farm service agencies in Lexington, Kentucky; Richmond, Virginia; and Columbia, Missouri; district and county offices of USDA farm service agencies in Mount Sterling, Kentucky and Pendleton, Oregon. In addition, we visited Forest Service headquarters in Arlington, Virginia; the Service’s Northwestern Region in Portland, Oregon; and the Service’s National Forest offices in Corvallis and Pendleton, Oregon; Food and Consumer Service headquarters in Alexandria, Virginia; Agricultural Research Service, Greenbelt, Maryland; APHIS headquarters in Hyattsville, Maryland, and regional office in Fort Collins, Colorado; Consolidated Farm Service Agencies’ office in Kansas City and Rural Economic and Community Development office in St. Louis, Missouri.

We requested written comments on a draft of this report from the Secretary of Agriculture. In response, we received written comments from the Assistant Secretary for Administration. These comments are discussed in chapter 4 and are reprinted in appendix I.

USDA’s Telecommunications Resources Are Not Managed Cost-Effectively

OIRM has not fulfilled its management responsibility to provide the guidance and oversight necessary to ensure that USDA’s agencies maintain basic management data on their telecommunications resources, obtain telecommunications equipment and services cost-effectively, verify the accuracy of telecommunications charges, and make proper use of government-provided resources and services. Without sufficient guidance and oversight, many USDA component agencies have not instituted sound management practices necessary to effectively manage the telecommunications resources they control. As a result, these agencies waste millions of dollars each year paying for (1) unnecessary telecommunications services and equipment, (2) leased equipment that is not used and services billed for but never provided, and (3) commercial carrier services that are more expensive than those provided under the FTS 2000 contract. Although USDA has some initiatives underway to improve telecommunications management, its actions do not fully resolve these inadequacies.

USDA Agencies Lack Telecommunications Inventories and Sufficient Management Controls

Federal laws and regulations require agencies to manage telecommunications resources cost-effectively. One of the most fundamental steps is maintaining current and complete inventory information on all telecommunications services and equipment. Without this, agencies lack the basic information they need to manage these resources cost-effectively. In addition to maintaining inventories, agencies also need to have appropriate management controls to ensure that all government-provided telecommunications resources are properly used. However, OIRM has not required USDA’s component agencies to maintain inventories of telecommunications resources and has not provided guidance to the agencies for establishing effective telecommunications management controls.

Department and Agencies Lack Basic Data Necessary to Manage Telecommunications

To ensure the cost-effective use of telecommunications equipment and services, the Federal Information Resources Management Regulation requires each agency to establish inventories of telecommunications resources and annually survey existing telecommunications systems to ensure that information on these systems is current, accurate, and complete. These surveys and inventories are fundamental to sound telecommunications management. According to the Federal Information Resources Management Regulation, inventories and surveys are necessary to, among other things, identify telecommunications resources that are outdated or no longer used and ensure that agencies pay for only those resources that they use.

USDA’s telecommunications policy does not require agencies to maintain inventories or conduct surveys of all their telecommunications resources. OIRM officials also acknowledge that USDA does not have a departmentwide inventory for telecommunications equipment and services and has not done surveys to collect such information. Although USDA has a directive requiring agencies to maintain inventories on property and has a property management system, the system lacks information on many types of telecommunications systems and services. Specifically, it does not record information on the types of voice, data, and video services used by the Department and where these services are located. It also lacks information on circuits, communications software, and many types of equipment, such as on-premises wiring, interface cards, modems, and other communications devices.

Even though OIRM officials agree that USDA telecommunications policy does not require agencies to maintain inventories or conduct annual surveys, they told us that agencies nonetheless should be doing this as part of their telecommunications management activities. However, agencies we contacted do not maintain agencywide inventories or conduct annual surveys of telecommunications resources, and OIRM has not followed up with these agencies to ensure they do so.

A lack of inventory information severely impairs USDA’s ability to ensure that resources are properly acquired, used, and maintained. OIRM does not know basic information, such as how much USDA pays for telecommunications, the type of services and equipment being used, and communications traffic volumes. Because of this, OIRM cannot effectively plan the future use of telecommunications resources, help agencies avoid acquiring redundant and overlapping equipment and services, and identify and eliminate systems and services that are not cost-effective.

For example, for years, USDA wasted thousands of dollars paying for numerous FTS 2000 Service Delivery Points (SDPs) within its headquarters office in Washington, D.C. Because USDA does not maintain a telecommunications inventory, OIRM did not know that headquarters had over 27 SDPs, many of which were redundant and unnecessary, until after the Secretary of Agriculture announced in November 1993 that the Department would reduce telecommunications costs at USDA headquarters by $1 million. In response to the Secretary’s direction, OIRM began collecting data on SDPs at headquarters and began eliminating these duplicate services. As a result, OIRM records show that USDA has achieved several hundreds of thousands of dollars in savings.

Also, a lack of inventory information hinders USDA’s effort to cost-effectively consolidate farm service agency offices. After the enactment of the Department of Agriculture Reorganization Act of 1994, the Secretary announced that 1,274 field offices would be closed, USDA personnel would be reduced by 11,000, and about 2,500 new field service centers would be established by September 1997. However, because basic inventory information is not available, USDA must now devote valuable time obtaining this information. Until this is done, USDA cannot effectively plan how to make the best use of existing equipment from offices that will close, what services need to be disconnected at these offices, and what additional equipment and services will need to be acquired for the new Service Centers.

Agencies Lack the Guidance Needed to Establish Adequate Management Controls

OIRM has not provided agencies with guidance on establishing management controls that are necessary for ensuring the proper planning and use of telecommunications resources. Specifically, OIRM has not provided the agencies with guidance for (1) monitoring acquisitions to ensure that telecommunications services and equipment are obtained cost-effectively, and (2) reviewing bills to verify the accuracy of telecommunications charges and ensure the proper use of government-provided resources and services.

Without such guidance, USDA agencies lack sufficient telecommunications management controls. For example, USDA has hundreds of field office sites where multiple USDA agencies, located in the same building or geographic area, obtain or use separate and often redundant commercial carrier services. This situation exists because agencies often acquire telecommunications services and equipment to meet their own needs without first determining what already exists and whether there are opportunities to share resources. Even within some agencies, telecommunications resources are sometimes purchased separately by different offices, and these purchases are not tracked agencywide to identify opportunities for sharing telecommunications resources.

Because of this, as we reported in April 1995, USDA is wasting millions on redundant FTS 2000 services. We reported that OIRM officials estimate that USDA could save between $5 million and $10 million annually by sharing FTS 2000 services. USDA has an even larger problem acquiring redundant commercial telecommunications services and equipment because agencies do not monitor and coordinate these purchases. According to OIRM officials, USDA could save as much as $15 million to $30 million annually by eliminating these redundancies and by sharing resources.

USDA also wastes millions more because many agencies do not verify whether they pay accurate charges for FTS 2000 and commercial telecommunications services and leased equipment and do not determine whether these resources are properly and cost-effectively used. According to the Federal Information Resources Management Regulation, agencies should establish call detail programs to verify usage of government-provided FTS 2000 and commercial long-distance services for which they are charged and deter or detect possible misuse of long-distance services. The regulation also requires agencies to pay for only those telecommunications resources being used and to cancel leases of underutilized resources.

To its credit, in October 1993, OIRM issued a policy establishing a program to review call detail reports for FTS 2000 services, and the office currently provides USDA agencies with these reports. However, many USDA agencies have not yet established an automated billing process for distributing FTS 2000 bills to each of their offices for the timely verification of the more than $36 million USDA pays annually for FTS 2000 services.

In addition, the Department also pays another $50 million each year for commercial telecommunications services and leased equipment that are not obtained under the FTS 2000 program. Currently, USDA pays over 23,000 bills each month for services obtained and equipment leased from over 1,500 private vendors across the country. However, very few of these commercial bills are ever reviewed because the Department and its agencies have not established sufficient procedures, such as those for reviewing call detail records, to verify charges by private vendors and ensure cost-effective use of telecommunications resources. Consequently, USDA is paying unnecessary and inappropriate charges. For example, our review of bills for the agencies at three locations we visited found that:

OIRM and USDA agencies pay tens of thousands of dollars each year to lease telephone equipment that is either no longer used or cannot be located. In some cases, we noted that fees for unused equipment have been paid for many years. For example, one commercial carrier’s bill for March 1995 showed that OIRM pays $6,262 a year to lease three unused 4800 baud modems at USDA headquarters. Although USDA has leased these modems since 1985, OIRM staff working at OIRM’s headquarters office told us that no one has used the modems for several years.

In this same bill, we found hundreds of cases where USDA agencies continue to pay exorbitant fees to lease outdated equipment. For example, we noted that USDA agencies pay about $7,800 dollars each year to lease 214 out-of-date rotary telephones.

More serious is that agencies were unable to locate some of this equipment. For example, although one agency pays over $10,000 a year to lease 16 2400 baud modems, telecommunications staff were unable to find any of them. The staff stated that, because no one uses this type of equipment any longer, it is likely that the equipment was disposed of many years ago.

USDA agencies often pay more than twice the cost charged under FTS 2000 by using commercial carriers to place toll calls within Local Access Transport Areas (LATAs) in states where such practices are allowed.Federal agencies have had the ability to use FTS 2000 service for intra-LATA calls since August 1993. In this particular instance, OIRM had notified agencies about this opportunity but agencies continued placing commercial calls within the LATAs. By not using FTS 2000 for intra-LATA toll calls, OIRM officials estimate that USDA is losing as much as $2 million each year.

Agencies pay about three times the amount charged under the FTS 2000 program for making long-distance calls. For example, according to March 1995 billing data, one USDA office in Fort Collins, Colorado, paid about $186 for long-distance calls that would have cost about $63 using FTS 2000 service. We noted many similar instances in which USDA agencies do not comply with the government’s mandatory FTS 2000 use. As a result, agencies pay significantly more than necessary for long-distance calls.

USDA also pays more than necessary for facsimile transmissions by obtaining these services from commercial vendors rather than FTS 2000. A March 1995 commercial carrier’s bill showed that one agency paid over $728 a month for commercial facsimile service. This represents more than 3 times the amount that is charged under the FTS 2000 program.

USDA agencies pay more for international calls than necessary because many agencies do not use the services available to USDA under the Department of Defense’s contract for international telephone service. According to OIRM officials, this contract offers a 34-percent savings over commercial rates. We noted many instances where agencies obtain such services outside of this contract.

Failure to Terminate Telecommunications Services at Offices Being Closed Results in Further Waste

USDA also wastes thousands of dollars paying for telecommunications services at field offices that have closed. This situation exists because office staff sometimes do not terminate vendor-provided services when they close offices. Since USDA does not generally review telephone bills, charges incurred after an office closes are not identified and USDA will continue to pay fees for vendor-provided services at these locations.

For example, in one case, USDA has continued to pay $483.78 each month for telephone services at a Rural Development Administration office in Levelland, Texas, even though the office closed in March 1994—over a year ago. According to staff who worked at the office until it closed, USDA’s lease on the building was discontinued in March 1994 and all telecommunications devices, such as telephones, were removed. However, no one terminated the telephone service at this office or followed up with the vendor to be sure that the account was closed. Consequently, USDA has so far paid about $6,200 for services being provided to an unoccupied building.

Offices that are being closed require a detailed analysis of billing records and an inventory of telecommunications lines and services. The analysis and inventory are essential for preparing orders for termination of services. Experience has shown that termination orders should be followed through several billing cycles to ensure that termination actually occurred.

However, as the Levelland office case illustrates, unless these steps are fully implemented at each office to be closed, USDA could incur thousands of dollars in vendor charges for services that are no longer needed.

Options Available for Reviewing Commercial Telephone Bills

OIRM officials told us that, although they have done so for services and equipment acquired under FTS 2000, they have not established a call detail program or prepared guidance on what options exist for reviewing commercial bills because these bills are handled differently. Bills for commercial carrier services are sent directly from the carriers to NFC where the bills are processed and paid. NFC receives thousands of bills in paper form each month and, in most cases, does not forward copies to the agencies for verification of charges.

While handling thousands for paper bills each month is a laborious task, it does not preclude agencies from reviewing commercial carrier bills or absolve OIRM of its responsibility to provide agencies proper guidance. According to NFC officials, agencies can obtain bills from NFC upon request. For example, a Forest Service office we visited recently began requesting monthly bills from NFC for review. At the time of our visit, the office reported that it had recently found a $1,400 overcharge for commercial services. After notifying the telephone company of the inaccurate bill, the charge was removed. Employees at the office also told us that several similar billing mistakes had been identified over the previous few months since they began to verify bills and estimated that about $10,000 annually could be saved at just this one site by reviewing bills. However, according to NFC records, during the month of April 1995, only 80 out of about 23,000 commercial carrier bills had been requested by USDA agencies for review.

Besides requesting specific bills from NFC, agencies have other options for obtaining bills for review. For example, NFC requires USDA agencies to set limits on bills and notifies agencies when bills exceed these limits. However, OIRM has not provided USDA agencies with guidance on setting limits and NFC officials reported that agencies often deliberately set these limits at unreasonably high dollar levels to avoid having to review bills. Consequently, many bills never exceed the limit and few are reviewed. NFC officials also noted that they regularly select a sample of about one percent of the bills and send them to the agency for review. However, these officials reported that agencies do not confirm that they reviewed the bills and found them to be accurate.

Without effective telecommunications policy and guidelines, such as guidance to establish a call detail program for verifying charges by private vendors, USDA agencies lack the management direction they need to institute effective management controls over telecommunications resources. In this regard, USDA agency officials cited the lack of guidance as a key problem, noting that they were often unaware of telecommunications management requirements or what such practices would entail. OIRM’s Associate Director of Policy agreed that USDA’s telecommunications policy has not been comprehensive enough to ensure that agencies have the necessary policy guidelines to effectively manage telecommunication resources. The Associate Director also stated that OIRM recognizes this problem and plans to develop additional agency guidance.

Oversight of Agencies Telecommunications Management Has Not Been Adequate

To monitor agencies’ management of IRM resources, including telecommunications, USDA established its IRM Review Program in accordance with federal requirements for conducting periodic reviews of IRM activities. USDA’s “IRM Review Program” is intended to (1) ensure that agencies comply with governmentwide and departmentwide IRM policies, regulations, rules, standards, and guidelines, (2) ensure that agencies efficiently acquire and effectively use resources, and (3) determine whether agencies’ controls over and reviews of their IRM resources provide effective management oversight. USDA policy states that OIRM’s Program Review Standards Division (PRSD) is required to conduct periodic selective reviews at each of USDA’s 29 agencies to validate the management of IRM and telecommunications resources, assure the Secretary that IRM policy is working as intended, and recommend agency improvement.

However, PRSD conducts very few IRM selective agency reviews, and in the cases when reviews were performed, agency management of telecommunications resources was not adequately addressed. For example, since 1990 PRSD has conducted only five selective agency reviews, of which only one addressed telecommunications management, but did not evaluate whether (1) adequate inventories of equipment and services had been established and annual surveys were conducted, (2) the acquisition of services are monitored to avoid redundancies, and (3) FTS 2000 and commercial telecommunications charges are verified to control costs.

By not conducting reviews, OIRM has no assurance that USDA agencies are following federal regulations or departmental policy, such as using mandatory services, or are making cost-effective use of telecommunications resources and sharing resources when there are opportunities to do so. The General Services Administration (GSA), which periodically reviews federal agencies’ IRM activities, and USDA’s Office of Inspector General have previously raised concerns about USDA’s inadequate agency review program. After reviewing USDA’s IRM program in 1990, GSA reported that OIRM needed to be more proactive and did not place adequate emphasis on performing agency reviews. In 1994, after returning to review USDA’s IRM program, GSA reported that OIRM had not made sufficient progress to improve its IRM selective review program. In 1993, USDA’s Office of Inspector General also reported the need for OIRM to perform IRM reviews.

PRSD’s Chief agreed that OIRM needs to conduct more selective reviews. According to this official, OIRM plans to have the ALOs develop IRM review proposals for selective reviews and participate on review teams for agencies.

USDA Actions to Strengthen Telecommunications Management Fall Short

Senior OIRM officials recognize the need to improve telecommunications management across the Department. To make improvements, OIRM has several initiatives either planned or underway. For example, in response to our April 1995 report, OIRM has shown more leadership on efforts to consolidate and optimize USDA’s FTS 2000 telecommunications services. Specifically, OIRM developed and issued policy requiring component agencies to order and use optimum service configurations and consolidate service access and met with USDA senior managers and has begun a process to systematically identify sites across the Department where FTS 2000 services could be consolidated and optimized.

With respect to strengthening controls over how telecommunications resources are acquired and managed by agencies, in early 1995 OIRM used an existing contract to begin developing a life-cycle management process for all IRM resources. OIRM’s Associate Director for Policy believes this initiative should provide agencies with more direction on what management practices are expected USDA-wide and therefore should ultimately improve management of telecommunications. In addition, USDA plans to collect inventory information and has begun investigating the possibility of establishing an electronic billing process to provide agencies with commercial call detail information for review and verification.

OIRM’s Initiatives Will Not Fully Resolve Telecommunications Management Weaknesses

OIRM’s initiatives are encouraging and, if fully carried out, they should generate departmentwide benefits. However, these efforts will not fully resolve the widespread telecommunications management weaknesses we found. This is because OIRM’s initiatives do not focus on the root causes of the weaknesses: a lack of comprehensive policy and implementing guidelines, and inadequate oversight of the agencies’ telecommunications management activities. For example, USDA has opportunities to save millions under its initiatives to consolidate and optimize FTS 2000 telecommunications services. However, although OIRM has prepared and issued a new policy requiring agencies to consolidate and optimize FTS 2000 services, OIRM has not (1) provided the agencies with specific guidelines for implementing these policies, such as procedures for regularly monitoring telecommunications purchases to consolidate services when it is cost-effective to do so or (2) devised a method for reviewing agency activities to ensure that this policy is effectively carried out. Consequently, agencies will likely continue making telecommunications purchases as they have in the past and perpetuate the use of redundant and duplicative telecommunications services.

Likewise, OIRM and the farm service agencies have begun collecting inventory information at sites scheduled for consolidation under the Secretary’s plan to establish Field Office Service Centers. While we agree this step is needed, OIRM has not defined how this inventory information will be updated and managed after it is collected. Moreover, according to OIRM’s Associate Director for Operations, there are no plans to advise the agencies’ senior managers about requirements to conduct regular surveys of telecommunications resources or to assist agencies in maintaining inventory information needed for implementing fundamental telecommunications management controls. By not doing so, it is highly unlikely agencies will take the initiative on their own to begin obtaining and maintaining inventory information that is essential to planning and managing resources.

OIRM has also not developed any action plans for providing guidance to USDA agencies to help them establish billing review practices. Although OIRM has made FTS 2000 billing data available for agency review, this information does not include commercial carrier bills for millions of dollars in services. OIRM officials, who have investigated electronic billing opportunities with several commercial carriers, have no plan for providing such capabilities nationwide and OIRM has done little to establish interim guidelines and procedures for agencies to follow to request and review paper bills on a periodic basis.

OIRM’s Associate Director for Policy agreed that these initiatives alone will not be enough to correct shortcomings in the Department’s management of telecommunications. However, this official noted that OIRM has just begun an effort to define a telecommunications management program for the Department, which he believes will provide improved telecommunications guidance to the agencies. This official also added that USDA needs to modernize its IRM program, including instituting performance measures to evaluate the agencies’ management practices and then holding the agencies directly accountable for needed improvements.

We agree with OIRM’s Associate Director that performance measures and accountability are critical for improving management of telecommunications resources. In May 1994, after reviewing how leading public and private organizations improved mission performance, we reported that increasing line accountability and involvement works because it immediately focuses information management decision-making on measurable mission outcomes of strategic importance. However, before setting measures and increasing accountability, an organization needs to first gain an understanding of its current performance, telecommunications systems and services spending, and major information management problems.

OIRM has not completed a thorough, systematic review of the agencies’ current telecommunications management practices to determine what management deficiencies exist and the reason for these deficiencies. Without such a review, OIRM does not know what actions are necessary to fully resolve management weaknesses, articulate what management practices are expected, and then define who is accountable for these processes. In 1991, the Department of Defense (DOD) established a program to analyze its communications management deficiencies and develop ways to solve those deficiencies. The goal of this program was similar to USDA’s initiatives—to improve communications management processes. We reported that for DOD’s effort to succeed, besides analyzing management deficiencies, the organization must (1) clearly articulate how telecommunications management processes are to be conducted DOD-wide and (2) precisely define the roles and responsibilities of all components involved in the telecommunications business and management processes.

OIRM’s efforts to define a telecommunications management program for the Department and establish an IRM life-cycle management program have potential for sustained departmentwide management improvements if developed and implemented properly. However, because OIRM is in the early stages of these efforts, it is unclear what impact these will have on resolving the management weaknesses we found.

Nevertheless, USDA’s failure to cost-effectively manage its annual $100 million telecommunications investment constitutes a material internal control weakness under the Federal Managers’ Financial Integrity Act of 1982 (31 U.S.C. 3512(b)and(c)). As previously discussed, federal regulations require agencies to establish inventories of telecommunications resources to, among other things, identify resources that are outdated or no longer used and ensure that agencies pay for only those resources that they use. These regulations also require agencies to establish adequate management controls to ensure the cost-effective use of telecommunications resources and detect possible misuse of government-provided FTS 2000 and commercial long-distance services for which they are charged. Because USDA does not maintain inventories or have adequate management controls established over its telecommunications resources and expenditures, the Department continues to pay millions for telecommunications services that are unnecessary or never used and equipment that is outdated or no longer needed.

Networks Are Not Planned to Support USDA’s Information and Resource Sharing Needs

USDA has many heterogeneous, independent networks acquired and developed over time by USDA agencies. As discussed in chapter 1, these “stovepipe” systems make it difficult for agencies to share information necessary to address complex, cross-cutting issues and effectively execute USDA programs. Despite the need to address this problem, USDA’s agencies continue developing their own networks that are often redundant and perpetuate information sharing problems rather than resolve them.

This is allowed to occur because OIRM continues to approve agencies’ plans for new network systems without (1) determining what information sharing needs USDA agencies have and what opportunities exist to share other agencies’ existing or planned networks and (2) ensuring that the planned networks adequately address the need to share information and resources. Consequently, USDA spends millions of dollars developing networks that do not make efficient use of the Department’s telecommunications resources and cannot support information sharing without costly modifications.

Agencies Plan Their Own Networks Without Considering Information and Resource Sharing Needs

Increasing demands for efficiency and for collaborative agency work on complex agricultural and environmental issues prompted the former Secretary to call for integrating networks and systems to increase data and resource sharing among agencies. Also, federal law requires that (1) USDA reduce expenses by jointly using resources, such as telecommunications services and equipment, at field offices where two or more agencies reside and (2) whenever USDA procures or uses information technology, it does so in a manner that promotes computer information sharing among agencies of the Department.

However, USDA agencies continue to plan and acquire their own costly new networks without incorporating requirements for sharing information among agencies. Also, agencies overlook opportunities to share resources because they independently design, build, and operate their own networks without considering whether other USDA agencies’ existing or planned networks would meet their communication needs.

For example, the Forest Service plans to spend almost $1 billion modernizing its information technology, part of which will be spent establishing a new agencywide network. However, specific requirements for sharing data with other agencies and how these requirements will be met are not addressed in planning documentation. Although Natural Resources and Conservation Service (NRCS) officials told us they could benefit from the exchange of ecosystem and natural resources information with the Forest Service, current plans do not address these needs.

In another example, the Animal and Plant Health Inspection Service (APHIS) plans to spend about $267 million modernizing its technology, which includes acquiring a new network that provides connectivity among its offices. These network plans do not take into account that APHIS offices are often collocated with other USDA agencies at field sites throughout the country. Instead, APHIS plans to acquire its own network to connect over 1,200 agency office sites, rather than exploit opportunities for sharing with other agencies’ existing or planned networks. Therefore, USDA risks losing an important opportunity to reduce communications costs by consolidating network resources.

In addition to the Forest Service and APHIS, other USDA agencies have developed or plan to develop their own networks. These include:

Over 2,500 field service centers—which house the Consolidated Farm Service Agencies, the Rural Housing and Community Development Service, and the Natural Resources and Conservation Service—are to be interconnected by a new $90 million network over the next 3 years.

During 1994, the Agriculture Marketing Service completed integrating 113 field offices and its Washington headquarters into a single network.

The National Agriculture Statistics Service is connecting 43 state statistical offices and the Washington and Fairfax headquarters via local area and wide area networks.

The Agriculture Research Service is providing local area networks in each of its 8 area offices and 122 research sites and plans to link these LANs via dedicated lines between area offices and dial-up access at the research centers.

Like the Forest Service and APHIS, these agencies are planning their own new networks without considering departmentwide interagency data and resource sharing needs. For example, each of the agencies listed above, except for RECD and NRCS, participate in USDA’s Integrated Pest Management Program to coordinate the Department’s research and extension programs with customers who implement pest management practices. This cross-cutting program requires the agencies to exchange information on pesticide use and research. However, the agencies’ network plans do not address this requirement. Therefore, the interagency sharing that must take place to consolidate this information for customers at a farm service center location will not occur. As a result, customers will be unable to obtain the information they need on pest management practices from a single location.

Monitoring Network Planning and Development Does Not Ensure Data and Resource Sharing Needs Are Addressed

Development of individual agency networks, such as the ones discussed above, is allowed to continue because OIRM approves each of these networks separately without having (1) determined whether some or all of the telecommunications services could be provided by other agency networks, (2) determined what information sharing needs USDA agencies have and what opportunities exist to share resources and (3) ensured that the planned networks adequately address these needs.

Besides its responsibilities for establishing USDA-wide telecommunications policy and overseeing telecommunications resources (discussed in chapter 2), OIRM is also required to review and approve agency IRM strategic plans and information and telecommunications technology acquisition plans. Among other things, such monitoring is necessary to ensure that agencies plan and acquire telecommunications networks cost-effectively and in accordance with departmental needs. Unlike PRSD’s IRM Review Program that is supposed to validate management of existing IRM and telecommunications resources, the ALO program and technical approval process provide OIRM with direct involvement in agencies’ IRM projects as they are being planned.

OIRM monitors agency IRM activities under the ALO program and Technical Approval process. OIRM formed its ALO program to improve coordination of agency IRM planning across the Department. Among other things, this program is intended to help ensure that agencies plan their use of information and telecommunications technology to meet departmental needs. However, the OIRM manager for this program told us that ALOs do not review agencies’ network plans to ensure that they incorporate information sharing needs and network sharing opportunities.

In addition, OIRM reviews of component agencies’ acquisition plans under USDA’s technical approval process have not ensured that data and resource sharing needs are being effectively addressed. For example, OIRM staff responsible for technical approvals told us that they evaluate proposed procurements individually and do not review them to assess whether or not data sharing requirements and opportunities to share network services among agencies have been addressed before approving acquisitions.

OIRM’s Associate Director for Policy, who has responsibility for USDA’s ALO program and also the technical approval process, told us that OIRM needs to do a better job determining whether data sharing needs and resource sharing opportunities are adequately addressed by agencies as part of ALO and technical approval staff monitoring activities. The Associate Director noted that in most cases, however, these staff often cannot effectively make such determinations because they lack detailed information describing agencies’ data sharing requirements and the composition and current configuration of all existing agency networks. According to this official, OIRM and the agencies have not taken sufficient steps to obtain the information that defines data sharing requirements and identifies what networks exist. Further, this official stated that OIRM needs to enhance staff expertise in telecommunications to improve monitoring activities.

Although the Associate Director acknowledged that more needs to be done by OIRM, he said that the Office has taken an important step by developing USDA’s strategic telecommunications plan. The plan, issued in September 1993, called for integrating existing USDA agency networks to achieve interoperability and enable agencies to share data where they need to and share resources where they can. According to the plan, OIRM, in cooperation with USDA agencies, would undertake initiatives that include (1) defining interagency data sharing requirements, (2) identifying all existing agency networks, and (3) aggregating networks and other telecommunications resources where opportunities exist for cost savings.

However, at the conclusion of our review, little progress had been made by OIRM and the agencies to carry out the Plan’s initiatives and gather detailed information necessary for identifying data sharing requirements and network sharing opportunities across the Department. Progress had been delayed because OIRM and the agencies have not yet developed a strategy for carrying out this critically important work. However, as mentioned in chapter 2, OIRM and the agencies have recently made some progress identifying opportunities to share existing network resources at some collocated agency office sites, such as USDA’s headquarters offices, and have begun to act on these opportunities.

Continued Development of Individual Agency Networks Poses Costly Risks

OIRM is continuing to approve individual agency networks without determining whether agency network plans meet departmental information and resource sharing goals. This poses costly risks to USDA. First, because agencies have planned their new networks separately and no one has ensured that these efforts are properly coordinated, the agencies may install new communications lines and circuits that overlap or are redundant, resulting in unnecessary costs. For example, collocated agencies at some offices in Kansas City, Missouri, and Washington, D.C., were wasting about $41,000 per year because they were maintaining networks with dedicated transmission service lines that were redundant or unnecessary. This occurred because the agencies acquired these circuits separately without identifying opportunities to share existing circuits with other collocated agencies. Following our April 1995 report, OIRM took action to eliminate these redundant or unnecessary lines.

Also, by allowing agencies to continue to develop networks without assurance that they incorporate data sharing requirements, USDA may need to spend millions in the future making modifications to interconnect networks so they can exchange data. For example, a May 1994 reportdeveloped for the National Institute of Standards and Technology noted that over the past 20 years organizations have evolved to support a wide variety of networks that cannot support required data exchange capabilities. The report states that attempts by organizations to interconnect their incompatible networks after the fact—rather than planning for network interface requirements—typically produced expensive but unsatisfactory results, characterized as “functionally disparate islands of technology.”

Conclusions, Recommendations, Agency Comments and Our Evaluation

Conclusions

USDA lacks the basic telecommunications inventory information and management controls necessary to properly plan and manage telecommunications resources. Consequently, the Department has wasted millions of dollars by not making cost-effective use of the $100 million it spends each year on these resources. This is because OIRM has not demonstrated effective departmentwide leadership by providing USDA agencies with the guidance and oversight they need to help them ensure that the Department’s telecommunications resources are used effectively and prudently. Without sufficient telecommunications guidance and oversight, many agencies have not established the fundamental management controls necessary to ensure that USDA does not (1) acquire separate telecommunications equipment and services that are redundant and unnecessary, (2) pay for leased equipment that is not used and for services billed but never provided, and (3) use more expensive commercial services than those services already provided under FTS 2000.

Although OIRM is aware of these problems, which are long-standing, it has done little to address the agency management shortfalls that allow these problems to persist. Until OIRM (1) provides the guidance and direction necessary to help USDA agencies establish adequate management controls and (2) takes additional actions to oversee that agencies effectively implement such controls and other telecommunications requirements in compliance with federal and departmental policies, the serious and widespread problems we found are likely to continue.

Further, if USDA is ever to successfully share information whenever and wherever it is needed, the Department must prevent agencies from planning and building their own stovepipe networks. However, because OIRM has not fulfilled its departmental responsibility to identify agencies’ information sharing needs and determine with the agencies how to address these sharing requirements, OIRM cannot ensure that new agency networks are compatible. Therefore, USDA risks wasting millions more building new networks that are redundant and may not provide the capabilities necessary for sharing information among agencies.

Recommendations

We recommend that the Secretary of Agriculture report the Department’s management of telecommunications as a material internal control weakness under the Federal Managers’ Financial Integrity Act. This weakness should remain outstanding until USDA fully complies with federal regulations for managing telecommunications and institutes effective management controls.

We also recommend that the Secretary of Agriculture direct the Under Secretaries and Assistant Secretaries to immediately conduct —in cooperation with USDA’s Chief Financial Officer, the National Finance Center, and OIRM—a one-time review of commercial telephone bills for accounts over 3 years old to identify instances where USDA may be paying for telecommunications services or leased equipment that are unnecessary or no longer used. Further, all accounts associated with any USDA office that has closed or moved within the last 3 years should also be reviewed to identify telephone services that private vendors may still be providing to closed offices. On the basis of this review, USDA should (1) take appropriate action with vendors to disconnect any unnecessary or unused telecommunications services and terminate leases for equipment no longer needed or in use by agencies and (2) seek recovery of expenditures for any vendor charges deemed inappropriate.

The Secretary should also direct the Under Secretaries and Assistant Secretaries to establish and implement procedures for reviewing telecommunications resources at offices USDA plans to either close or relocate to ensure that (1) all unneeded telecommunications services are terminated promptly and vendor accounts closed and (2) telecommunications equipment is properly accounted for and reused where it is practical and cost-beneficial to do so.

We further recommend that the Secretary of Agriculture direct the Assistant Secretary for Administration to take immediate and necessary action to address and resolve the Department’s telecommunications management and network planning weaknesses. At a minimum, the Assistant Secretary should require the Office of Information Resources Management to revise departmental policies to require USDA agencies to establish and maintain agencywide telecommunications inventories that contain, at a minimum, circuit information, equipment and service types, network usage levels, costs, and other information agencies need to effectively manage and plan telecommunications resources in accordance with federal requirements; develop additional departmental policy requiring agencies to establish management controls over the acquisition and use of telecommunications resources and assist agencies in carrying out these requirements by completing a systematic review of the agencies’ current telecommunications management practices to (1) identify and correct telecommunications management deficiencies that exist and (2) establish an agency telecommunications management program that sets performance expectations over agency telecommunications activities and assigns responsibility and accountability necessary to ensure these activities are effectively carried out; provide USDA agencies with explicit guidelines that include, at a minimum, procedures to (1) monitor acquisitions of telecommunications services and equipment and coordinate purchases with other agencies to ensure that resources are cost-effectively obtained and (2) implement call detail programs and other necessary procedures to regularly review vendor-provided bills for telecommunications services and leased equipment to verify the accuracy of these charges and ensure the proper use of FTS 2000 and other government-provided resources and services; strengthen oversight by conducting periodic reviews of agency telecommunications management activities in accordance with federal requirements to ensure that (1) inventories of telecommunications equipment and services are properly maintained, (2) sufficient management controls exist over telecommunications resources and expenditures, and (3) redundant or uneconomical services are eliminated; determine, with assistance from the Under Secretaries and Assistant Secretaries for USDA’s seven mission areas, interagency information sharing requirements necessary to effectively carry out the Department’s cross-cutting programs and include these data sharing requirements in departmental and agency strategic IRM and telecommunications plans; enhance the ALO and technical approval programs by increasing the technical focus of reviews of agency telecommunications strategic plans and network acquisition plans, and providing explicit implementing guidance to ensure that information sharing requirements and opportunities to share network resources are identified; and preclude USDA component agencies from developing networks that do not address departmentwide sharing needs by requiring that OIRM technical approvals be made contingent on the component agencies having considered and sufficiently addressed information sharing requirements and opportunities to share network resources.

Agency Comments and Our Evaluation

USDA’s Assistant Secretary for Administration provided written comments on a draft of this report. The Assistant Secretary agreed with most of our recommendations, noting that the draft report contained many excellent recommendations which were well received by the Department.

The Assistant Secretary stated, however, that he disagreed with two of our recommendations. Regarding our recommendation to determine interagency information sharing requirements, the Assistant Secretary stated that USDA’s existing policy is adequate to meet departmental requirements. This statement is not consistent with the facts. USDA’s written policy does not require OIRM and the component agencies to identify the interagency information sharing requirements that must be met to effectively and fully carry out cross-cutting programs. Therefore, this recommendation remains unchanged.

The Assistant Secretary also disagreed with our recommendation to enhance ALO and technical reviews of agency telecommunications plans and activities, noting that USDA’s ALO and selective review programs are not technical functions. However, when the ALO program was established, USDA told the Congress that ALOs would perform the in-depth tasks necessary to improve system compatibility and data sharing across agencies and would strengthen coordination of the agencies’ telecommunications projects. Further, the report addressed technical reviews, not the selected GSA reviews USDA discusses in its comments. We revised the report to clarify this, but the recommendation remains unchanged.

The Assistant Secretary also raised questions about how much money is wasted due to ineffective departmental management and planning of telecommunications. The dollar amounts included in our report are based on USDA documentation and on interviews with USDA’s OIRM staff. For example:

We obtained USDA commercial telephone billing records, which are maintained at NFC, showing that USDA pays tens of thousands of dollars each year to lease telephone equipment that is either no longer used or cannot be located. We also obtained billing records showing that USDA pays more than it should because agencies fail to make long-distance telephone calls using available FTS 2000 services and fail to terminate telecommunications services at offices being closed.

OIRM’s Telecommunications Services Division staff, who are responsible for identifying opportunities to consolidate telecommunications, told us USDA could save as much as $15 million to $30 million annually by eliminating redundant commercial telecommunications services and by sharing resources, and save as much as $2 million each year by using FTS 2000 to make intra-LATA telephone calls.

We held numerous meetings with OIRM and NFC staff during our review in which these amounts were discussed in great detail. We also included these dollar amounts in the information we provided to the Assistant Secretary, the Deputy Assistant Secretary, and the OIRM Director during an exit conference held with these officials on July 12, 1995. At that time, we also provided copies of the billing records that contained the dollar amounts we cite in the report to USDA’s Deputy Chief Financial Officer and the NFC Director, so the Department could discontinue payments for leased equipment and services that are not being used.

Finally, the Assistant Secretary said the draft report did not give USDA sufficient credit for OIRM actions recently taken to improve departmentwide telecommunications management. The report discusses each improvement initiative undertaken by OIRM that we could substantiate with available USDA documentation.

The Assistant Secretary’s written comments and our response are provided in appendix I.