United States General Accounting Office

GAO Report to Congressional Committees

February 1999 ACQUISITION

REFORM

NASA’s Internet Service

Improves Access to

Contracting

Information

GAO/NSIAD-99-37

GAO United States

General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-280239 

February 9, 1999

Congressional Committees

In response to the Federal Acquisition Streamlining Act (FASA) of 1994, we

reviewed the National Aeronautics and Space Administration’s (NASA)

Acquisition Internet Service (NAIS).1

 Specifically, we determined whether

NAIS was an effective mechanism for disseminating procurement

information to industry, including small businesses. In addition, we

identified the status of efforts to develop a governmentwide electronic

procurement information system similar to NAIS.

In 1993, NASA began an initiative called the Midrange Pilot Program for

procurements between $25,000 and $500,000.2

 The purpose of the program

was to test new processes and tools to streamline the acquisition process

while maintaining or improving the quality of procurements. A key

objective of the program was to develop and test an alternative to paper

documents as the primary means of communicating procurement

information to prospective contractors. After researching available

options, NASA chose to implement an Internet-based electronic

procurement notice and publication system, which ultimately became NAIS.

In December 1996, the Department of Commerce, in partnership with the

Government Printing Office, introduced CBDNet, an Internet version of

the Commerce Business Daily (CBD),3

 which is available to the public free

of charge. Many other agencies have developed a variety of other

electronic posting systems to help improve access to, and the visibility of,

their contracting opportunities.

Results in Brief NAIS is a simple, effective, and user-friendly system for disseminating

information on contract opportunities. NAIS has contributed to the

development of a more standardized and streamlined acquisition process

at NASA and provides a central electronic source of procurement

information from NASA’s decentralized facilities. It allows businesses to

obtain procurement information immediately, without waiting for mail

1

P.L. 103-355, section 5062, authorized NASA to conduct a test of electronic notice and publication for

procurements and requires GAO to report on the results of NASA’s test.

2

In fiscal year 1997, NASA increased the threshold to $2 million. NASA officials estimate that 80 to

90 percent of all NASA’s annual contract awards are in the midrange category.

3

The CBD gives a brief description (or synopsis) of federal acquisitions over $25,000. The primary

purposes of CBD notices are to improve small businesses’ access to acquisition information and

enhance competition by identifying contracting and subcontracting opportunities.

Page 1 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

delivery of printed information. Vendors especially like the electronic mail

(e-mail) notification service that automatically sends announcements

about procurements of interest to them. Vendor feedback about NAIS came

primarily from small businesses and was generally positive. Procurement

data showed that offers and awards to small businesses did not change

significantly after NAIS implementation. NASA noted that data limitations

made it difficult to quantify other NAIS benefits.

NASA, the General Services Administration (GSA), and other federal

agencies are working together to develop a single, governmentwide

Internet entry point for information on federal procurement opportunities.

But a number of steps must still be taken and many obstacles remain.

Even if the new system is successfully developed and implemented

governmentwide, current statutory requirements for publication of

procurement notices and minimum waiting periods for mail delivery may

continue to limit the potential benefits of an electronic procurement

information system. The same legislation that encouraged NASA and others

to work together also requires the Office of Federal Procurement Policy to

submit to Congress annual reports assessing compliance with the

requirement to provide direct access to procurement information through

a single governmentwide electronic entry point.

Background In the early 1990s, federal agencies and Congress recognized that doing

business electronically could save resources, streamline the procurement

process, and improve the private sector’s access to federal contracting

opportunities. NASA first used NAIS at its Marshall Space Flight Center in

1994 to post midrange procurement opportunities on the Internet. NAIS

services and features expanded rapidly, and the system began to be used

NASA-wide in 1995. It was the first agencywide procurement information

system on the Internet.

With some exceptions, federal agencies must publish notices of

competitive contracting opportunities above $25,000 in the CBD. To

accommodate timely delivery of notices through a paper-based mail

delivery system, the law generally prescribes a 15-day minimum interval

between publication of a notice in the CBD and the issuance of a

solicitation by an agency. After a solicitation is issued, agencies generally

must give contractors a minimum of 30 days to prepare and deliver bids or

offers. In support of NASA’s test of alternatives to the use of paper

documents for communicating business opportunities to prospective

contractors, FASA authorized NASA to waive the CBD notice and waiting

Page 2 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

period requirements for 4 years (until Oct. 13, 1998) and for up to

$100 million in procurement life-cycle costs.

Congress has supported the use of federal electronic commerce4

 and

electronic access to procurement information. Under FASA, this capability

was to be provided through the Federal Acquisition Computer Network

(FACNET), a governmentwide systems architecture for acquisitions based

on electronic data interchange, which is the computer-to-computer

exchange of routine business documents using standardized data formats.

FACNET implementation proved to be problematic, however, and in 1997,

section 850 of the National Defense Authorization Act for Fiscal Year 1998

replaced the preference for FACNET with a more flexible policy that

promotes the use of electronic commerce techniques whenever

practicable or cost-effective.5

Section 850 also requires that in implementing electronic commerce,

agencies apply standards consistent with nationally and internationally

recognized standards that broaden interoperability and ease of use.

Agency notices of procurement opportunities must be provided in a form

that allows convenient and easy user access through a single

governmentwide point of entry. While CBDNet satisfies the requirement

with respect to notices, neither it nor other currently available electronic

procurement posting systems satisfy the requirement for a single

governmentwide point of entry for access to solicitations and other

procurement information.

The law also requires that the Administrator of the Office of Federal

Procurement Policy (OFPP) submit to Congress yearly reports through 2003

on the progress being made in implementing section 850. The reports must

include a strategic plan for implementation of governmentwide electronic

commerce and an assessment of compliance with the requirement for a

single point of entry.

4

Electronic commerce is conducted using electronic techniques for business transactions, including

electronic mail or messaging, World Wide Web technology, electronic bulletin boards, purchase cards,

electronic funds transfers, and electronic data interchange.

5

We reported previously on numerous concerns about FACNET implementation, including FASA’s

requirements. See Acquisition Reform: Obstacles to Implementing the Federal Acquisition Computer

Network (GAO/NSIAD-97-26, Jan. 3, 1997) and Acquisition Reform: Classes of Contracts Not Suitable

for the Federal Acquisition Computer Network (GAO/NSIAD-97-232, Sept. 17, 1997).

Page 3 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

NAIS Is an Effective

Information Service

Since 1994, NAIS has evolved into a procurement information system with

many operational features that are effective, easy to use, and beneficial to

NASA and vendors. By eliminating many of the steps in the paper-based

process of publicizing contract opportunities and issuing solicitations, NAIS

has contributed to a more streamlined agency acquisition process.

Feedback from NAIS users, particularly small businesses, has generally

been positive. However, the lack of data makes it very difficult to quantify

specific benefits of NAIS, such as the amount of time it actually saves. NASA

exceeded the $100-million ceiling authorized by FASA for the test. After NAIS

implementation, the number of offers per contract and small businesses’

share of contract actions and dollars showed some increases.

NAIS Is User-Friendly and

Works

Since July 1994, NAIS has evolved to provide a broad range of

procurement-related functions and information. For example, in

November 1995, NASA added an on-line procurement reference library. In

March 1997, NASA completed the NAIS Electronic Posting System (EPS),

which enables procurement staff anywhere in the agency to prepare and

post notices and place solicitation files directly onto the NAIS Internet site.

In March 1997, NASA also added an e-mail notification service that

automatically notifies subscribers of new procurement information

available on NAIS.

We found that the system’s operational features were easy to use and

worked as described. Procurement staff demonstrated how they use the

EPS to generate, edit, and post notices and place solicitation files directly

onto the Internet. The EPS automatically formats each NAIS posting, inserts

links to other relevant documents and external information, updates the

business opportunities index library, and constructs and feeds messages to

the NAIS e-mail notification service. Appendix I provides more information

on the development and evolution of the EPS and the approach NASA used

to design, develop, and operate the system.

By going to the NAIS Internet site, users can

• identify, at one location, notices of and solicitations for competitive

business opportunities and awards over $25,000 at all NASA centers;

• search for business opportunities by type of product and/or service, NASA

center, or posting date;

• locate and download documents related to a specific procurement; and

Page 4 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

• view summaries of NASA’s current contracts (with contractor name, value,

obligations, and description) and search current contracts by state,

congressional district, business type, or type of product and/or service.

The e-mail notification service automatically transmits announcements to

subscribers immediately after new procurement information is posted on

NAIS. This allows subscribers to track developments in the solicitation

cycle and learn of new releases. Subscribers can register free of charge by

providing an e-mail address and selecting the types of contracting

opportunities in which they have the greatest interest. Thus, instead of

having to search for information, subscribers can have it selected and

delivered to their desktop electronically. As of July 1998, there were over

7,100 subscribers to the e-mail notification service. According to NASA, user

feedback showed that this service was the most popular feature of NAIS.

NAIS Contributes to a

Streamlined and

Accessible Procurement

System

One of NASA’s goals in establishing NAIS was to streamline and standardize

the agency’s procurement process by using the Internet as the gateway to

contracting opportunities for businesses. NAIS streamlined or eliminated

many of the steps required by the paper-based process for publicizing

synopses of contracting opportunities and issuing solicitations.

The NAIS process involves fewer steps for procurement staff than the paper

process. The EPS, for example, automatically formats notices, performs

edit checks, automatically posts notices on the Internet through the NAIS

server, and transmits them to the Government Printing Office for

publication in the CBD. Using the paper process, procurement staff have to

perform analogous functions manually and must also respond to requests

for copies of solicitations by preparing mailing labels and mailing the

copies. With NAIS, the solicitation files are available for review and

retrieval (“downloading”) through the Internet.

We believe the NAIS process is also easier for contractors to use. Using the

paper-based system, contractors must either scan the CBD or periodically

call or visit each of NASA’s procurement offices to identify contracting

opportunities. In contrast, using NAIS, contractors can search for and view

as many synopses and solicitations as they wish on-line. They can search

by specific commodity, service, or location; view and provide comments

on draft solicitations; and access the Federal Acquisition Regulation, NASA

procurement policies and procedures, and NASA procurement forms.

Page 5 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

NASA officials point to the absence of protests6 from contractors

concerning NAIS and the lack of complaints about NAIS to the NASA

Ombudsman Program7 as evidence that the contractor community

approves of NAIS.

User Feedback Was

Positive

Both businesses and NASA procurement staff have provided positive

feedback about NAIS. Overall, industry responses to specific NAIS

operational features were positive. In a tally of on-line comments from NAIS

users, 87 percent of respondents said they found NAIS to be beneficial. The

respondents identified themselves primarily as small, small disadvantaged,

or women-owned businesses. For the most part, feedback from the

procurement staff was positive, and comments were often similar to those

made by businesses. Federal electronic commerce officials from other

agencies and industry representatives told us that systems with the

operational features of NAIS can benefit both industry and government,

especially when all buying organizations are using the same system.

An on-line NAIS feedback system was established in 1995 to obtain input,

especially from businesses, about the system. The NAIS team8

 also requests

feedback from the procurement staff at NASA centers. This feedback was

compiled in December 1997 and again in June 1998. The team compiles

survey feedback data, develops a consolidated summary of industry

suggestions, and uses the summary to determine which changes and

enhancements should be made.

Lack of Data Makes It

Difficult to Quantify NAIS

Benefits

NASA’s evaluation plan for the Midrange Pilot Program did not include a

separate NAIS evaluation plan. In May 1998, NASA published the NAIS

Business Case, which provides details of current NAIS features, along with

its history, benefits, and funding alternatives. However, NASA did not

collect data on or evaluate individual procurements advertised through

NAIS. The information needed to determine how much time NAIS actually

saved or to quantify other NAIS benefits, such as cost avoidance for the

procurement process, was not available. We did not find any evidence that

6

A protest is a written objection by an interested party to a solicitation or proposed award or award of

a contract. Protests are usually filed either with the procuring agency or with GAO.

7

The program was established in October 1995 to facilitate communications between NASA and

current and potential contractors. It provides offerors, potential offerors, and contractors with a single

point of contact to address their concerns.

8

The NAIS team consists of a small group of procurement and, in some cases, technical staff from each

NASA center and headquarters.

Page 6 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

NAIS had negative effects, and there is substantial qualitative evidence of

the positive impact of NAIS. Appendix II provides more information on our

assessment of the NAIS Business Case.

NASA Exceeded Waiver

Authority’s Dollar Ceiling

Although FASA allowed NASA a 4-year test authority to waive CBD

publication requirements and waiting periods up to a limit of $100 million

in life-cycle costs, NASA did not establish a mechanism to track compliance

with the waiver provision. NASA officials told us that when the waiver

authority was originally requested, they anticipated that NAIS, though not

yet developed, would be used primarily by the Marshall Space Flight

Center, the focal point of the pilot test, and that the $100-million ceiling

would be adequate to cover Marshall’s procurements. However, the

Internet posting capability was rolled out to all NASA centers by April 1995

and was being used agencywide for all competitive procurements over

$25,000 1 month before NASA began waiving the CBD notice requirements in

October 1995.

In early 1997, NASA officials realized that the $100-million ceiling authorized

by FASA for the test would not cover agencywide use of NAIS. In May 1997,

the Administration submitted a legislative proposal to increase the ceiling

for NASA’s use of the waiver authority and extend the test period from 4 to

6 years. Congress did not act on the proposal. NASA used the CBD waiver

authority through the end of the fiscal year–September 30, 1997. NASA

calculated that it awarded approximately 1,585 contracts worth a total of

about $177 million using the CBD waiver authority.

No Significant Changes in

Offers and Awards

Reported After NAIS

Implementation

In an effort to determine the impact of NAIS on the amount of competition

for awards, NASA compared the number of vendors’ proposals received

before and after NAIS implementation. NASA compared the number of offers

received per contract action in fiscal year 1994–the year prior to

agencywide use of NAIS−to the number of offers per contract action in

fiscal year 1997–a year in which all midrange procurements were only

advertised through NAIS. As table 1 shows, the average number of offers

received per acquisition increased from 6.1 to 7.2 after NAIS

implementation. NASA said in the NAIS Business Case that the increase in

the average number of offers received per acquisition was especially

promising because the majority of these acquisitions were set aside for

small businesses.

Page 7 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

Table 1: NAIS’s Impact on Competition

Fiscal year

Dollars in thousands

Procurement activitiesa 1994 1997

Number of contracts awarded 644 396

Offers received from businesses 3,959 2,852

Offers per contract 6.1 7.2

Number of contracts to small businesses 380 252

Small-business share (percent) 59.01 63.64

Dollar value of contract awards $3,296 $930

Dollars awarded to small businesses $431 $283

Small-business dollar share (percent) 13.1 30.4

aCompetitive contract actions over $25,000.

We also calculated and compared the percentages of contract actions and

dollars awarded to small businesses in fiscal years 1994 and 1997,

respectively. Table 1 shows these calculations. The percentage share of

awards to small businesses did not change significantly after NAIS

implementation, but the percentage share of dollars awarded to small

businesses increased substantially in 1997. A NASA official said this was

due in large part to a drop in awards to large businesses and not to any

substantial change in dollars awarded to small businesses.

Efforts Are Underway

to Establish a

Governmentwide EPS

Today most federal agencies use the Internet in some way to advertise

their contracting opportunities. Vendors search all these various Internet

sites for government procurement information. Recent legislation requires

the government to provide direct access to notices of agencies’

requirements and solicitations through a single governmentwide electronic

point of entry. NASA, GSA, and other agencies are testing a multiagency

electronic posting system. Agency officials are concerned that the benefits

of providing direct electronic access to contracting opportunities may be

offset by existing statutory requirements for publication of procurement

notices and minimum waiting periods.

Status of Efforts to

Develop a

Governmentwide EPS

According to the San Antonio Electronic Commerce Resource Center,9

more than 400 different Internet sites provide federal procurement

information. This multitude of sites makes it difficult and time-consuming

9

Electronic Commerce Resource Centers are a network of government-sponsored centers that provide

electronic commerce training and support to the contractor community.

Page 8 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

for contractors to obtain governmentwide information on contracting

opportunities.

The March 1998 report, Electronic Commerce for Buyers and Sellers, A

Strategic Plan for Electronic Federal Purchasing and Payment,

10 described

planned enhancements to the electronic version of the CBD, CBDNet,

which now provides a single point of entry only for procurement notices.

GSA and NASA approached the Department of Commerce and the

Government Printing Office to form an interagency team to implement an

enhanced CBDNet and to make an Internet-based EPS with features

comparable to those of the NAIS system available to the entire federal

sector.11 However, the agencies could not reach agreement on an

enhanced CBDNet implementation approach. Among unresolved issues

were the source and level of funding needed for the project, the costs to

develop and implement the system, and the potential financial impact on

the operating and user agencies.

NASA, GSA, and other members of the Interagency Acquisition Internet

Council12 are developing and testing a pilot multiagency posting system

derived from the NAIS EPS to enable agencies to post notices, solicitations,

and other acquisition-related documentation directly onto the Internet,

where it would be available through a single point of entry. GSA is leading

this effort and spent over $400,000 in the past year developing this

multiagency EPS software. In October 1998, GSA began using the pilot

software as the agency’s single EPS. If testing demonstrates that the

multiagency EPS is capable of providing effective access to notices and

solicitations through a single point of entry, the Administrator of the Office

of Federal Procurement Policy will consider designating it as the single

governmentwide point of entry required by section 850.

10Prepared by the Electronic Processes Initiatives Committee of the President’s Management Council

and submitted to Congress in response to the requirement in section 850 of the Fiscal Year 1998

Department of Defense Authorization Act for a strategic plan for the implementation of a

governmentwide electronic commerce capability.

11The Department of Commerce is responsible for keeping the business community informed about

federal procurement actions through publication of the CBD. In 1961, the Government Printing Office

assumed CBD publication and distribution functions. However, all editorial functions remained within

the jurisdiction of the Department of Commerce.

12The Council was established in May 1996 to seek and promote ways to optimize the use of the

Internet in streamlining the federal acquisition process and increasing communication of federal

acquisition-related information.

Page 9 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

Challenges to

Governmentwide

Implementation

The difficulty of establishing a single governmentwide electronic point of

entry for federal procurement opportunities was emphasized by several

agency officials, who said they were concerned about the costs involved in

developing, operating, and maintaining such a system; cost sharing among

different agencies; and the linkages that need to be established with

existing agency systems through the Internet. They also questioned how

and when decisions would be made concerning who would lead such a

project, who would be responsible for operating and maintaining the

system, and who would mandate its use by agencies.

Agency officials also said that the benefits of a governmentwide electronic

notice capability could be enhanced through changes to the current

statutory minimum waiting periods for notices and solicitations, which

were fashioned around a paper-based process. Acquisition officials noted,

in particular, that the law requires a 15-day waiting period after

publication of a notice before a solicitation can be issued, even though the

Internet enables agencies to publicize notices and provide solicitation files

simultaneously, giving contractors immediate access to these documents

and saving time in the procurement process. Agency officials said that

changing these requirements could increase the benefits of using

electronic commerce. OFPP plans to study what type of legislative action

may be needed to enhance the benefits of electronic publication of

business opportunities.

Conclusions NASA has shown that it can successfully use NAIS to effectively disseminate

solicitations and other procurement information, giving contractors easy

and immediate access to business opportunities at each of its

geographically dispersed procurement offices. Feedback from users has

been positive, and the system has allowed increased standardization of the

procurement process agencywide. However, benefits such as the time

savings obtained through use of the system cannot be readily quantified

because of data limitations. While NASA believes that NAIS has had a

positive impact on competition, this conclusion was based on only 2 years

of data.

NASA’s on-line electronic posting system was the model for the pilot

multiagency EPS software that is being tested by GSA, NASA, and several

other agencies. If successful, this EPS may be considered for

governmentwide expansion and designation as the single governmentwide

point of entry for both notices and solicitations. While the March 1998

report assessing electronic commerce initiatives discussed enhancements

Page 10 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

to the CBDNet, it did not discuss the multiagency EPS pilot program. If a

governmentwide EPS is successfully implemented, its benefits potentially

could be enhanced by changing the statutory notice and waiting periods.

Recommendations We recommend that the Administrator of the Office of Federal

Procurement Policy describe the current plan for implementation of a

governmentwide single point of entry to procurement information in its

next annual report to Congress. This plan should take into consideration

the status of and lessons learned from the multiagency EPS pilot and its

potential as the basis for a comprehensive governmentwide electronic

posting system.

We also recommend that the Administrator of the Office of Federal

Procurement Policy determine whether the issues raised by agency

officials concerning procurement notice requirements set forth in statute,

including waiting periods, require legislative action and make

recommendations to Congress on legislative changes, if necessary.

Agency Comments In commenting on the draft of this report, NASA and OFPP generally agreed

with our findings and recommendations. NASA stated that it was proud of

the NAIS staff for pioneering the use of the Internet in federal procurement;

and our review largely corroborated the agency’s assessment of the value

of NAIS.

OFPP stated that NAIS has created the foundation for the ongoing

multiagency pilot of an electronic posting system. If successful, the EPS

will provide easy access to procurement information across the

government through a single point of entry. OFPP was encouraged by the

promising findings identified in the report. OFPP also commented on our

recognition that current statutory requirements for procurement notices

and minimum waiting periods may limit the potential benefits of electronic

commerce and indicated that the feasibility of legislative action would be

studied.

The comments from NASA and OFPP are reprinted in their entirety in

appendix III and IV, respectively.

Page 11 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

Scope and

Methodology

To determine whether the NAIS electronic notice and publication system

was an effective mechanism for disseminating procurement information,

we obtained information about the system’s design, development,

operation, costs, and benefits. We obtained documents from and

interviewed officials at NASA Headquarters in Washington, D.C., and at the

Marshall Space Flight Center (the lead center for NAIS project

management) in Huntsville, Alabama. We also accessed NASA’s

procurement Internet site (http://procurement.nasa.gov) and reviewed the

on-line NAIS operational features available to both public users and NASA

contracting staff to determine whether the system worked as NASA

represented. To evaluate NASA’s claim that NAIS usage had a positive impact

on competition and was beneficial to small businesses, we compared NASA

procurement data for fiscal year 1994 (the year prior to agencywide use of

NAIS) with data for fiscal year 1997. We did not independently verify the

NASA procurement data obtained from the NASA acquisition management

information system.

We also reviewed NASA’s evaluation of the system contained in the

May 1998 NAIS Business Case. We reviewed the data in the NAIS Business

Case, including calculations of cost avoidance and other benefits

attributed to NAIS. We examined on-line industry feedback and narrative

responses from vendors and NASA buyers about NAIS. To determine the

agency’s compliance with statutory requirements, we reviewed the goals,

objectives, and time frames established by FASA for NASA’s test of electronic

notice and publication.

To assess the status of efforts to establish a governmentwide EPS based on

the NAIS model, we asked electronic commerce coordinators from several

federal agencies, federal Electronic Commerce Resource Centers, and two

industry groups for their observations. We received responses from

coordinators at five agencies, two resource centers, and one industry

group. We also interviewed acquisition officials at GSA, the Department of

Commerce, and OFPP to obtain information on interagency efforts to

develop a governmentwide electronic procurement posting system. We

also reviewed the goals, objectives, and milestones established for the

governmentwide electronic commerce program by the National Defense

Authorization Act for Fiscal Year 1998.

We performed our work between January and December 1998 in

accordance with generally accepted government auditing standards.

Page 12 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

We are sending copies of this report to the Administrator, NASA; the

Administrator, OFPP; and the Director, Office of Management and Budget.

We will also make copies available to others upon request.

Please contact me at (202) 512-4841 if you or your staff have any questions

concerning this report. Major contributors to this report are listed in

appendix V.

Allen Li

Associate Director

Defense Acquisitions Issues

Page 13 GAO/NSIAD-99-37 Acquisition Reform
B-280239 

List of Congressional Committees

The Honorable Fred Thompson

Chairman

The Honorable Joseph I. Lieberman

Ranking Minority Member

Committee on Government Affairs

United States Senate

The Honorable John W. Warner

Chairman

The Honorable Carl Levin

Ranking Minority Member

Committee on Armed Services

United States Senate

The Honorable Christopher S. Bond

Chairman

The Honorable John F. Kerry

Ranking Minority Member

Committee on Small Business

United States Senate

The Honorable Dan Burton

Chairman

The Honorable Henry A. Waxman

Ranking Minority Member

Committee on Government Reform

House of Representatives

The Honorable Floyd Spence

Chairman

The Honorable Ike Skelton

Ranking Minority Member

Committee on Armed Services

House of Representatives

The Honorable James M. Talent

Chairman

The Honorable Nydia M. Velázquez

Ranking Minority Member

Committee on Small Business

House of Representatives

Page 14 GAO/NSIAD-99-37 Acquisition Reform
Page 15 GAO/NSIAD-99-37 Acquisition Reform
Contents

Letter 1

Appendix I 

The NAIS Electronic

Posting System

18

Appendix II 

NASA’s Business Case

for NAIS

20

Appendix III 

Comments From the

National Aeronautics

and Space

Administration

22

Appendix IV 

Comments From the

Office of Federal

Procurement Policy

23

Appendix V 

Major Contributors to

This Report

25

Tables Table 1: NAIS’s Impact on Competition 8

Table II.1: Annual NAIS Cost Estimates 21

Page 16 GAO/NSIAD-99-37 Acquisition Reform
Contents

Abbreviations

CBD Commerce Business Daily

EPS Electronic Posting System

FACNET Federal Acquisition Computer Network

FASA Federal Acquisition Streamlining Act

FTE full-time equivalent

GSA General Services Administration

NAIS NASA Acquisition Internet Service

NASA National Aeronautics and Space Administration

OFPP Office of Federal Procurement Policy

Page 17 GAO/NSIAD-99-37 Acquisition Reform
Appendix I 

The NAIS Electronic Posting System

Staff at the National Aeronautics and Space Administration (NASA)

responsible for developing and implementing the NASA Acquisition Internet

Service (NAIS) consisted of a small group of procurement and, in some

cases, technical staff from each NASA center and headquarters. They

adopted an incremental approach to system development and operations.

This approach included the following steps: (1) general requirements were

drawn up, (2) commercially available software options were researched,

(3) a prototype was designed and developed if a commercial solution was

not available, (4) a prototype for testing was installed, (5) requirements

were refined and a small pilot initiated, and (6) the pilot was expanded

and the software solution added to the NAIS system.

NASA’s development of its Electronic Posting System (EPS) is characteristic

of the way the NAIS team applied the concept of incremental design,

development, and implementation of NAIS features. Prior to agencywide

use of EPS, each NASA center had its own unique process for preparing and

posting procurement notices and solicitations on the Internet. Some

centers used word processors while others used some form of automated

document generation system. The completed files were then provided to a

Web curator at the center who would post them on an Internet server and

manually update a readable index of files. Changes to the posting process

occurred spontaneously at various centers with little central control or

agreement. The 10 centers’ differing processes required 10 separate efforts

to implement a single change that had to be agreed on by the larger NAIS

team. These process differences also made it difficult to maintain a

standard “look and feel” when searching out business opportunities. A

vendor was faced with 10 different entry points to search.

NAIS operated for a year in this fashion. Then an ad-hoc group of NAIS team

members began to study ways to make the centers’ posting processes

more uniform. The team concluded that the NAIS posting process should be

centralized to a single server and that the application for posting and

maintaining documents should be standardized by using on-line forms to

be completed by users. NAIS technical team members then designed and

developed the uniform Web posting tools that were first piloted and then

deployed agencywide.

According to NASA, implementation of the EPS eliminated much duplication

among the centers, standardized posting practices across all NASA centers,

and lowered NAIS maintenance costs. The EPS application provides a

common look and feel of acquisition opportunities.

Page 18 GAO/NSIAD-99-37 Acquisition Reform
Appendix I 

The NAIS Electronic Posting System

The incremental, evolutionary approach to deploying an electronic posting

system avoided the costs and delays often associated with attempting a

single solution to standardize disparate, decentralized systems.

Additionally, because much of the system is based on commercial

standards, commercially available software, and standard hardware, it was

easily adopted by each of the centers without major disruptions.

All NAIS features have been built using commercial Internet standards and

Web methodology so that even casual Internet users can easily use NAIS.

For internal procurement staff using standard NASA computer hardware

and software, NAIS tools may include user instructions. For example, the

NAIS EPS has an on-line user’s manual linked directly to the application. For

external users with a wide variety of hardware and software, NAIS features

are based on widely accepted Internet protocols and standards, and the

features are designed to be used without any formal training.

Page 19 GAO/NSIAD-99-37 Acquisition Reform
Appendix II 

NASA’s Business Case for NAIS

In May 1998, NASA completed and published an evaluation of NAIS in a

document called the NAIS Business Case. The document sought to justify

continued funding for NAIS by showing that the system’s quantitative and

qualitative benefits far exceed NASA’s investment. According to NASA’s

assessment, the NAIS tools implemented to date had produced outstanding

results.

We found weaknesses in the evaluation’s methodology. For example, NASA

claimed that the Midrange Pilot initiatives, including NAIS, reduced

procurement leadtimes by 40 percent.1

 We examined the data used by NASA

to support this calculation. In a number of cases, the data showed that the

official purchase requests arrived at NASA procurement offices after

solicitations for the requested products or services had already been

issued. Consequently, we could not verify NASA’s claimed 40-percent

reduction in leadtime. However, it is obvious that the reduction in mail

delays and in the manual steps and efforts needed to post a procurement

notice should contribute to reduced leadtimes.

We also found that the NAIS Business Case’s discussion of the resources

used in developing and operating NAIS was incomplete. It described only

the full-time equivalent (FTE)2

 staff years of effort, both funded and

unfunded, which were devoted to NAIS development and support in fiscal

year 1997 and did not reflect the additional resources such as technical

support, hardware, software, and communications services provided by

various NASA centers.

Because the evaluation only identified NAIS FTE staffing for 1 year and did

not identify hardware, software, and other costs associated with NAIS, we

asked NASA to provide fiscal year estimates of the total costs associated

with NAIS development and ongoing operations. The table below depicts

NASA’s estimates of the staffing, hardware, software, and miscellaneous

costs associated with NAIS for fiscal years 1995 through 1998. Amounts for

fiscal years 1995 through 1997 are estimates, while the amount for fiscal

year 1998 is a projection.

1

Procurement leadtime, broadly speaking, is the time it takes to accomplish a contracting action.

Leadtime can be calculated in a variety of ways and is affected by many variables. Leadtime in the

NAIS evaluation represented the period of time from receipt of a purchase request by the procurement

office to award of the contract.

2

Federal civilian employment is generally stated on an FTE basis. It is the total number of hours

worked (or to be worked) divided by the number of compensable hours applicable to each fiscal year.

Page 20 GAO/NSIAD-99-37 Acquisition Reform
Appendix II 

NASA’s Business Case for NAIS

Table II.1: Annual NAIS Cost Estimates

Fiscal year NASA staff

Contractor

staff

Hardware/

software/misc. Total

1995 $268,620 $56,000 $145,000 $469,620

1996 529,100 108,500 145,000 $782,600

1997 345,090 248,500 185,900 $788,490

1998 488,400 280,000 145,000 $913,400

User Feedback on NAIS

Was Positive

NASA reported in the NAIS Business Case that data from the NAIS on-line

feedback system showed that over 85 percent of those who responded to

the question, “Is NAIS, as a whole, beneficial?,” answered “yes.” We

reviewed summary reports of responses tallied from the NAIS system and

found that of the 2,310 persons answering this question, 2,007 answered

yes (87 percent). The survey respondents identified themselves primarily

as small, small disadvantaged, or women-owned businesses. Overall, we

found both industry and NASA procurement staff comments about specific

NAIS operational features were positive.

We also reviewed feedback responses submitted from 1995 through

June 1998 to an open-ended question about enhancements that would

improve NAIS. As of June 30, 1998, the NAIS team had completed actions or

had actions underway related to 42 of 89 suggestions.

Federal officials and groups representing industry perspectives told us

that the NAIS Business Case properly identified the areas in which both

government and industry may experience time and cost savings through

an effective agencywide EPS. They noted that electronic commerce

systems with the functionality and user-friendly features of NAIS have clear

potential to benefit both industry and government, particularly if all buying

centers are using the same system.

Page 21 GAO/NSIAD-99-37 Acquisition Reform
Appendix III 

Comments From the National Aeronautics

and Space Administration

Page 22 GAO/NSIAD-99-37 Acquisition Reform
Appendix IV 

Comments From the Office of Federal

Procurement Policy

Page 23 GAO/NSIAD-99-37 Acquisition Reform
Appendix IV 

Comments From the Office of Federal

Procurement Policy

Page 24 GAO/NSIAD-99-37 Acquisition Reform
Appendix V 

Major Contributors to This Report

National Security and

International Affairs

Division, Washington,

D.C.

Delores Cohen

Richard J. Herley

Dan Hoagland

Stefano Petrucci

Patricia D. Slocum

Atlanta Field Office Bobby Hall

Office of General

Counsel, Washington,

D.C.

John Carter

(707346) Page 25 GAO/NSIAD-99-37 Acquisition Reform
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100