ANAGEMENTINVOLVEMENT 

. 

• 

-PLANNING 

-CONTRACTING 

-DESIGNING 

-DEVELOPING 

-TESTING 

-1M PLEM ENTING 

-OPERATING 

, 
For we by the Superintendent of Documenta. U.S. Gove:mment 

Prlntinl Offlee. WaahJ.n&tOrl. D.C. 20402 

Stock No. 020-000-00138-1 
FOREWORD c. 

I n most Federal departments and agencies, managers involved in 

planning, executing, and evaluating programs rely heavily on computer-based management information systems for decisionmaking. 

Departments and agencies not only must acquire and maintain these 

systems, especially the financial management information systems, 

but also, as federal programs become larger and more complex, must 

design and develop new systems to provide the essential information 

needed for program control, evaluation, feedback, reporting, and 

day-to-day management. 

Often, however, agencies lack the number of skilled profes· 

sionals necessary to accomplish the large, one-time development of 

such systems. In th is situation, the agencies frequently contract with 

the accounting and management consulting firms, which can give the 

Government valuable, prompt help in designing, developing, and 

implementing the systems. 

Federal agencies' experiences with contracting for financial and other 

systems development have varied. All too frequently, the systems 

have not satisfied agency information needs, were more costly than 

anticipated, or failed to meet scheduled implementation dates. But 

many agency-contractor efforts have met system concept, design, 

implementation, and operation objectives, while keeping agency 

costs reasonable and creating a smoothly functioning system. 

GAO'S INTEREST IN COLLECTING 

LESSONS LEARNED 

GAO is responsible for (1) approving and (2) reviewing in 

operation, accounting systems of the executive agencies. As it reviews the systems, GAO is increasingly concerned not only that the 

systems produce accurate data in accordance with prescribed prin' 

ciples and standards, but also that the information produced is 

accepted and used by operating managers. Actual use is the ultimate 

test. 

I n approving and reviewing Federal agency accounting systemsmany of wh ich are designed and developed with contractor assist-
ance-GAO noticed common problems and difficulties which are not 

being systematically documented so other agencies could learn howto 

avoid them. Similarly, there was no mechanism for telling others about 

successfu I practices. 

BOOKLET PURPOSE 

We prepared this booklet to disseminate the lessons learned by 

many Federal agencies and contractors in designing, developing, and 

implementing management information systems. Although we were 

initially interested in the agencies' development of their accounting 

systems, it became apparent that the lessons apply to the development of all types of management information systems. 

COLLECTING THE LESSONS 

We interviewed officials of many Federal agencies, civilian and 

military, to learn of their experiences and gain insight into the contracting and development processes. Sen ior members of accounting 

and management consultant firms and professional accounting and 

management societies and organizations also contributed their 

experiences and perspectives. The Office of Federal Procurement 

Policy, the General Services Administration, and project managers of 

the Joint Financial Management I mprovement Program provided 

expertise on specific Government policies, practices, and regulations. 

After we had collected the lessons, we prepared a draft booklet 

and obtained comments on it from each official interviewed. We 

conducted six conferences with agency officials and firm managers 

and partners. We also solicited comments from the American Institute of Certified Public Accounts. (See back cover for contribution 

sources.) 

BOOKLET USE 

The chapter format divides the complex contracting and development process. Chapters 4 and 5 assume that an agency has 

decided to use a contractor to develop its computer-based system. 

However, if an agency decides to design, develop, and implement a 

ii 
system in house, lessons about defin ing the problem and planning for 

system change (ch. 3) and contractor/agency performance (ch. 6) are 

still applicable. Throughout the booklet, we assume that the system 

to be developed will be computer-based. 

The scale of system development efforts vary greatly in complexity, size, scope, and cost. Officials and contractors constantly 

emphasized scale when determining which lessons to apply. When 

large systems are acquired, each guidance item may be important. 

We hope that this synthesis of guidelines for the acquisition of 

management information systems, particularly financial management 

systems, will be useful not only to Federal agencies, but also state 

and local governments and accounting and management consultant 

firms that assist government agencies and others in designing, developing, and implementing systems. 

August 2, 1976 

iii 

f...a.~ Comptroller General 

of the United States 

CONTENTS 

CHAPTER Page 

FOREWORD 

, OVERVIEW 

Complete problem definition 1 

Well-qualified and effective project coordinator 2 

Continuous communication and coordination 3 

Maintaining adequate records of agreements and deci-

~ons 4 

Effective planning, timing, and phasing of system 

development 4 

Fair selection of a competent contractor 5 

Management involvement at key points 6 

2 CASE STUDIES 

Case study one 

Case study two 

Case study three 

3 DEFINING THE PROBLEM AND PLANNING FOR 

7 

7 

9 

11 

SYSTEM CHANGE 15 

Determining user requirements 

I n-house development versus contracting 

Project coordinator selection and functions 

Planning the system's scope 

4 PREPARING THE WORKSTATEMENT AND SOLICIT15 

18 

19 

21 

ING CONTRACTORS 23 

Cooperation and communication 

Deta i led work statement 

Request for proposals 

R FP Distribution 

Phased competitive negotiation 

v 

23 

24 

26 

31 

32 
Page 

5 PROPOSAL, EVALUATION, NEGOTIATION, AND 

AWARD 35 

Preproposal conference 

Evaluation 

Negotiation 

Award 

6 SYSTEM CONTRACT PERFORMANCE 

Qu ickly resolve questions and issues 

System design 

Implementation 

Operation 

APPENDIX 

Guidance item listing 

II Responsibility matrix 

ABBREVIATIONS 

GAO General Accounting Office 

R FP Request for proposal 

vi 

35 

36 

38 

40 

41 

41 

42 

45 

47 

51 

58 
CHAPTER 1 

OVERVIEW 

Agency and contractor officials stated that considering certain 

lessons is essential to successfully develop a system. These lessons are 

discussed in th is overview and are referenced to 68 guidance items. 

The referenced guidance items are discussed in detail in chapters 

3 through 6. We have also provided a list of guidance items 1 to 68 in 

appendix I as a reference document. 

Appendix II is a matrix of organization or individual respon· 

sibilities for each guidance item. Recognizing variations in department and agency organizations and responsibilities, we present this 

matrix as a departure point for assigning responsibility for each 

gu idance item. 

COMPLETE PROBLEM DEFINITION 

The most important step in 

developing a system is determining 

its requirements. Not only must 

problems be identified and defined, 

but key agency officials must 

agree on the problems defi ned 

and the scope of the system 

needed. Contractors and agency 

officials state that often too 

linle time and effort are devoted 

to preparing statements identi· 

fying and documenting problems 

with the current system's pro· 

ducts and processes. Agency 

management, system development 

groups, and user groups must 

participate to develop complete 

statements of current problems 

and new requirements. 

1 

SYSTEMO 

DE.VI&'OPMlNT 
For example, if an agency is planning for the development of a 

new financial management system, it should involve its accounting 

office in defining the problem and determining system requirements. 

If the user group is excluded from defining the problem, the definition will probably be incomplete or unacceptable to the user group. 

Basing contracted development on such a problem definition may 

result in a system which is not implemented or which does not meet 

the accounting office's information requirements. 

The task of defining problems is continuous. As problems and 

requirements are defined and refined through communication, 

coordination, and agreement, a series of planning documents is 

generated: 

1. An initial statement of user requirements. 

2. A priority requirement statement. 

3. A detailed work statement. 

4. A request for proposal (R FP). 

5. Contract and contract amendments. 

6. Final system documentation. 

(See related guidance items " 5,20,21, and 50.) 

WELL-QUALIFIED AND EFFECTIVE PROJECT 

COORDINATOR 

Many agencies and contractors view the project coordinator as 

the most important individual in system development and implementation. The project coordinator should be selected early and be a 

well·qualified agency official. He or she should be committed to the 

system change and have the full support of upper management. 

The project coordinator must know the agency and its problems 

and be in contact with decision making officials. He or she must be 

able to gather and maintain the necessary agency resources and 

2 
talents to select, aid, monitor, and direct the contractor in designing, 

developing, and implementing the system. 

The project coordinator's functions and duties are so crucial to 

the success of the project that they should be documented in a 

charter. The coordinator's relationship to other agency officials and 

the contractor should be clearly defined, and any limitation on 

decisionmaking authority should be explained. (See related guidance 

items 11 and 12.) 

CONTINUOUS COMMUNICATION AND 

COORDINATION 

System procurement is complex-many individuals and organizational elements are involved and affected. If the communication 

system does not provide continuous opportunity for input and 

changes, necessary information will be left out. Decisions, commitments, and positions must be coordinated within the agency during 

planning and contracting. 

After the contractor begins to design and develop the system, coordination between agency and contractor personnel becomes critical. 

The project coordinator and team are key to coordinating, and communicating problems, decisions, and progress to agency and contractor personnel. 

(See related guidance items 18, 26, 27, 28, 29, 32, 33, 34, 48, 51, 

53,56, and 57.) 

3 
MAINTAINING ADEQUATE RECORDS 

OF AGREEMENTS AND DECISIONS 

During each project phase alternatives and constraints will be 

considered and decisions and commitments made. Adequate records 

of these should be kept and circulated to assure effective communication and coordination. 

(See related guidance items 2, 4, 13,22,35, and 58.) 

EFFECTIVE PLANNING, TIMING, AND 

PHASING OF SYSTEM DEVELOPMENT 

Most contractors considered inadequate agency planning a cause 

of many system development difficulties. Agencies should begin 

early to plan for system change or development and should plan 

continuously throughout the process. If too little time or personnel 

are devoted to planning, essential details and factors will be overlooked, leading to costly changes in the design and development 

phases, poor system performance, or delayed implementation. 

A key planning consideration is timing the release of the RFP. 

Many agencies allow funding uncertainties to delay the R FP until 

just before the end of the fiscal year. If there is a flood of R FPs, 

contractors must limit the number of proposals to which they will 

respond, thereby reducing competition on a given R FP. Proposals 

prepared at this time may not be as technically responsive or innovative as at an earlier, less rushed time. If an agency's evaluation of 

contractor proposals is hurried because of the compressed schedule, 

selection criteria may be improperly or inadequately applied. 

Proper phasing of system design is necessary for establishing 

control points at which the agency coordinator can evaluate the 

contractor's progress and approve initiation of subsequent phases. 

The phases usually consist of the conceptual, general, and detailed 

designs. 

4 
Implementing the system in segments offers additional agency 

control. As modules are developed, tested, and integrated, problems 

can more easily be identified and corrected. Both the contractor and 

the agency should be involved in implementation to insure that the 

system does what it was designed to do. A special effort should be 

made to insure that all changes during phased implementation are 

properly docu mented. 

(See related guidance items 3, 6, 7, 14, 15, 16, 17, 54, 59, 60, 61, 

62, 63, 64, 65, and 66.) 

FAIR SELECTION OF A COMPETENT 

CONTRACTOR 

In seeking contractor assistance, the agency is trying to obtain a 

system which meets its information requirements at the lowest cost 

in the least ti me. 

The best method is competitive contracting, which enables the 

agency to obtain alternative system development approaches and 

cost estimates from qualified firms. Agencies must carefully develop, 

explain, publish, and apply criteria for selecting a contractor. It is 

especially important to assign a weight to each criterion. 

Because of complexities and differences in alternate technical 

approaches, conducting competitive negotiations with firms which 

have been found highly qualified is generally the best approach. 

Agencies should base selection of contractors on a combination of 

proposed technical design, staff qualifications, and cost. 

5 
In deciding on the method of contracting-cost reimbursement 

or fixed price-the agency should consider the complexity and 

specific ity of system requirements, division of risk between the 

agency and contractor, and the availability of funds. If the type of 

contract used and the systems development work required are not 

compatible, qualified contractors may not respond to the RFP. In 

addition, relationships between the agency and the successful contractor may be impaired. 

(See related guidance items 23,24, 25,31, 36,31, 39, 40,41,42, 

44, 45, 46, and 41.) 

MANAGEMENT INVOLVEMENT AT K EY 

POINTS 

The final major lesson learned is the importance of early and 

continuous involvement of agency upper management. Management's 

commitment to a system change will set the tone for the development effort. I f management is neutral or unenthusiastic, the system 

development project team will be too. Upper management must 

participate in determining user requirements, selecting and chartering 

the project coordinator, committing resources, selecting the contractor, and making contract decisions. 

Periodically, management should plan reviews and briefings to 

provide project visibil ity, evaluate problems and progress, and make 

key decisions. Management must also make itself available to the 

project coordinator and principal contractor manager. 

Developing successful systems requires close contractor-agency 

teamwork. Management should help maintain good relationships 

between agency and contractor personnel and thus insure the 

achievement of the system's objectives. 

(See related guidance items 8, 9, 10, 19, 30, 38, 43, 49, 52. 61. and 

68.) 

6 
CHAPTER 2 

CASE STUDIES 

Agencies and contractors presented many examples of both 

good and bad experiences in developing management information 

systems. We have chosen the three included in this chapter to illustrate the importance of following the guidance items presented in 

chapters 3 through 6. 

CASE STUDY ONE 

This case study demonstrates the problems that can be encountered, especially in the case of large systems, if good systems 

development procedures are not followed_ 

In 1965 a constituent agency authorized the development of 

three administrative information systems and a general-use data-base 

management system. Although the four systems have been under 

development for several years, none are operating. About $7.7 

million has been spent to develop the systems and acquire large-scale 

computers. 

In 1965 the agency contracted for studies of design concepts 

necessary to develop a staff requirements and personnel information 

system. Then, in 1968, the agency decided to develop the system 

in-house on the basis of the contractor studies. However, the studies 

did not contain information l11!Cessary to justify the decision to 

proceed with the system development. For example, the studies did 

not include agency management's information needs, a description of 

how the system was to help accomplish the agency's programs, or a 

concise and complete statement of system specifications. (Guidance 

item 1.) 

In 1969 the constituent agency decided to develop the generaluse data-base management system-at an estimated cost exceeding 

$700,()()().-without obtaining departmental approval, or evaluating 

the capabilities of other available systems. During the late 1960s, 

many commercially developed systems, as well as a system sPonsored 

by other Government agencies, became available. (Guidance items 5 

and 8.) 

7 
In 1971 the agency authorized development of another comprehensive information system-at an estimated cost of $1.8 

million-without identifying the requirements of operating managers, 

defining specific objectives of the system (user requirements). or 

considering such alternatives as streamlining and improving existing 

systems. (Guidance items 1, 2, and 5.) 

In 1971 the agency authorized development of a logistics 

information system without an underlying study of need or a costbenefit analysis. This project was halted after the agency had spent 

about $118,000 on such efforts as determining what data should be 

included and training personnel in operating the system. (Guidance 

items 5, 6, and 7.) 

In 1971 and 1972 the agency purchased and installed, at a cost 

of $3.1 million, two large-scale computers and related peripheral 

equipment to support the systems discussed above. The agency did 

not critically analyze its data processing workload before buying the 

computers, consider any alternative to buying an additional backup 

computer, nor make sure the systems were ready for operation 

before installing the computers. As a result, when we visited the 

fac ilities the computers were operating at less than 8 percent of 

capacity. 

I n summary: The development cycle has been prolonged. 

Costly equipment has been acquired prematurely. A systems development project was unable to satisfy the user requirements. (Guidance 

items 1, 7, and 17.) 

We believe the following factors were lacking: 

An adequate user requirement study. 

(Guidance item 3.) 

A priority requirement statement. 

(Guidance item 4.) 

A final statement of requirements. 

(Guidance item 8.) 

8 
A qualified, committed project manager. 

(Guidance items 11 and 12.) 

A cyclical, detailed planning process. 

(Guidance items 15 and 16.) 

CASE STUDY TWO 

An agency had grown phenomenally in both the volume and 

complexity of its programs, many of which involve large loans. 

Loan balances were $4 billion in 1968 and $14 billion in 1975, and 

will approach $24 to $30 billion in 1977. Major legislation in 1972 

changed the agency's emphasis from farm development to rural 

community development, including a community facilities loan 

program and a business· industrial loan program. These programs are 

conducted through a nationwide system of 1,750 county offices 

where services are easily accessible to people living on farms and in 

the rural community. 

The new multiple-loan programs obviously have tremendously 

increased the data and information needed by the agency's local 

supervisors to manage the increased loan activity. However, the 

existing management systems in the agency were created on an 

incremental, patchwork basis, were unrelated, and did not provide 

management with the data requ i red to manage the programs effectively and efficiently. 

When the current administrator was appointed 2 years ago, he 

immediately recognized the problem and set out to create a new 

management information system. After 6 months of intense preparation, an RFP was released and given maximum publicity and distribution through the Commerce Business Daily and industry associations. (Guidance item 31.) 

A preproposal conference was conducted shortly after the R FP 

was issued. All questions submitted before the conference were 

answered and replies were sent to all potential contractors. There 

were 136 requests for the RFP, but only 6 firms submitted offers. 

The succinct, clear selection and evaluation criteria included in the 

9 
R FP discouraged unqualified contractors from submitting a proposal. 

Thus, the cost of evaluation and negotiation for the system procurement was reduced. (Guidance items 26, 30, 32, and 35.) 

Key agency and departmental officials from design, user, 

contract administration, and computer system groups attended the 

conference and answered prospective contractors' questions. Adequate discussion time was allowed. (Guidance items 36,38, and 39.) 

A key selection criterion was that the contractor had previously 

developed a system comparable in scope and complexity to that 

required in the RFP. In addition, the personnel who had developed 

the previous system were to be assigned to the current development. 

The evaluation team not only visited the potential contractors' 

offices to verify their statements but also visited the organizations 

for whom they had developed and implemented an operating system. 

I ndepth discussions were held with those organizations to determine 

how well the contractor had performed during the system's design, 

development, and implementation stages and how well the current 

system was operating. (Guidance items 37, 44, and 45.) 

The technical evaluation of the offers was conducted separately. 

The evaluation team ranked contractors in each evaluation areatechnical design; firm and personnel qualifications; price (based on 

detailed cost analysis); and delivery schedules-and assigned total 

points to each offer. In the final negotiations with three offerors, 

sessions were conducted with individual firms, all information was 

kept confidential, and complete records were maintained to explain 

why unsuccessful firms were not selected. (Guid~nce items 40, 41, 

42, 43, 46, and 47.) 

I n the procurement, the evaluation team closely adhered to the 

selection criteria in ranking proposals and selecting the firm. Records 

were kept on the entire evaluation process and the five unsuccessful 

offerors were fully briefed. The two unsuccessful finalist wrote 

letters complimenting the team on the fairness, thoroughness, and 

professionalism of the entire procurement process and particularly 

the evaluative and briefing processes. At the time of our study, the 

system was not yet operational,but indications were that it would 

10 
achieve the objectives of the system development. (Guidance item 

49.) 

CASE STUDY THREE 

During 1972, an agency drastically changed its financial 

management operations by replacing its decentralized, manual, 

obligations-oriented, allotment-type accounting system with a totally 

centralized, automated, cost-based financial management system. 

The plans for the system provided for: 

A unified financial management information and control 

system which includes budgeting, accounting, and data 

processing activities. 

Integration of accounting functions, such as billings, collections, payments, payroll, general ledger, cost reporting, 

obligations reporting, and property records, into a single 

automated system. 

Complete financial data for developing and using cost-based 

operating budgets for internal planning and control. 

Financial data with a consistent basis for comparing the 

performance of similar organizational units and activities. 

Prompt, accurate internal and external financial reports. 

The agency's administrative budget alone totals more than $100 

million, with approximately $55 million appropriated and the 

remaining $45 million collected for services. 

Since most of the agency's funds involve more than one 

division, the financial management system had to provide for budget 

planning and control by organizational unit as well as by budget 

project, budget activity, and fund. The agency is responsible for 5 

appropriated funds, including 6 operating activities,and 43 projects, 

and 16 trust fund fee accounts. I n addition to the administrative 

budget, the agency controls appropriated funds which total more 

than $1.2 billion annually. 

11 
While the old accounting system generally met external reporting requirements, such as those of the department secretary, the 

Office of Management and Budget, and Treasury, it produced few 

internal reports suitable for use by top management in reviewing 

program operations and making financial decisions. Budgeting and 

accounting were unrelated. Decentralization caused problems in 

coordinating and controlling accounting records-reports were late 

and errors difficult to trace and correct. 

Recognizing that this system was grossly inadequate to answer 

the increasing demands for prompt, accurate financial data, the 

agency decided to design a totally new system. During the fiscal year 

1969, it contracted with a firm to develop, document, and install an 

accounting system that would meet its needs, as well as GAO 

standards. 

After 2 years of effort by the contractor and agency employees, 

the new system was approved by GAO and began operating in 1972. 

The new unified financial management system provided for (1) 

complete cost data for cost-based operating budgets, (2) integration 

of related activities, and (3) extensive automation. 

Two comparatively minor problems were encountered in 

implementing the system. 

I ncomplete training was provided; many people found themselves working with a system they did not fully understand. 

(Guidance items 59 and 64.) 

Programming difficulties delayed the planned implementation because additional agency and contractor efforts were 

needed. (Guidance items 65 and 66.) 

However, this system development was successful principally because: 

Planning was started more than a year before a system 

development contract was signed. A well thought out plan of 

action was adopted. Management acceptance and backing of 

12 
the project were obtained early and a well·documented R FP 

was prepared. (Guidance item 14-1 

The system development was directed by a full time project 

coordinator who reported to the agency's deputy adminis· 

trator for management. (Guidance item 12-1 

The contractor and agency personnel worked closely on a 

cooperative basis. (Guidance item 52-1 

Both headquarters and field program offices were involved 

heavily during design phases. Monthly design reviews were 

held with key program staff. (Guidance item 55.) 

Numerous progress reports and drafts of reports to be 

produced by the system were prepared and sent to key 

administrative and program people for comments. (Guidance 

items 55 and 56.) 

A detailed implementation plan including parallel testing was 

developed. The contractor participated fully in implementing 

it. (Guidance items 60, 61,62, and 63-1 

13 
HAS THIS HAPPENED TO Y

l-"'-..I"' --~ ~~ ~~;--_~

~:.~ 

As requested by user 

- - \...~ ~ -. . ,,--....., "-

As desi rned by contractor 

j 

\~ _VJ'~~· . ___ 

As proposed by project 

coordinator 
CHAPTER 3 

DEFINING THE PROBLEM AND 

PLANNING FOR SYSTEM CHANGE 

DETERMINING USER REQUIREMENTS 

Because information needs are constantly changing, agencies 

must develop or modify management information systems. The new 

requirements may arise from within the agency or elsewhere, such as 

from congressional actions or admin istrative reorganizations. As 

agencies establish and implement new internal procedures and 

programs, information processing and reporting requirements will 

usually increase. 

When complex and extensive system changes are needed, the task 

of defining problems and identifying user requirements must be 

carefully considered early in the system development process. When 

user requirements are well established, the agency has a firm basis to 

consider alternate ways of developing a system to meet them. 

1. Determine user requirements as the first step in modifying or 

initiating a management information system. 

Agencies should encourage those functional groups primarily 

supported by the system to continuously recommend improvements. 

The user group is in the best position to recogn ize unsatisfied 

information requ irements. Agency management may also suggest 

improvements. 

For the agency to continuously update user needs, a central 

collection point should be established to record and categorize 

system problems and the requirements of users and management. 

This accumulated information should be continuously analyzed and 

evaluated to provide a basis for a prompt decision to change the 

system. Our study showed that systems have frequently been developed on a crash basis. 

2. Obtain user group agreement on all externally proposed 

changes. 

Recommended changes can originate from a variety of sources 

other than the user group, including agency management, the 

systems development group, internal and external auditors, and 

15 
unsolicited offerors. Because the user group will be the primary 

organization working with the new information system, it should 

review all externally proposed system capabilities and outputs. 

As documented requirements and analyses accumulate, positions 

and justifications for and against certain changes will begin to form 

within the agency. At this point, the agency should begin a formal, 

coordinated effort to study, val idate, and rank new information 

requirements. 

3. Allow adequate leadtime to study requirements and to relate 

them to agency objectives and long range plans. 

To insure effective involvement and coordination by all affected 

organizational elements, a task group should be formed to analyze 

information on problems and requirements in relation to agency 

objectives and long range plans. The task group leader should be the 

leading candidate for project coordinator. (See guidance item 11.) 

The task group members selected must have adequate technical 

ability and receive sufficient time to study, evaluate, and rank all 

user requirements. 

Since few agencies possess unlimited budgets for creating and 

improving systems, the study and analysis should eventually concentrate on ranking the information the user activity must have and can 

afford. The study's objective- and management's responsibility 

throughout the procurement-is to distinguish between needs and 

wants, mandatory and desired capabil ities, current and future 

requirements, and feasible versus unrealistic system performance. 

4. Make the study's final product a statement of requirements in 

their order of priority. 

Once the user group has reviewed and evaluated all proposed 

changes, the task group should develop a priority statement of user 

requirements which will give management a basis for carrying out the 

agency's program objectives and long range plans. This statement 

will give both the user group and management a basis for understanding the planned system performance. 

16 
5. Before considering design and implementation of a totally new 

system, examine feasible alternatives. 

A completely new system may be unnecessary to satisfy user 

requirements. Possible alternate solutions which should be con· 

sidered are to: 

Use in-house personnel to modify the existing system. 

Use the technical personnel of another agency or a contractor 

to redesign or ;ebuild the present system. 

Adopt all or part of another agency's system. 

Acquire and modify an existing, commercially available 

system. 

In screening alternatives, the study group should consider: 

The urgency of putting the new system into operation. 

Computer programming problems which may be encountered. 

The capabilities of existing equipment. 

The cost of modification. 

The types and availability of skills necessary to design, develop, and implement the alternatives. 

6. Convert each system requirement into design tasks and identify 

the skills needed to complete each task. 

Each requirement should be converted into a design or development task and the skill levels and number of people necessary to 

complete each task should be identified. Computer machine time 

may be necessary to complete some tasks. The types of personnel 

and staff time needed to complete all tasks will give the study group 

a basis for determining if the system can be developed internally. 

17 
7. Identify the people available to design, develop, and implement 

a new system. 

The resources inventory should identify all agency personnel 

who have the required design skills and can be made available for the 

system development project. By comparing available agency person· 

nel skills with those necessary to develop the system, the agency can 

bener evaluate alternate development solutions and, if it decides a 

contractor is necessary, can better describe what skills a potential 

contractor should provide. 

8. Do not consider contracting for the system design until upper 

management has approved the statement of requirements and 

design approach. 

Agency upper management must review and approve the requirements statement and the study group's proposed system 

development project. Upper management should require a presenta' 

tion by the study group on the statement and the proposed design 

approach along with justifications and rationale. I f upper manage· 

ment is dissatisfied with the study, it might obtain advice from 

executive agencies or organizations which have developed similar 

systems or those agencies which have been involved in systems 

development such as the Joint Financial Management Improvement 

Program, the Office of Management and Budget, the General Services 

Administration, and GAO. Management might also consider con· 

tracting with qualified individuals or a professional services firm to 

help review the requirement statement and the proposed system 

development approach. 

IN-HOUSE DEVelOPMENT VERSUS 

CONTRACTING 

Once the agency has defined and evaluated its user require· 

ments, it must decide whether to commit in·house personnel to 

develop and implement the new system or contract for development. 

9. If the decision is made to internally develop a new information 

system, management should make as few changes as possible in 

personnel committed to system development. 

18 
The GAO study showed that agency personnel assigned to 

system development projects are changed too frequently. To assure 

that effective and continuous effort is applied to attain a reliable 

system, agency management should minimize changes in the agency 

system development team. 

10. If the decision is made to contract, capable agency personnel 

should be assigned to assist the system development. 

Capable agency personnel should work full time to monitor and 

aid the contractor in designing and implementing the system. These 

personnel should be able to understand complex system needs, guide 

the contractor's performance, and report on problems and progress. 

PROJECT COORDINATOR SELECTION 

AND FUNCTIONS 

Whether in-house or contractor development is chosen, an 

agency project coordinator is needed to integrate and direct the 

development team and to monitor its performance. Our study 

showed that those development efforts which had no qualified 

project coordinator had more difficulty achieving successful systems. 

11. Select the project coordinator as the first step after deciding to 

hire a contractor. 

The project coordinator is an important catalytic agent in 

system development and implementation and should be selected at 

the earliest possible moment in the development process, especially 

when contractor assistance is anticipated. The project coordinator 

should participate in the user requirements study and the development of the requirements statement. (The coordinator may have 

been the task group leader for the requirement study.) 

In the early planning stages, the project coordinator or task 

group leader brings together the agency's personnel, including the 

user group, to concisely define user requ irements and plan for system 

design, development, and implementation. 

19 
12. Select a well-qualified agency official as the project coordinator 

and give him or her adequate resources. 

The project coordinator generally should be selected from the 

prime functional area that the proposed system will support. 

Important requisites for a project coordinator are the abilities to (1) 

plan, direct, and control resources, (2) communicate and cooperate 

with management and others involved in the development work, and 

(3) make necessary decisions to insure the success of the project. 

The project coordinator should be given (1) the authority to 

cross functional lines to communicate and coordinate the project and 

(2) direct access to key agency managers. The coordinator can get 

advice from a technical evaluation panel (to evaluate proposals) and a 

technical review panel (to review the contractor's design performance).fGuidance items 3, 38, and 55.) 

A project coordinator's job should be a full-time assignment 

until the system is operating satisfactorily; too often agencies assign a 

project coordinator part time. 

I n large system development projects, the agency may want to 

designate a deputy project coordinator to assure continuity of 

project management. 

13. Document the project coordinator's authorities and responsibilities. 

At an early stage, the project coordinator's authorities and 

responsibilities should be stated, including (1) the limits of his or her 

decisionmaking authority over system development and (2) who has 

the authority not delegated to him or her. A statement of the project 

coordinator's responsibilities will strengthen his relationship with 

both agency and contractor personnel and will help avoid indecisive 

contract management and delays in system development. 

20 
PLANNING THE SYSTEM'S SCOPE 

Most of our guidance items pertain to specific steps in the 

development process and are in rough chronological order. However, 

agency officials and contractors emphasized several points bearing on 

the whole planning process. These are presented as guidance items 14 

to 17. 

14. Plan early to provide maximum communication and coordination among key personnel and groups. 

In the planning process, the agency should organize its system 

objectives and long range goals and quantify risks, costs, and benefits 

for detailed evaluation and control of the system development. 

Planning should begin early for the specific steps required to 

accomplish the system's objectives, identify needed resources, and 

develop schedules for completion. Plans should provide for adequate 

management control over the development, including periodic 

reviews at key decision points and reviews of the adequacy of project 

staff. 

21 
15. Continuously involve all affected groups in the planning 

process. 

Developing effective user requirements for an information 

system requires the continuous involvement of individuals from 

many organizational elements. The involvement cannot be a one·time 

exposure; the process must permit and requ ire continuous exposure 

to updated plans and involvement in key decisions. Continuous 

involvement in planning insures that changes in system objectives and 

design will be communicated to all affected groups. 

16. Plan in detail to adequately identify and satisfy user require· 

ments. 

Aher the agency has coordinated user requ irements and 

constraints, the deta ils must be included in the planning documents, 

such as the statement of user requ i rements and the detai led work 

statement, and followed up throughout the acquisition. Careful 

attention to planning details can prevent spending unnecessary time 

and money in later system development stages. For example, needed 

information sources should be identified and arranged early in the 

system development to insure that needed information and data 

inputs will be supplied when the system becomes operational. If 

major changes are made in the system design without corresponding 

changes in information sources, system operation may be delayed. 

Training is another example of a planning detail which frequently has 

not been changed to match changes in the system design. 

17. Direct the planning toward an organized set of detailed system 

requirements. 

The work statement is the prime product of the early plann ing 

process and includes the detailed requ irement statement, the proc· 

essing requirements, cost· benefit expectations, all system constra ints 

and the tasks to be performed. Once the work statement is com· 

pleted, the user activity can obtain final management commitment 

and approval. The agency can then proceed to select the contractor. 

22 
CHAPTER 4 

PREPARING THE WORK STATEMENT 

AND SOLICITING CONTRACTORS 

Successful contractor solicitation to a large extent depends on 

the effectiveness of planning before solicitation. A detailed work 

statement, a key product of that planning, should be developed and 

approved before either internal or contracted systems development. 

During solicitation, the project coordinator should work closely 

with the agency's contracting officer to insure compliance with 

procurement regulations. The contracting officer is the agency's 

formal representative in procurement actions, such as the formal 

solicitation of proposals. The contracting officer should establish 

controls to assure that proper justifications and documentations are 

prepared and maintained, appropriate approvals are obtained, and 

funds are correctly obligated and disbursed. Before issuance, the 

contracting officer shou ld review the R FP to be sure it effectively 

communicates the system requirements and the related tasks to be 

performed by the contractor. 

COOPERATION AND COMMUNICATION 

la. Arrange a meeting of the project coordinator and contracting 

officer as soon as possible after deciding to contract. 

These officials should discuss the ground rules and approach to 

the procurement, the need for a preproposal conference, and all 

other matters requiring their mutual agreement and coordination. 

Early and continuous contact between the project coordinator and 

the contracting officer will avoid later problems. 

23 
19. Confirm the continued commitment of agency officials to the 

proposed system before preparing the detailed work statement. 

I nformal agreements and commitments may have been made 

during the user requirements study (Guidance item 3). Many changes 

may have subsequently been made in the system requirements which 

changed the impact on different groups with in the agency. 

The project coordinator, therefore, should obtain a full, formal 

commitment to the system development before preparing the work 

statement which will be included in the RFP. Obtaining recommit· 

ment by agency officials to the project at the beginning of the 

solicitation stage can help insure that (1) the project coordinator will 

receive all resources needed, (2) all agency organizations are fully 

informed of the system development plans, and (3) full support will 

be given to the project. 

DETAILED WORK STATEMENT 

The detailed work statement is the most important document in 

the system development project, especially one to be completed with 

contractor assistance as opposed to an internally developed system. 

The work statement serves as the agency's basic control over the 

contractor's performance since it specifies the tasks to be done. 

20. Set, as a primary work statement objective, understanding of 

the planned system, its processes, and products. 

The detailed work statement is included in the RFP provided 

the potential contractors. However, the work statement does more 

than just tell the contractor what ires· it I rves s 

a as is upon which to evaluate proposals. If it is specific, the work 

statement WIll narrow the cost ran e of offers an co tent of r · 

posa s. ne agency issued an R F P and work statement so vague t at 

proposed prices ranged from $200,000 to $2.8 million, a range of 

1,400 percent. 

24 
Agencies should prepare the work statement well enough to 

minimize subsequent changes. If the agency does not prepare a good 

work statement, subsequent changes may, in tum, require costly 

design and reprogramming changes and may cause delays in system 

implementation. If the detailed work statement is vague, mIsInterpretations and possible contract disputes on the grounds of 

changes in contract scope may arise. 

21. Identify end-product requirements clearly and completely. 

Although the agency should avoid specifying a detailed system 

design, it must very specifically define what the system must 

accomplish. 

-

Although the requirements will differ in each system, the work 

statement should include: 

Reports to be generated. 

Data base to be maintained. 

I nput and processing. 

Programs to be developed. 

Hardware available or to be acquired. 

Each requirement should be described as a specific task which the 

contractor will fulfill. 

22. Adequately identify and document all agency resouroe commitments and constraints. 

I mportant items frequently left out or vaguely stated in the 

work statements are agency resource commitments made to the 

contractor and constraints on s stem desi n. If any of this information IS omltte or subject to later interpretation, disputes may arise 

requiring senior agency and contractor management attention. 

25 
To assure satisfactory proposals and avoid disputes during 

contract performance, the R FP should spell out the services and 

resources which wil l be provided by the agency, such as: 

Personnel (number and qualifications). 

Work space. 

Computer time. 

New form preparation. 

Training. 

The R FP should also spell out the constraints which will be 

placed on the contractor and the system. such as Io mltab ons on 

changes In organizations affected by the system and time to complete Integral phases of the s opment. More responsive 

pro osa s an ewer a ministrative disputes will result if the work 

statement includes information of th is type. 

REQUEST FOR PROPOSALS 

Because the R FP is the contractors' basis for preparing proposals, it must include all pertinent data. I n addition to the work 

statement, the R FP must include: 

Type of contract (fixed price or cost reimbursement). 

Contractor selection criteria. 

Contractor administrative requirements. 

Acceptance test plan. 

Implementation plan. 

Glossary of terms. 

System documentation requ irements and standards. 

26 
23. Choose the procurement approach and type of contract which 

will help achieve the final objective-an efficient, effective 

system. 

A poor choice of procurement approach can hinder system 

development because of disputes and misunderstandings. In 

the long run, problems arising from contract type can discourage 

good firms from submitting proposals. The contract type should facilitate the acquisition of the system, not hinder it. 

Competitive negotiated procurement is, in many cases, the more 

advantageous approach in obtaining what is needed at the lowest 

cost. Competitive negotiation is initiated by an R FP, which contains 

both the agenc 's re uirements and the criteria for evaluating offers. 

An contemplates the submission 0 time y responsive proposals 

by a maximum humber of responsible offerors, and is usually fol· 

lowed by discussion with those offerors who are in the "competitive 

range." The contract is awarded to the one whose offer is the most 

advantageous to the Government, price and other factors considered. 

Competitive negotiation is similar to formal advertising in that 

(1) the goal is to achieve as much competition as possible and (2) the 

specifications must be as precise and meaningful as possible. It is 

different from formal advertising in that it permits evaluation on the 

basis of a combination of factors including price and allows dis· 

cussion and clarification of ambiguities, misunderstandings, and 

mistakes during the selection process. Further, in competitive 

negotiations agencies may use any allowable contract type, and 

the system design specifications need not be as precise at the time 

of solicitation as are those required in formal advertising. 

24. Use sole·source procurement as a last resort. 

An agency should only use sole source procurement when it has 

been unable to find more than one qualified contractor who is 

willing to undertake the job. I n the field of financial and other 

management systems design, such a circumstance should be ex· 

tremely rare. In this field, there are many firms that have the 

necessary expertise. Some firms may be able to supply more efficient 

methods of achieving the end product, but only competition can 

expose the differences in approach and cost. 

27 
25. Carefully consider the type of contract. 

The type of contract depends upon the circumstances of each 

individual procurement. The agency must consider the 

innovativeness of system design, 

complexity of the procurement, 

degree of R FP specificity, 

allocation of risk between the Government and the contractor, 

degree of Government control, and 

delivery schedule. 

There is no general rule to use when deciding on the type of con· 

tract. 

The cost·reimbursement and fi xed-price contracts have advantages and disadvantages in individual cases. The fixed-price 

contract places maximum risk on the contractor. The contractor has 

a maximum profit incentive to control costs and perform the contract effectively. The fixed-price contract is suitable for procurements when reasonably specific design specifications are available 

and whenever fair and reasonable prices can be established before 

procurement. The fixed-price contract is particularly su itable in 

purchasing standard or modified systems for which sound cost 

esti mates can be developed. 

The cost-reimbursement type of contract provides for allowable 

costs incurred in the performance of the contract to be paid to the 

contractor. This type of contract establishes an estimate of total cost 

for the purpose of obligating funds and establishing a ceiling which 

the contractor may not exceed without prior approval of the contracting officer. Cost·reimbursement contracts place less financial 

risk on the contractor and more risk on the Government than do 

fixed-price contracts. The cost-reimbursement type contract is 

su itable when the cost of erformance cannot be reasonabl estima e . 

28 
I f performance specifications cannot be detailed or if the 

agency wishes to participate with the contractor in evaluating trade· 

offs among alternative performance specifications which will have 

varying impacts on project costs and scheduled completion, a cost· 

reimbursement contract may be the best vehicle. 

26. Develop contractor selection criteria when selecting the con· 

tracting method and include the criteria in the R FP. 

Three areas in which basic criteria must be established in 

procuring systems are price, technical desig,n, and technical qualifica· 

tions. Competition in terms of technical qualifications is more 

appropriate for complicated systems because success depends on 

technical ability of contractor staff. But high quality may mean high 

price. Therefore, ossible com remise is to evaluate proposals on 

the basis of a combinati . 

tion criteria . . 

The R FP must state the selection criteria the project coor· 

dinator will be using. Since competitive negotiation calls for as much 

competition as possible, the R FP must clearly specify what the 

evaluation factors are and identif their relative im ortance. In thiS 

t e omptroller General has stated: 

"Intelligent competition requires, as a matter of sound 

procurement policy, that offerers be advised of the evalua· 

tion factors to be used and the relative importance of those 

factors. Each offeror has a right to know whether the pro· 

curement is intended to achieve a minimum standard at the 

lowest cost or whether cost is secondary to quality, Com· 

petition is hardly served if offerors are not given any idea of 

the relative values of technical excellence and price," 

27. Include in the R FP all contractor administrative reporting 

requirements. 

The R FP should include all reporting requirements so con· 

tractors can understand the controls to be exercised by the agency, 

and be more responsive to the RFP, 

29 
Adequate performance reporting is required to insure good 

communication and control. The reporting requirements should be 

supplemented by informal communication to resolve any problems 

which may arise between reporting intervals. The lack of reporting 

requirements may result in inadequate project control. 

The following information is the minimum necessary to manage 

the contract. 

Actual versus scheduled progress. 

I ncurred versus estimated cost. 

Problem areas to be resolved by the contractor. 

Problems needing resolution by project coordinator and their 

effect on the performance schedule. 

Problems identified in prior reports which have not yet been 

solved. 

Work to be done during the next reporting period. 

The contractor must supply the information, and the project coor· 

dinator must critically evaluate and follow up on matters requiring 

his attention. Through the reports the project coordinator can 

analyze the contractor's progress, cost, and problem areas. Progress 

reporting should be as frequent as necessary and be complemented 

by continuous communication between the coordinator and contractor. 

28. Include the acceptance test and implementation schedules in 

the RFP. 

The project coordinator should re are 

implE1menting t e system fore issuing the R FP. ot testing an 

implementation will require contractor resources; therefore, the 

plans must be included in the RFP to assure that contractors consider these requirements in preparing their proposals. 

30 
29. Include a glossary in the RFP to insure that system and per· 

formance requ irements are understood. 

A glossary gives the contractor and the project coordinator a 

common basis for communication and coordination. It should help 

achieve responsive proposals and effective progress reports after the 

contract has been awarded. 

30. Review the R FP for clarity, comprehensiveness, and legality 

before distributing it. 

Before distributing the RFP, the contracting officer should 

request a review by the (1) project coordinator and user group for 

technical accuracy, (2) general counsel for legality, (3) financial 

officer for availability of funds, and (4) upper management for final 

approval. 

RFP DISTRIBUTION 

31. Give the R FP maximum publicity and distribution. 

To assure a good response to the R FP, arrangements need to be 

made to publicize the system procurement and provide effective 

distribution of the RFP. This is usually done by the contracting 

officer. 

To give contractors not on a contractor I ist a chance to com· 

pete, the agency should place a notice of solicitation in the Com· 

merce Business Daily. The notice should be placed in time to give 

prospective contractors adequate opportunity to request and receive 

a copy of the R FP, to attend the preproposal conference, and to 

submit a proposal by the due date. 

Recent studies have shown that most R FPs for system design 

and development are issued in the last 2 months of the fiscal year. 

Contractors see this last·minute distribution as one of the biggest 

problems in agencies' procurement of systems and one which 

adversely affects contractors as well as the agencies. If agencies want 

maximum competition and responsiveness from highly qualified 

firms, the solicitation, evaluation, negotiation, and contractor 

selection process should begin early in the fiscal year to avoid an 

end·of·the·year rush. 

31 
PHASED COMPETITIVE 

NEGOTIATION 

Negotiated contracts are usually preceded by a single R FP 

which describes the agency's requirement for a system and requests a 

formal proposal. The agency then evaluates the proposals, establishes 

a competitive range, and negotiates with the firms having the best 

proposals. 

An alternative is the phased solicitation process. The agency 

first sends out and publishes in the Commerce Business Daily a 

notice of intent to procure a system. This notice gives an overview of 

the agency's system needs and requests a statement of the firm's 

capability to meet its needs, including a list of personnel to be 

assigned to the project, past experience related to the proposed 

project, and a general approach to the development of the system 

required. 

The selection criteria discussed in Guidance Item 26 are crucial 

in using the phased process. The agency must present clear, detailed 

selection factors as an aid to contractors deciding whether or not to 

express an interest. Even though the notice of intent to procure is a 

synopsis of the R FP and is intended to generate as much competition 

as possible, neither the agency nor obviously non-competitive firms 

are well served if the selection criteria are so vague that a large 

number of those firms decide to respond anyway. 

The agency reviews the material submitted and sends the R FP 

with the detailed work statement to firms selected on the basis of 

evaluation criteria included in the notice. 

The agency should consider using the phased negotiated process 

if the system is highly complex. The phased process permits the 

agency to consider proposals from the more competitive firms and 

gives those firms a better opportunity to fully study the problem. 

This lowers the costs of evaluation and may result in (1) better 

communication between the agency and firms and (2) proposals that 

are more responsive to agency requirements. 

32 
The phased process lowers the firms' risk of preparing costly, 

noncompetitive proposals and may encourage better competition. 

The process is appropriate when proven design capabi I ity of the 

firm's staffs and an extended opportunity for the final offerors to 

interact with operating personnel are considered necessary to develop 

the best system design. Firms which decide not to compete benefit 

from the phased process because less time and money are spent on 

detailed proposals. 

P H ASE ON E 

In reviewing a contractor's competitiveness, the agency should 

consider three primary areas; capability, experience, and general 

approach to systems development. The contractor's capabi lity shou Id 

be reviewed in light of the backgrounds of the management staff to 

be assigned; ability to finance the contract; organization; methods of 

operation; special expertise, including automatic data processing 

capability; and past experience with projects of similar magnitude or 

complexity. 

The contractor's experience shou Id be reviewed in light of the 

number and types of clients presently serviced; types of services 

rendered to clients; and past experience with Federal, State, and 

local governments. A special effort shou ld be made to determine 

the effectiveness of the contractor's staff performance on similar 

system development projects. 

The contractor's general approach to systems development 

should be evaluated in terms of (1) the quality and quantity of 

personnel to be assigned to each task, (2) staff organization, (3) plans 

for working with Government personnel, (4) planned controls over 

time and expenditures, (5) planned techniques to perform the work, 

and (6) evidence of ability to document systems procedures and 

develop useful reports. 

The award criteria should be detailed in the notice of intent to 

procure, and the value assigned to each item should be listed. The 

eva luation team should use the established criteria to select firms 

qualified to participate in the second phase. 

33 
PHASE TWO 

The project coordinator and the manager of the user organization, using the established criteria, should select offerors and 

encourage them to submit fully developed technical proposals with 

required cost information. This selection process should not be used 

for administrative expedience but to encourage firms with the 

requisite capability to submit proposals. 

The second phase is like the normal competitive negotiation 

except that the R FP is sent selectively to those firms judged most 

competitive. Firms not requested to participate in phase two which 

desire to compete may still request and receive a detailed R FP and 

submit a proposal. 

34 
CHAPTER 5 

PROPOSAL, EVALUATION, 

NEGOTIA TION, AND A WARD 

PREPROPOSAL CO NFERE NCE 

32. Conduct a preproposal conference shortly after issuing RFP. 

The contracting officer and the project coordinator are often 

called upon to answer oHerors' individual inquiries or to provide 

information. Information of a substantial nature given to one firm 

must be given to all firms. The contracting officer must determine 

whether the information requested will give one firm an advantage 

over others and subject the agency to accusations of favoritism. 

Unsuccessfu I contractors have based bid protests on such accusations. 

The preproposal conference is a meeting to which all potential 

offerors should be invited (date, time, and place should be indicated 

in the R FP) to provide general reactions to the R FP and to promote 

uniform interpretation of all statements and specific requirements 

contained in it. The conference gives all potential oHerors an 

opportun ity to clear up questions about the proposed project. Ques· 

tions may be written to the project coordinator before the con· 

ference or asked orally or in writing during the conference. 

33. Have key agency officials involved in the system project attend 

the conference and answer questions. 

The contracting officer shou Id conduct the conference and the 

project coordinator should answer technical questions on the desired 

system. In preparing for the' conference, the contracting officer and 

project coordinator must coordinate with all other agency officialsdesigners, users, contract administrators- involved in the system 

project. Written questions involving technical matters submitted to 

the project coordinator or the contracting officer should be research· 

ed by technical personnel to insure that appropriate answers are 

supplied. 

34. Include major concerns or issues in the conference agenda and 

allow adequate discussion time. 

To insure an orderly and productive conference, an agenda 

should be developed and given to all attending: 

During the conference, the agency should: 

35 
Explain the background and need for the system, including 

performance target dates. 

Describe similar current and past efforts. 

Clarify all R FP statements and specifications. 

Resolve all contractor questions, even if agency research is 

required. 

Clearly explain the criteria methods for selecting the con· 

tractor. 

Outline the award schedule. 

Stress agency commitment and full support. 

35. Document the conference proceedings and distribute the record 

to all potential offerors. 

Have a stenographer present to record essential questions, 

answers, and issues discussed at the meeting, which should then be 

summarized and provided to all potential offerors. 

EVALUATION 

The objective of proposal evaluation is to select all competitive 

proposals. A proposal is competitive unless it is so technically inferior or costly that meaningful negotiations are precluded. The contracting officer is responsible for deciding whether a proposal is in 

the competitive range. 

36. Arrange for several knowledgeable members of the user organization to be available to answer the final offerors' questions. 

The project coordinator must insure that all questions arising 

from the detailed RFP are answered fully and accurately while the 

contractors are preparing their proposals. I f necessary, knowledgeable members of the user organization shou Id respond to offerors' 

questions. 

36 
37. Visit the office of each offeror. 

The project coordinator and the contracting officer should visit 

each offeror's office to determine the adequacy of facilities, 

personnel, and automatic data processing support. In relatively small 

efforts, this step may be too costly and unnecessary. In other more 

complex efforts, where personnel, facilities, and automatic data 

processing support are key elements, site visits may be of great 

importance. 

38. Carefully select an evaluation team. 

The evaluation team should include the contracting officer, the 

project coordinator, the user organization manager, a panel of tech· 

nical experts, and a representative from the legal staff. The overriding 

concern should be the objectivity of team members. If the procure· 

ment is especially important, upper management may wish to partici· 

pate with the project coordinator and the contracting officer in 

selecting the evaluation team. 

39. Review each proposal as soon as it comes in. 

The contracting officer and the project coordinator should 

determine that each offer is complete and satisfies the R FP require· 

ments and any subsequent amendments. If any incomplete offers are 

submitted before the final proposal cutoff date, the project coordi· 

nator should advise the offerors of the missing elements. After the 

evaluation team has been selected and the project coordinator and 

contracting officer have briefly reviewed each offer, the team should 

begin detailed analysis. 

40. Give the panel of agency technical experts responsibility for 

evaluating each proposal on its technical merits. 

The first step in the detailed evaluation process is an analysis of 

each proposal's technical merit by the panel of technical experts. 

Once the panel determines that the proposals are technically accept· 

able, the entire evaluation team should review each proposal in its 

entirety. 

37 
41. Make the evaluation team responsible for in·depth review of 

each offeror's qualifications, price quotations, overall design 

concept, delivery schedule, and administrative procedures. 

As the evaluation process continues, the team should rank each 

major element of the proposals according to preestablished criteria 

and determine composite rankings. The ranking will form the basis 

for establishing the competitive range and initiating negotiations with 

those firms in the competitive range. 

42. Conduct a cost analysis of each proposal. 

The RFP should require the offerors to break down and 

justify their estimated costs and identify the cost. The cost 

analysis, i.e., verifying the cost and the estimates' reasonableness 

should be performed by agency audit and technical personnel at the 

offerors' pi ants or offices. 

43. To avoid future litigation, adhere to the selection criteria in the 

R FP, document the evaluation process, and prepare to brief all 

unsuccessfu I offerors who so request. 

Deviating from the stated selection criteria, failing to document 

the entire evaluation process, and failing to explain to unsuccessful 

firms the reasons for their nonselection may result in bid protests. 

NEGOTIATION 

Once all proposals have been evaluated, the evaluation team 

should have identified offerors whose proposals merit negotiations. 

Before beginning formal negotiations, the following two preliminary 

steps should be taken. 

44. Organize a negotiation team headed by the contracting officer. 

The contracting officer should be given the primary responsi· 

bility for selecting the negotiation team members. The team should 

include the project coordinator, one or more representatives from 

38 
the user organization, technical advisors, and financial analysts. The 

contracting officer is responsible for establishing procedures, an 

agenda, and a timetable f?n the negotiations. The contracting officer 

should also establish the egotiation team's positions on selection 

considerations, priorities, nd goals. 

If not technically qualified, the contracting officer should delegate technical negotiation to a team member. 

45. Obtain basic negotiating data from each final offeror. 

Examples of data which should be obtained are: 

Department(s) within each firm where the contract work will 

be done. 

Names and functions of personnel who will be working on 

the project. 

The purpose of these preliminary steps (sometimes called the 

prenegotiation process) is to confirm that the key personnel included in the proposals negotiate and if selected, perform the work. 

46. Conduct individual negotiating sessions with each offeror. 

During each individual session, the contracting officer should 

attempt to resolve all major issues to the satisfaction of the agency as 

well as the contractor. When negotiations have been completed with 

each offeror in the competitive range and at least one of the sessions 

has resulted in an acceptable agreement, the contracting officer 

should announce an official cutoff date for a "best and final" offer 

from each offeror. The offers should include any combined adjust· 

ments of price, personnel, and technical aspects to make proposals 

more competitive. 

If a cost type contract is to be awarded, care should be exer· 

cised that the offerors' final offers are realistic cost estimates and do 

not represent an attempt to "buy·in" without risk, since payment 

will be based on actual cost. 

39 
47. Insure that all information obtained during negotiations is kept 

confidential. 

The offerors have proprietary interest in their proposals, and 

therefore the evaluation team must safeguard the confidentiality of 

the proposal information at all times. 

AWARD 

The final award is the contracting officer's responsibility. He 

should carefully consider the negotiating team's findings and recom· 

mendations, the agreements reached during negotiations and all proposal changes. In major management information systems acquisitions there may be a source selection board that reviews the results 

of evaluation and negotiation. There may also be a source selection 

authority, sometimes the head of the agency, who reviews the recommendations of the source selection board and makes the final award 

decision. 

48. Before the contract is awarded, submit it to the project coordinator, user group manager, and legal staff for review. 

As a last step before contract award, the contract must be reviewed to insure that the agreements reached are acceptable to the 

project coordinator and will not present legal difficulties later. 

49. Promptly brief unsuccessful finalists as requested. 

To lower the probability of bid protests, agencies, when requested by unsuccessful offerors, should brief them on the reasons 

for nonselection. An important second objective of these briefings is 

to promote more responsive proposals on subsequent development 

efforts. 

40 
CHAPTER 6 

SYSTEM CONTRACT PERFORMANCE 

This chapter covers the agency's working relationship with the 

selected contractor during contract performance. 

I t is important that an agency-contractor team establish rapport 

early so that both parties are working to achieve the contract's objec· 

tives. 

The guidance included in this chapter applies to systems developed internally or with contractor assistance. If the system is internally developed, "Agency systems group"can be substituted for"contractor" in the guidance items. 

QUICKLY RESOLVE QUESTIONS 

AND ISSUES 

After the contract has been awarded and the contractor begins 

to work with the agency, questions and issues will begin to arise over 

definitions, interpretations, and responsibilities. Questions may originate about the user requirements, responsibility for carrying out certain tasks, and proper working relationships. 

50. Resolve any re'llaining problems which arose during previous 

stages. 

Before the contractor begins working, the coordinator should 

resolve any outstanding problems. Effective coordination and communication are especially important during contractor performance. 

Communication among the coordinator, the contracting officer, the 

contractor, the user activity, and agency functional areas should be 

open and frequent. 

51. Conduct an agency postaward conference. 

The project coordinator and contracting officer should conduct 

the conference to discuss matters requiring clarification or resolution, contractual requirements, and areas of responsibility and 

authority, to insure that they are understood by all participants. The 

conference provides an opportunity for each participant to ask any 

questions regarding his/her role. The contracting officer, the project 

coordinator, system users, and automatic data processing or other 

special functional personnel should participate in the conference. 

41 
52. Conduct a postaward conference with the contractor. 

The purpose of this conference is to develop an effective work· 

ing relationship among the contractor, project coordinator, contract· 

ing officer, and user personnel. The conference will insure mutual 

understanding of the contract requirements and identify the authori· 

ties and responsibilities of the individuals representing the Govern· 

ment. The agency shou Id also answer contractor questions and 

clarify administrative matters. 

SYSTEM DESIGN 

The approach to the system design can affect how well the 

system meets the user's requirements and the cost or time required 

to test and implement the system. 

53. During the contracting process, determine the system design 

approach. 

The design approach provides the framework for controlling the 

contractor's design, and therefore, shou Id be part of the contract. 

The agency should have obtained the best design approach by using 

the evaluation criteria and contractor negotiations. When a design 

approach provides adequate review and control by the agency, the 

chances of receiving a system which satisfies the user's requirements 

are improved. 

The two most common system design approaches are phased 

design and total design. The latter method gives the least control over 

the contractor's design, because the agency does not review the 

design until the contractor develops the complete system and sub· 

mits it for approval. Since this approach does not permit periodic 

agency review during the design and development processes, the con· 

tractor may develop a system which does not satisfy the agency's 

requirements. 

The phased design approach, on the other hand, gives the proj· 

ect coordinator an opportunity to effectively monitor the con· 

tractor's design development. This approach normally consists of 

42 
three major phases-the conceputal design, the general design, and 

the detailed design. At a minimum, the contractor's design should be 

submitted for agency review and approval at the end of each phase. 

If the system development effort is large and complex, the contract 

could require agency review during and after major tasks within each 

phase. 

The contract should specify what the contractor is to present 

for what type of review. At each review point the project coordinator and the technical review team can evaluate the contractor's 

design, including the input and output, the availability of source 

data, and adherence to system constraints specified in the contract. 

54. Use the phased design approach whenever possible. 

By contracting for the phased design approach, the agency gets 

the opportunity to either (1) negotiate with the contractor on subsequent phases or (2) solicit competitive offers and negotiate each 

successive phase of the system development. 

The agency may have chosen a cost reimbursement arrangement 

for the conceptual and general design phases, in order to permit 

participation with the contractor in deciding which system design 

alternative will best satisfy the agency's need. After the general 

design has been developed and the fi nancial ri sk and contracti ng 

uncertainties are reduced, a fixed-price contract for detailed design 

and implementation may be negotiated. Using this approach in the 

later phases, the agency wi II have greater control over costs and 

implementation schedules, and the contractor will have more specific 

system specifications upon which to base staff, time, and cost estimates. I f the phased design approach is used, the project coordinator 

must not only allocate enough time to insure a thorough review of 

the contractor's design but also establish review completion dates to 

prevent system delays. 

55. Require other functional and user group personnel to review the 

design for acceptability. 

The project coordinator, the contracting officer, key representatives of the user group, and each affected functional area should be 

43 
given the opportunity to review and evaluate all design segments, to 

insure that the design is practicable from all standpoints. 

56. Require the contractor to state new input requirements when 

identified. 

If new data input requirements are not identified, documented, 

and evaluated early, development may be delayed substantially by 

the need to revise the system to provide for accumulating the new 

data. New data inputs usually require that new forms be designed, 

approved, and printed, ultimately necessitating additional staff. 

57. During the general design phase, have the agency's internal audit 

group evaluate the contractor's design for adequate audit trails 

and internal controls. 

Providing appropriate audit trails and internal controls in the 

initial design of the system will minimize changes required after the 

general design has been completed. Late design changes inevitably 

delay system development and increase cost. 

58. Once each phase has been approved, insure that the contractor 

and agency employ strict management to control change. 

Changes to the system's design can delay implementation, dis· 

rupt user activity, and raise costs. Proposed changes should be evalu· 

ated and if they are not critical to system operation, the project 

coordinator should consider deferring them until the system is opera· 

tional. I n any case the changes should be fully documented. 

59. Plan and coordinate required changes as early as possible. 

Duri ng the general and detailed design phases, the agency and 

the contractor will usually identify additional changes needed in (1) 

organization of the user group or other agency component, (2) staff 

levels, and (3) computer and other resources. The project coordi· 

nator should effect these changes quickly to insure expedient imple· 

mentation and operation of the system. 

44 
IMPLEMENTATION 

The project coordinator should develop ways to promptly 

identify and solve problems to prevent delays in system implementation and operation . 

As discussed earlier, the phased design approach provides maximum control and feedback to the contractor. Just as the design 

approach is important in controlling the contractor's design development and meeting the user's requirements, an effective implementation approach can minimize system problems. 

60. Consider implementing the system in modules. 

The two primary approaches to implementing the system are 

the complete system approach and the modular system approach. In 

the complete approach, the entire system is implemented and tested 

at the same time. If problems arise in any segment of the system, 

other segments generally will be affected. Problems are thus magnified. With this approach, segments of the system cannot become 

operational before the entire system is implemented. 

Under modu:ar implementation, the system is designed and 

developed in small segments, called modules, wh ich will operate 

independently. As each module is developed, it can be tested separately and then in combination with others. After the module has 

been satisfactorily tested and approved, it can be implemented. 

The modular approach helps to identify problem areas in the 

various subsystems and eases correction. In cases where specific system segments are urgently needed, modular implementation is 

especially effective. When using the modular approach, the agency 

must have a master plan for developing and integrating system 

modules. 

45 
61. Get contractors responsible for developing the design involved 

in implementing the system. 

Agencies should not contract for systems development and then 

rely solely on internal expertise for implementation. The contractor 

designing the system has developed a knowledge base which is indis· 

pensable during implementation. The contractor can provide invaluable help in identifying problems and in "debugging" the system 

during implementation. 

62. Test the system modules and/or the total system completely 

before implementation. 

The objectives and scope of testing may vary considerably, 

depending on the type of software, the operational environment, 

needed reliabil ity, and other factors. The agency should insure that 

the contractor intends to thoroughly test the software, even if the 

agency is planning to accept the system solely on the results of 

acceptance testing. I nsisting on successful completion of early, less 

formal testing increases the chances that the software will perform 

satisfactori Iy. 

The types of testing can generally be defined as: 

1. Debugging, when the programmer is attempting to create 

an error·free program. 

2. Development testing, when the programmer is testing 

small segments of the software (e.g., at the routine level). 

46 
3. I ntegration testing, when segments are joined to form 

modules. 

4. Acceptance testing for each subsystem and for the total 

system. 

5. Operational demonstrations. 

Testing is generally a building·block affair. Each successive type 

of testing examines larger segments of integrated software, accom· 

plishes different objectives, and is conducted with increasing formal· 

ity. 

Test objectives can be tailored to fit the modular development 

approach and can even be modified at almost any level of testing to 

address the system specifications of performance, accuracy, user flex· 

ibility, and other characteristics. If an error is found and corrected, 

retesting is necessary to demonstrate that previously accepted seg. 

ments will not be affected by the change. 

63. Test the new system concurrently with the old one (if any) 

until it satisfies the design requirements. 

The implementation and acceptance test plan should provide 

for parallel system operation (old and new system functioning simul· 

taneously). This is especially important if the user activity must have 

reliable information for operations and reporting. The system should 

not be converted until all tests have been completed and data outputs analyzed to insure proper system operation. If concurrent or 

parallel operation is not practical, the system should be debugged 

and fully tested before system conversion. 

OPERATION 

System conversion or implementation is a key step in acquiring 

a new system. If the system does not operate satisfactorily, the 

causes can usually be traced to deficiencies in the previous phases of 

system acquisition-planning, contracting, designing, developing, and 

implementing. 

47 
64. Finish arrangements for operating the new system, acquire the 

necessary resources, and train personnel before the system 

begins operating. 

Any organizational changes required by the new system should 

have been developed, coordinated, and initiated before system implementation. If any of the arrangements have not been completed, the 

agency may have serious problems operating the system. 

65. If required personnel are untrained, arrange for the contractor 

to operate the system and train the personnel. 

Our study showed that inadequate training of agency system 

operators and users frequently caused system operational problems 

and delays even when a system was successfully designed and developed. Detailed training plans were often made early but were not 

modified to meet system design or implementation schedule changes. 

66. Provide for contractor assistance after the system has been 

accepted. 

System operation problems which require contractor help may 

arise after system acceptance and implementation. Therefore, the 

agency should consider providing in the contract for continuing 

assistance to be billed on an hourly basis. Such a provision would 

reduce the time required to make necessary corrections or modifications. 

AFTER OPERATION AND CLOSEOUT 

After the system becomes operational and the contract has been 

closed out, the agency should evaluate the development process and 

the system's performance so it can identify its mistakes and successes 

for use in future system acquisitions. 

48 
07. Determine how efficiently and effectively the system is meeting 

the requirements identified in the planning stage. 

Management should have the agency's internal audit staff review 

the system after it becomes fully operational, and periodically there· 

after. The reviews should use computer·assisted audit techniques to 

inform management about the system's operating efficiency and 

effectiveness. 

Our study showed that reviews have helped management to 

insure that systems are producing accurate and useful information. 

68. Document and accumulate the lessons learned. 

Valuable knowledge and experience is gained from each system 

development. To prevent loss of knowledge and recurrence of past 

problems, lessons learned should be systematically accumulated for 

future use. 

Agency management should arrange for key parties in the sys· 

tem development to prepare written comments on: 

How effectively user requirements were identified. 

How effectively the contractor performed. 

The techniques used in system development and acquisition. 

What procedures or techniques should be used in future sys· 

tem development projects and why. 

At a minimum, the project coordinator, contractor representatives, 

the contracting officer, and key user personnel should be asked to 

comment. 

49 
This is what happens with good systems development management 

50 
Appendix I Appendix I 

GUIDANCE ITEM LISTING 

DEFINING THE PROBLEM AND 

PLANNING FOR SYSTEM CHANGE 

(CH. 3) 

Determining user requirements 

1. Determine user requirements as the first step in modify· 

ing or initiating an information system. 

2. Obtain user group agreement on all externally proposed 

changes. 

3. Allow adequate leadtime to study requirements and to 

relate them to agency objectives and long·range plans. 

4. Make the study's final product a statement of requirements in their order of priority. 

5. Before considering design and implementation of a 

totally new system, examine feasible alternatives. 

6. Convert each system requirement into design tasks and 

identify the skills needed to complete each task. 

7. Identify the people available to design, develop, and 

implement a new system. 

8. Do not consider contracting for the system design until 

upper management "'as approved the statement of requirements and design approach. 

In-house development versus contracting 

9. If the decision is made to internally develop a new infor· 

mation system, management should make as few changes 

as possible in personnel committed to system develop· 

ment. 

51 
Appendix I Appendix I 

10. I f the decision is made to contract, capable agency personnel should be assigned to assist the system development. 

• Project coordinator selection and functions 

11. Select the project coordinator as the first step after deciding to hire a contractor. 

12. Select a well-qualified agency official as the project 

coordinator and give him or her adequate resources. 

13. Document the project coordinator's authorities and 

responsibi I ities. 

Planning the system's scope 

14. Plan early to provide maximum communication and 

coordination among key personnel and groups. 

15. Continuously involve all affected groups in the planning 

process. 

16. Plan in detail to adequately identify and satisfy user requ i rements. 

17. Direct the planning toward an organized set of detailed 

system requ i rements. 

PREPARING 

THE WORK STATEMENT/AND 

SOLICITING CONTRACTORS (CH. 4) 

Cooperation and communication 

18. Arrange a meeting of the project coordinator and contracting officer as soon as possible after deciding to contract. 

52 
Appendix I Appendix I 

19. Confirm the continued commitment of agency officials 

to the proposed system before preparing the detailed 

work statement. 

Detailed work statement 

20. Set, as a primary work statement objective, under· 

standing of the planned system, its processes, and prod· 

ucts. 

21. Identify end·product requirements clearly and com· 

pletely. 

22. Adequately identify and document all agency resource 

commitments and constraints. 

Request for proposals 

23. Choose the procurement approach and type of 

contract which will help achieve the final objectivean efficient, effective system. 

24. Use sole-source procurement as a last resort. 

25. Carefully consider type of contract. 

26. Develop contractor selection criteria when selecting the 

contracting method and include the criteria in the RFP. 

27. Include in the RFP all contractor administrative reporting 

requirements. 

28. Include the acceptance test and implementation sched· 

ules in the RFP. 

29. I nclude a glossary in the R FP to insure that system and 

performance requirements are understood. 

30. Review the R FP for clarity, comprehensiveness, and legal· 

ity before distributing it. 

31. Give the RFP maximum publicity and distribution. 

53 
Appendix I Appendix I 

PROPOSAL, EVALUATION, 

NEGOTIATION, AND AWARD (CH. 5) 

Preproposal conference 

32. Conduct a preproposal conference shortly after issuing 

the RFP. 

33. Have key agency officials involved in the system project 

attend the conference and answer questions. 

34. I nelude major concerns or issues in the conference agenda 

and allow adequate discussion time. 

35. Document the conference proceedings and distribute the 

record to all potential offerors. 

Evaluation 

36. Arrange for several knowledgeable members of the user 

organization to be available to answer the final offerors' 

questions. 

37. Visit the office of each offeror. 

38. Carefully select an evaluation team. 

39. Review each proposal as soon as it comes in. 

40. Give the panel of agency technical experts responsibility 

for evaluating each proposal on its technical merits. 

41. Make the evaluation team responsible for indepth review 

of each offeror's qualifications, price quotations, overall 

design concept, delivery SChedule, and administrative procedures. 

42. Conduct a cost analysis of each proposal. 

43. To avoid future litigation, adhere to the selection criteria 

in the RFP, document the evaluation process, and prepare to brief all unsuccessful offerors who so request. 

54 
Appendix I Appendix I 

Negotiation 

Award 

44. Organize a negotiation team headed by the contracting 

officer. 

45. Obtain basic negotiating data from each final offeror. 

46. Conduct individual negotiating sessions with each offeror. 

47. Insure that all information obtained during negotiations 

is kept confidential. 

48. Before the contract is awarded, submit it to the project 

coordinator, user group manager, and legal staff for review. 

49. Promptly brief unsuccessful finalists as requested. 

SYSTEM CONTRACT PERFORMANCE 

eCHo 6) 

Quickly resolve questions and issues 

50. Resolve an', remaining problems which arose during previous stages. 

51. Conduct an agency postaward conference. 

52. Conduct a posta ward conference with the contractor. 

System design 

53. During the contracting process, determine the system 

design approach. 

54. Use the phased design approach whenever possible. 

55 
Appendix I Appendix I 

55. Require other functional and user group personnel to review the design for acceptability. 

56. Require the contractor to state new input requirements 

when identified. 

57. During the general design phase, have the agency's internal audit group evaluate the contractor's design for adequate audit trails and internal controls. 

58. Once each phase has been approved, insure that the contractor and agency employ strict management to control 

change. 

59. Plan and coordinate required changes as early as possible. 

Implementation 

60. Consider implementing the system in modules. 

61 . Get contractors responsible for developing the design 

involved in implementing the system. 

62. Test the system modules and/or the total system completely before implementation. 

63. Test the new system concurrently with the old one (if 

any) until it satisfies the design requirements. 

Operation 

64. Finish arrangements for operating the new system, 

acquire necessary resources, and train personnel before 

the system begins operating. 

65. If required personnel are untrained, arrange for the contractor to operate the system and train the personnel. 

66. Provide for contractor assistance after the system has 

been accepted. 

56 
Appendix I Appendix I 

67. Determine how efficiently and effectively the system is 

meeting the requ irements identified in the planning stage. 

68. Document and accumulate the lessons learned. 

57 
Responsibility keys 

P = primary 

J = joint 

I = involvement 

Project coordinator 

Contracting officer 

(J'l User activity (tech. experts) co 

Contractor personnel 

Agency functional areas 

Agency computer group 

Internal auditors 

Upper management 

General Counsel 

RESPONSIBILITY MATR

GUIDANCE ITEMS 

1 2 3 4 5 6 7 . 8 . 9 10 . 1 1 . 12 . 13 . 14 15 1

P P P P P P I I I P P P 

P P I I I I I I I I I I I 

I I I I I I I I I I I 

I I I I I I I I I 

I I I I I 

I P P P P P 
Project coordinator 

Contracting officer 

User activity (tech. experts) 

!B Contractor personnel 

Agency functional areas 

Agency computer group 

I nternal auditors 

Upper management 

General ~ounsel 

RESPONSIBILITY MATRI

GUIDANCE ITEMS 

25 26 27 28 29 30 31 32 33 34 35 36 3

I I I J P I I I I P P 

P P P I I P P P P P P I I 

I I I I I I I 

I I I 

I I I I I I 

I J I I I I I 

I 

I I I I 
Project coordinator 

Contracting officer 

User activity (tech, experts) 

Contractor personnel 

~ Agency functional areas 

Agency computer group 

In'(ernal auditors 

Upper Management 

General Counsel 

RESPONSIBILITY MATRIX 

GUIDANCE ITEMS 

46 47 48 49 50 51 52 53 54 55 56 57 58 

I I I P J J P P P P I P 

P P P P I J J I I I I 

I I I I I 

J I 

I I I I I 

I I I 

P 

I 

I 
CONTRIBUTION SOURCES 

• Executive agencies: 

Department of Agriculture 

Department of Commerce 

Department of Defense 

Department of the Air Force 

Department of the Army 

Department of Defense Computer Institute 

Department of the Navy 

Department of Health, Education, and Welfare 

Department of Labor 

Department of Transportation 

General Services Administration 

Office of Federal Procurement Policy 

Office of Management and Budget 

Sm ithsonian Institution 

• Accounting firms: 

Alexander Grant & Company 

Arthur Andersen & Co. 

Arthur Young & Company 

Coopers & Lybrand 

Elmer Fox & Company 

Ernst & Ernst 

Haskins & Sells 

Peat Marvvick Mitchell & Co. 

Price Waterhouse & Co. 

Stay, Malone and Company 

Touche Ross & Co. 

• Consulting firms: 

Booz Allen & Hamilton, Inc. 

McKinsey & Company, Inc. 

Planning Research Corporation 

TRW Systems Group 

• Professional societies: 

American Institute of Certified Public Accountants 

(various committees, task forces, and technical 

directors) 

Association of Government Accountants 

District of Columbia Institute of Certified Public 

Accountants 

National Council of Professional Service Firms 

.Joint Financial Management Improvement Program Staff 

61