GAO

United States General Accounting Office

Report to the Chairman, Committee on

Small Business, U.S. Senate

May 2000 INFORMATION

TECHNOLOGY

MANAGEMENT

SBA Needs to

Establish Policies

and Procedures for

Key IT Processes

GAO/AIMD-00-170

Page 1 GAO/AIMD-00-170 SBA IT Management

United States General Accounting Office

Washington, D.C. 20548

Accounting and Information

Management Division

B-285295 Letr

May 31, 2000

The Honorable Christopher S. Bond

Chairman

Committee on Small Business

United States Senate

Dear Mr. Chairman:

As the Small Business Administration (SBA) tries to transform itself into a

“21st Century leading edge financial institution,” it needs to identify and

address operational problems that have agencywide implications.

Evaluating SBA’s management of information technology (IT) is a critical

part of efforts to assess whether it has a sound foundation for addressing

these problems. As you requested, our objective was to evaluate SBA’s IT

management in five key IT process areas: investment management,

architecture, information security, software development and acquisition,

and human capital management. On April 7, 2000, we briefed your office on

the results of this work. The briefing slides are included in appendix I.

This report provides a high-level summary of the information presented at

the briefing, including (1) background on SBA’s mission and programs, IT

environment, budgets, and staffing and (2) our review of SBA’s policies,

procedures, and practices in each IT area. SBA provided us with comments

on a draft of the briefing, and we considered those comments in developing

this report. SBA’s comments are discussed in the “Agency Comments and

Our Evaluation” section and are reprinted in appendix II.

Results in Brief Although SBA plans to improve its key IT processes, many of SBA’s policies

and procedures for managing IT are currently in draft form or not yet

developed. Specifically, SBA has not yet established policies to manage IT

investments and human capital. In addition, procedures for maintaining

SBA’s enterprisewide IT architecture and for implementing information

security policies are still in draft form and incomplete. Also, standards and

procedures to support new software development are being adopted, and

IT guidance for software acquisition is obsolete. In each of these areas,

SBA intends to implement needed policies and procedures.
B-285295

Page 2 GAO/AIMD-00-170 SBA IT Management

While SBA intends to pursue best practices for IT planning, monitoring, and

evaluation, its current practices do not generally adhere to defined

processes. In particular, investment management activities are limited

largely to reviewing IT proposals, architecture related activities are

performed without a defined process, and software development and

acquisition practices are predominantly ad-hoc. In the information security

area, SBA lacks centralized oversight of the activities of its field and

program offices. In addition, risk assessments have not been performed

periodically on all mission-critical systems and security training has not yet

been provided to employees and contractor staff. Human capital

management activities are limited to a non-IT-specific training needs

survey, and a human capital assessment has not been performed to identify

short- and long-term IT knowledge and skills requirements. To its credit,

SBA recognizes many of these IT management weaknesses and plans to

make improvements in each key process area.

To improve SBA’s IT management, we have made a number of

recommendations in each area. SBA has agreed with our recommendations

and has stated that efforts are underway to address them. SBA also

emphasized that it is committed to improving IT management practices.

Background SBA’s mission is to maintain and strengthen the nation’s economy by aiding,

counseling, assisting, and protecting the interests of small business and by

helping businesses and families recover from natural disasters. SBA

administers small business programs, including 8(a)1 federal contracting

set-asides and 7(a)2 loans to help economically disadvantaged firms start,

grow, and stay in business. SBA’s disaster loan program offers financial

assistance to businesses and families trying to rebuild in the aftermath of a

disaster.

1

Sec. 8(a), Small Business Act, 15 USC 637(a): SBA’s 8(a) program assists in the development

of small companies that are owned and operated by socially and economically

disadvantaged individuals. An 8(a) company is eligible for federal contracting set-asides and

other business development support to gain access to the economic mainstream.

2

Sec. 7(a), Small Business Act, 15 USC 636(a): the 7(a) loan program is for business start-ups

and to meet the varied short- and long-term needs of existing small businesses. Under 7(a),

SBA guarantees loans to small businesses that cannot obtain financing on reasonable terms

through other channels.
B-285295

Page 3 GAO/AIMD-00-170 SBA IT Management

For fiscal year 2000, SBA’s budget request was about $995 million, including

$762 million in regular appropriations and $233 million for

contingency/emergency appropriations to support the disaster loan

program. Based on the total IT budget expenditures incurred by the Office

of the Chief Information Officer, the Office of Disaster Assistance, and the

Office of the Chief Financial Officer, SBA had an average IT budget of

about $39 million annually from fiscal year 1997 through fiscal year 2000. IT

expenditures were primarily for operations and maintenance activities, and

limited funds were allocated for systems development activities and IT

training.

To support the management of its programs, SBA depends on its IT

environment, which includes 42 mission-critical systems running on legacy

mainframe and minicomputers. Ten of these systems support

administrative activities, the remaining 32 support loan activities, including

loan accounting and collection, loan origination and disbursement, and

loan servicing and debt collection.

SBA’s self-assessment of its IT environment has shown that the legacy

systems are not effectively integrated and thus provide limited information

sharing. The assessment has also shown that SBA cannot depend on the

systems to provide consistent information. Because of these problems,

SBA has embarked on an agencywide systems modernization initiative to

replace its outmoded legacy systems.

In fiscal year 1999, SBA reported having 127 IT staff to set policies, plan and

oversee IT projects, operate and maintain computer systems, and provide

computer training to employees. Also, SBA used about the same number of

contractor staff for technical support and day-to-day operations and

maintenance of systems.

Investment

Management Policies

and Procedures Are

Needed, Limited

Project Selection

Reviews Were

Performed

IT investment management is an integrated approach that provides for the

life-cycle management of IT investments. This investment process requires

three essential phases: selection, control, and evaluation. In the selection

phase, the organization determines priorities and makes decisions about

which projects will be funded based on the technical soundness of the

projects, their contribution to mission needs, performance improvement

priorities, and overall IT funding levels. The costs, benefits, and risks of all

IT projects are assessed and the projects are compared against each other

and ranked. In the control phase, all projects are consistently controlled

and managed. Progress reviews, in which progress is compared against
B-285295

Page 4 GAO/AIMD-00-170 SBA IT Management

projected cost, schedule, and expected mission benefits, are conducted at

key milestones in each project’s life cycle. The evaluation phase compares

actual performance against estimates to identify and assess areas in which

future decision-making can be improved.

SBA has made progress in establishing an investment review board and is

beginning to define an investment selection process. However, it has not

yet established IT investment management policies and procedures to help

identify and select projects that will provide mission-focused benefits and

maximum risk-adjusted returns. Likewise, SBA has not yet defined

processes for investment control and evaluation to ensure that selected IT

projects will be developed on time, within budget, and according to

requirements and that these projects will generate expected benefits.

Regarding investment management practices, SBA has performed only

limited reviews of major IT investments and these reviews were ad-hoc

since little data have been captured for analyzing benefits and returns on

investments.

Without established policies and defined processes for IT investment

management practices, SBA cannot ensure that consistent selection

criteria are used to compare costs and benefits across project proposals,

that projects are monitored and provided with adequate management

oversight, or that completed projects are evaluated to determine overall

organizational performance improvement. In addition, the agency lacks

assurance that the collective results of postimplementation reviews across

completed projects will be used to modify and improve investment

management based on lessons learned.

To address IT investment management weaknesses, SBA plans to develop

and implement an investment selection process that includes screening,

scoring, and ranking proposals. It also plans to use its target architecture to

guide IT investments. In addition, SBA plans to develop and implement an

investment control process to oversee and control projects on a quarterly

basis. As part of investment control, SBA plans to collect additional data

from all investment projects and compare actual data with estimates in

order to assess project performance.
B-285295

Page 5 GAO/AIMD-00-170 SBA IT Management

IT Architecture

Maintenance

Procedures Have Not

Been Established

An IT architecture is a blueprint—consisting of logical and technical

components—to guide the development and evolution of a collection of

related systems. At the logical level, the architecture provides a high-level

description of an organization’s mission, the business functions being

performed and the relationships among the functions, the information

needed to perform the functions, and the flow of information among

functions. At the technical level, the architecture provides the rules and

standards needed to ensure that the interrelated systems are built to be

interoperable and maintainable.

SBA has made progress with its target IT architecture by describing its core

business processes, analyzing information used in the business processes,

describing data maintenance and data usage, identifying standards that

support information transfer and processing, and establishing guidelines to

migrate current applications to the planned environment. However,

procedures do not exist for change management to ensure that new system

installations and software changes will be compatible with other systems

and SBA’s planned operating environment.

Without established policies and systematic processes for IT architecture

activities, SBA cannot ensure that it will develop and maintain an

information architecture that will effectively guide efforts to migrate

systems and make them interoperable to meet current and future

information processing needs.

To address IT architecture weaknesses, SBA plans to establish a change

management process for architecture maintenance to ensure that new

system installations and software changes will be compatible with other

systems and SBA’s planned operating environment. In addition, it plans to

incorporate in the target architecture specific security standards for

hardware, software, and communications.

Systems Development

Procedures Are Being

Adopted, Software

Acquisition Guidelines

Are Obsolete, Practices

Are Inconsistent

To provide the software needed to support mission operations, an

organization can develop software using its staff or acquire software

products and services through contractors. To effectively manage software

development and acquisition processes, the organization needs to establish

policies and procedures and assign organizational responsibilities for their

implementation. To manage its software projects, the organization should

have well-defined software development and acquisition processes,

including the methodologies and standards that will be used. Key processes
B-285295

Page 6 GAO/AIMD-00-170 SBA IT Management

for software development include requirements management, project

planning, project tracking and oversight, quality assurance, and

configuration management. Additional key processes needed for software

acquisition include acquisition planning, solicitation, contract tracking and

oversight, product evaluation, and transition to support.

SBA lacks policies for software development and acquisition to help

produce information systems within the cost, budget, and schedule goals

set during the investment management process that at the same time

comply with the guidance and standards of its IT architecture. SBA’s IT

guidance and procedures for software acquisition are obsolete and thus

rarely used for acquisition planning, solicitation, contract tracking and

oversight, product evaluation, and transition to support. An existing

systems development methodology is being adopted to replace outdated

guidelines that lack key processes for software development. Our review of

the selected software projects indicates that SBA’s practices are typically

ad-hoc for project planning, project tracking and oversight, quality

assurance, and configuration management.

Without established policies and defined processes for software

development and acquisition, practices will likely be ad-hoc and not adhere

to generally accepted standards. Key activities, such as requirements

management, planning, configuration management, and quality assurance,

will be inconsistently performed or not performed at all when project

managers are faced with time constraints or limited funding. These

weaknesses can delay delivery of software products and services and lead

to cost overruns.

To address software development and acquisition weaknesses, SBA plans

to implement formal practices, such as software requirements management

and configuration management on a project basis before establishing these

practices agencywide. Specifically, SBA has selected the Loan Monitoring

System (LMS) project as a starting point for identifying, developing, and

implementing a new systems development methodology and associated

policies, procedures, and practices. LMS therefore will serve as a model for

future systems development projects.
B-285295

Page 7 GAO/AIMD-00-170 SBA IT Management

Information Security

Procedures Are Still in

Draft Form, Periodic

Risk Assessments Are

Not Performed

Information security policies address the need to protect an organization’s

computer-supported resources and assets. Such protection ensures the

integrity, appropriate confidentiality, and availability of the data and

systems of an organization. Integrity ensures that data have not been

altered or destroyed in an unauthorized manner. Confidentiality ensures

that information is not made available or disclosed to unauthorized

individuals or entities. Availability ensures that data will be accessible or

usable upon demand by an authorized entity.

Key activities for managing information security include risk assessment,

awareness, controls, evaluation, and central management. Risk

assessments consist of identifying threats and vulnerabilities to

information assets and operational capabilities, ranking risk exposures,

and identifying cost-effective controls. Awareness involves promoting

knowledge of security risks and educating users about security policies,

procedures, and responsibilities. Evaluation involves monitoring

effectiveness of controls and awareness activities through periodic

evaluations. Central management involves coordinating security activities

through a centralized group.

SBA’s computer security procedures for systems certification and

accreditation are in draft form. With respect to information security

activities, SBA has not conducted periodic risk assessments for all missioncritical systems; the agency only recently conducted a risk assessment for

one system. Training and education have not been provided to promote

security awareness and responsibilities of employees and contractor staff.

Further, security management responsibilities are currently fragmented

among all of SBA’s field and program offices.

Without security policies, SBA faces increased risk that critical information

and assets may not be protected from inappropriate use, alteration, or

disclosure. Without defined procedures, practices are likely to be

inconsistent for such activities as periodic risk assessments, awareness

training, implementation of controls, and evaluation of policy compliance

and effectiveness of controls.

To address information security weaknesses, SBA has hired additional staff

to develop procedures to implement computer security policies and to

manage computer accounts and user passwords. These staff are also

responsible for performing systems security certification reviews of new
B-285295

Page 8 GAO/AIMD-00-170 SBA IT Management

and existing IT systems. In addition, SBA plans to finish development and

testing of a comprehensive disaster recovery and business continuity plan.

Human Capital Policies

and Procedures Are

Needed, Workforce

Strategies and Plans

Are Not Yet Developed

The concept of human capital centers on viewing people as assets whose

value to an organization can be enhanced through investment. As the value

of people increases, so does the performance capacity of the organization

and therefore its value to clients and other stakeholders. To maintain and

enhance the capabilities of IT staff, the agency should conduct four basic

activities: (1) assess the knowledge and skills needed to effectively perform

IT operations to support the agency mission and goals; (2) inventory the

knowledge and skills of current IT staff to identify gaps in needed

capabilities; (3) develop strategies and implementation plans for hiring,

training, and professional development to fill the gap between

requirements and current staffing; and (4) evaluate progress made in

improving IT human capital capability and use the results of these

evaluations to continuously improve the organization’s human capital

strategies.

SBA has not established policies and procedures to identify and address its

short- and long-term requirements for IT knowledge and skills. Similarly,

SBA has not conducted an agencywide assessment to determine gaps in IT

knowledge and skills in order to develop workforce strategies and

implementation plans. Further, SBA has not yet evaluated its progress in

improving IT human capital capabilities or used data to continuously

improve human capital strategies.

Without established policies and procedures for human capital

management, SBA lacks assurance that it adequately identifies the IT

knowledge and skills needed to support its mission, develops appropriate

workforce strategies, and plans to hire and train staff to effectively perform

IT operations.

To address IT human capital management weaknesses, SBA plans to

conduct a comprehensive assessment of training needs with a special

emphasis on the needs of its IT staff. The survey is scheduled for fiscal year

2001 and will be conducted at both headquarters and SBA field offices.

Recommendations To improve IT management practices, we recommend that the SBA

Administrator direct the Chief Information Officer (CIO) to establish
B-285295

Page 9 GAO/AIMD-00-170 SBA IT Management

policies and procedures for managing information technology and define

and implement processes for each of the following areas:

In the investment management area, we recommend that the Administrator

direct the CIO to adopt policies and procedures and define processes for

• investment selection to ensure that IT projects result in mission-focused

benefits and that risk-adjusted return on investment is maximized;

• investment control to determine whether selected projects are being

developed on time, within budget, and according to requirements, and to

take corrective actions as appropriate; and

• investment evaluation by conducting postimplementation reviews to

determine whether completed projects are generating expected

mission-focused benefits.

In the IT architecture area, we recommend that the Administrator direct

the CIO to

• develop a systematic process for architecture development to ensure

that the architecture will meet the agency’s current and future

information processing needs,

• establish policies and procedures for architecture maintenance to

ensure that new systems and software changes are compatible with

other systems and SBA’s planned operating environment, and

• set a target date for implementation of the maintenance processes.

For software development and acquisition, we recommend that the

Administrator direct the CIO to

• complete the systems development methodology and develop a plan to

institutionalize and enforce its use agencywide, and

• establish policies, procedures, and processes for software development

and software acquisition and develop a mechanism to enforce them.

These policies, procedures, and processes need to address areas such as

requirements management, project planning, project tracking and

oversight, software quality assurance, configuration management,

acquisition planning, solicitation, contract tracking and oversight,

product evaluation, and transition to support.
B-285295

Page 10 GAO/AIMD-00-170 SBA IT Management

For information security, we recommend that the Administrator direct the

CIO to

• conduct periodic security risk assessments to identify and rank threats

and vulnerabilities;

• implement a complete, effective security awareness program;

• periodically update policies and procedures on information security and

implement security controls to address identified vulnerabilities;

• complete the development and testing of its comprehensive disaster

recovery and business continuity plan, which should then be updated

and tested periodically;

• conduct periodic security evaluations to determine whether policies,

procedures, and controls are effective against identified vulnerabilities

and take remedial action as needed; and

• develop and implement a centralized mechanism to monitor and enforce

compliance on information security by employees, contractors, and

program offices.

In the human capital management area, we recommend that the SBA

Administrator direct the CIO to

• identify SBA’s IT knowledge and skills requirements,

• perform periodic IT staff assessments to identify current levels of IT

knowledge and skills,

• develop workforce strategies and implement plans to acquire and

maintain the necessary IT knowledge and skills to support the agency

mission, and

• periodically evaluate progress in improving SBA’s IT human capital

capability and use the results to continuously improve human capital

strategies.

Agency Comments and

Our Evaluation

In its written comments on a draft of the briefing, SBA agreed with our

recommendations and stated that actions are already underway to address

many of them. SBA also agreed with our findings but expressed concerns

about the presentation of results, some statements in the draft briefing that

do not reflect SBA’s latest status, and assumptions on the appropriate level

of detail in SBA planning documents.

Concerning the presentation of results, SBA requested that we clearly

describe our assessment criteria to allow for a fair interpretation of its

findings—since many of these criteria include industry standards that had
B-285295

Page 11 GAO/AIMD-00-170 SBA IT Management

emerged only in the last few years. Our briefing slides identify the criteria

and standards that we applied in assessing SBA IT management. These

standards have sufficient flexibility to make possible the development of

key IT processes appropriate for the size and complexity of the IT

environment of any organization.

SBA also contended that other small federal agencies would not show

compliance much beyond SBA’s. We note that SBA is the first federal

agency for which we have used indicators to graphically depict our

evaluation results. Regardless of where SBA operations may stand relative

to similar size federal agencies, comparison with industry standards is a

sound approach for identifying activities that can be improved to enhance

the capability of supporting the agency’s mission and obtaining a positive

return on IT investment.

Concerning statements in the draft briefing report that do not reflect SBA’s

current status and our assumptions on the level of detail in SBA planning

documents, we updated appropriate briefing slides to include information

recently provided by SBA. Appendix II contains specific revisions made to

the briefing report and also provides the full text of SBA’s comments and

our responses to comments not discussed above.

The SBA Deputy Administrator also provided oral comments on a draft of

this letter. He was concerned that our report did not fully reflect SBA’s

commitment to improve IT management as demonstrated in its recent

actions in planning for the loan monitoring system and suggested that we

recognize this. We agree that SBA has demonstrated a commitment to

improve IT management and, accordingly, we made changes to reflect this

comment in this report.

Objective, Scope, and

Methodology

As requested, our objective was to evaluate SBA’s management of

information technology in the areas of investment management,

architecture, software development and acquisition, information security,

and human capital management. These five key areas encompass major IT

functions and are recognized by the IT industry as having substantial

influence over the effectiveness of operations. In each IT area, we reviewed

SBA’s IT policies and procedures and compared them against applicable

laws and regulations, federal guidelines, and industry standards. We

evaluated SBA’s IT management using the Clinger-Cohen Act, Computer

Security Act, and guidelines issued by the Chief Information Officer’s

Council, the Office of Management and Budget, the General Services
B-285295

Page 12 GAO/AIMD-00-170 SBA IT Management

Administration, the National Institute of Standards and Technology, the

Software Engineering Institute, the Institute of Electrical and Electronics

Engineers, Inc. (IEEE), and ourselves. We also reviewed selected SBA IT

projects and activities to determine if practices complied with SBA’s

policies and procedures and industry standards. The projects selected for

review included the Loan Monitoring System, SmartStream, PRO-Net,

HubZones, and Subsidy Rate. These selected projects represent a mix of

ongoing and completed IT projects of various cost and duration. We also

reviewed activities related to current investments.

For each IT area we reviewed, we depicted our evaluation results and

judgments on the current state of SBA policies, procedures, and practices

by using three broad indicators. SBA is the first federal agency in which we

have used these indicators to graphically represent our assessment results.

Accordingly, there is no basis for comparing SBA against other agencies

using this type of depiction.

We conducted our review at various SBA headquarters offices including the

Office of the Chief Information Officer, the Office of Disaster Assistance,

the Office of the Chief Financial Officer, the Office of Human Resources,

and the Office of Field Operations. We also worked at the Office of

Financial Systems in Denver and at the Disaster Office in Sacramento. We

conducted our work from August 1999 through April 2000 in accordance

with generally accepted government auditing standards.

As agreed with your office, unless you publicly announce the contents of

this report earlier, we will not distribute it until 30 days from the date of

this letter. At that time, we will send copies to the Honorable Aida Alvarez,

Administrator, Small Business Administration; the Honorable Jacob J. Lew,

Director, Office of Management and Budget; and other interested parties.

Copies will also be made available to others upon request.

If you have questions on matters discussed in this report, please contact me

at (202) 512-6253, or James R. Hamilton, Assistant Director, at (202) 512-

6271. We can also be reached at willemssenj.aimd@gao.gov and

hamiltonj.aimd@gao.gov, respectively. Key contributors to this report were
Page 13 GAO/AIMD-00-170 SBA IT Management

William G. Barrick, John T. Christian, Mike J. Dolak, Myong S. Kim, Anh Q.

Le, Thomas F. Noone, Edward R. Tekeley, and Hai V. Tran.

Sincerely yours,

Joel C. Willemssen

Director, Civil Agencies Information Systems
Page 14 GAO/AIMD-00-170 SBA IT Management

Appendix I

Briefing on Small Business Administration’s Apendxies

Management of Information Technology ApendxiI

Small Business Administration’s

Management of Information

Technology

Briefing for Committee on Small Business

United States Senate

April 7, 2000
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 15 GAO/AIMD-00-170 SBA IT Management

2

Purpose and Outline

• Briefing purpose is to present results of our review and analysis of the Small

Business Administration’s (SBA) management of information technology (IT).

• Briefing outline:

• Objective

• Scope & Methodology

• SBA’s IT Profile

• SBA’s IT Policies, Procedures, and Practices

• Investment Management

• Architecture

• Software Development and Acquisition

• Information Security

• Human Capital
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 16 GAO/AIMD-00-170 SBA IT Management

3

Objective

Our objective was to evaluate SBA’s information technology policies,

procedures, and practices in the areas of investment management, architecture,

software development and acquisition, information security, and human capital.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 17 GAO/AIMD-00-170 SBA IT Management

4

Scope & Methodology

• We reviewed SBA’s IT policies and procedures for investment management,

architecture, software development and acquisition, information security, and

human capital and compared them with applicable laws and regulations,

federal guidelines, and industry standards.

• We reviewed selected IT projects and activities to determine if practices

comply with agency’s policies and procedures and industry standards. The

selected projects represent a mix of ongoing and completed IT projects of

various costs and duration. We also reviewed activities related to current

investments.

• We conducted the review at various SBA headquarters offices including the

Office of the Chief Information Officer (OCIO), the Office of Disaster

Assistance (ODA), the Office of the Chief Financial Officer (OCFO), the Office

of Human Resources, and the Office of Field Operations. We also worked at

the Office of Financial Systems in Denver and at the Disaster Office in

Sacramento. We conducted our work from August 1999 through April 2000, in

accordance with generally accepted government auditing standards.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 18 GAO/AIMD-00-170 SBA IT Management

5

IT Profile

Mission and Programs

• SBA’s mission is to maintain and strengthen the nation’s economy by aiding,

counseling, assisting, and protecting the interests of small business and by

helping businesses and families recover from natural disasters.

• SBA’s programs include

• 7(a) loans for business start-ups and existing small businesses,

• 8(a) federal contracting set-asides to assist business development of small

companies owned and operated by individuals who are determined by SBA to be

socially and economically disadvantaged, and

• disaster assistance loans for disaster victims, both businesses and individuals.

• SBA’s budget request for fiscal year 2000 was about $995 million, including

$762 million in regular appropriations and $233 million for

contingency/emergency appropriations to support the disaster loan program.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 19 GAO/AIMD-00-170 SBA IT Management

6

IT Profile

IT Environment

• To support the management of its programs, SBA depends on 42 missioncritical systems running on legacy mainframe and minicomputers. Ten of

these systems support administrative activities. The remaining 32 support

SBA loan activities, including loan accounting and collection, loan origination

and disbursement, and loan servicing and debt collection.

• SBA’s self-assessment of its IT environment has shown that legacy application

systems are not effectively integrated and thus provide limited informationsharing. The self-assessment also showed that SBA cannot depend on the

systems to provide consistent information. Because of these problems, SBA

has embarked on an agencywide modernization effort.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 20 GAO/AIMD-00-170 SBA IT Management

7

IT Profile

IT Environment (continued)

• Phase 1 consists of the Loan Monitoring System (LMS), which is expected to

aid SBA in managing its loan guarantee programs. The system is intended to

support loan monitoring and lender oversight.

• Phase 2 consists of two programs: the Joint Accounting and Administrative

Management System, which is intended to modernize SBA’s existing financial

management, human resource, and procurement systems; and the Credit

Management Modernization, which is intended to create a fully integrated

paperless process for disaster-relief home loans.

• Phase 3 consists of SBA’s IT programs to modernize systems supporting

government contracting, entrepreneurial development, and minority enterprise

development.

SBA’s systems modernization initiative consists of three phases:
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 21 GAO/AIMD-00-170 SBA IT Management

8

IT Profile

IT Responsibilities and Functions of SBA’s CIO

The Chief Information Officer (CIO) is the principal advisor to the Administrator

on IT matters and has overall responsibility for development, procurement,

management, and monitoring of enterprise-wide IT systems, projects, personnel,

and expenditures. The CIO is responsible for ensuring agency compliance with

governing laws and regulations and with implementing polices that prescribe the

use and management of information technology, such as the Clinger-Cohen Act,

the Paperwork Reduction Act, OMB Circular A-130, the Computer Security Act

of 1987, and Presidential Decision Directives 63 and 67. The CIO also oversees

organizational units for voice and data telecommunications, end user support,

and electronic services (Internet and World Wide Web homepage).
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 22 GAO/AIMD-00-170 SBA IT Management

9

IT Profile

IT Functions of Other SBA Organizations

• ODA is responsible for administering SBA’s Disaster Assistance Program

through four area offices: Niagara Falls, Atlanta, Ft. Worth, and Sacramento.

It operates the Automated Loan Control System at each area office, and is

responsible for maintaining the system software and hardware. It is also

involved in the development and acquisition of systems.

• OCFO is responsible for overseeing all financial management activities. It

operates systems at its Denver Finance Center. These systems perform

functions such as: (1) exchanging data with business partners, (2) processing

and maintaining disbursement and collection records, and (3) interfacing with

the Loan Accounting System. This office is also involved in the development

and acquisition of systems.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 23 GAO/AIMD-00-170 SBA IT Management

10

IT Profile

IT Budgets

• Total reported IT budgets for

OCIO, ODA, and OCFO

averaged about $39 million

per fiscal year from 1997 to

2000.1

• OCIO, ODA, and OCFO data

shows that IT costs were

primarily for operations and

maintenance (O&M) activities

and that limited funds were

allocated for systems

development activities and

training.

Source: SBA.

1 SBA does not maintain an agencywide IT budget.

$ Million

0

5

10

15

20

25

30

35

40

45

1997 1998 1999 2000

Other

Development

O&M
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 24 GAO/AIMD-00-170 SBA IT Management

11

IT Profile

IT Staffing

• In fiscal year 1999, SBA reported

having 127 IT staff (64 in OCIO, 44 in

ODA, and 19 in OCFO). OCIO staff

set IT policies, plan and oversee IT

projects, and administer infrastructure

systems. IT staff at ODA and OCFO

operate and maintain their computer

systems and provide IT training to

their users.

• SBA also used contractor staff for

technical support and day-to-day O&M

of IT systems. In fiscal year 1999,

SBA used 128 contractor staff in total

to support IT activities at OCIO, ODA,

and OCFO.

Source: SBA.

FY 1999 Staff

0

20

40

60

80

100

120

OCIO ODA OCFO

IT staff

Contractor staff
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 25 GAO/AIMD-00-170 SBA IT Management

12

IT Areas Evaluated

To evaluate IT management, we focused on five key areas that encompass

major IT functions and are recognized by the industry as having substantial

influence over the effectiveness of operations:

• IT investment management helps select projects that will best support mission needs,

provide an optimum return on investment, and control project development to identify

problems and quickly solve them. Investment management has three essential phases--

selection, control, and evaluation--that are supported by processes, data, and decisions.

• IT information architecture helps align the requirements for agency-sponsored

information systems with the processes that support the agency’s mission and goals,

achieve interoperability and security of information systems, and promote the application

and maintenance of standards by which the agency evaluates and acquires systems.

The information architecture has components that delineate the (1) business processes,

(2) information flows and relationships, (3) applications, (4) data descriptions and

relationships, (5) technology infrastructure, (6) technical reference model, and

(7) standards profiles. To implement and maintain the architecture, an agency should

have processes for change management and legacy systems integration.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 26 GAO/AIMD-00-170 SBA IT Management

13

IT Areas Evaluated (continued)

• Software development and acquisition activities help produce information systems

within the cost, budget, and schedule goals set by the investment management process,

while complying with the guidance and standards of the information architecture. Key

processes for software development include requirements management, project

planning, project tracking and oversight, quality assurance, and configuration

management. Additional key processes that are needed for software acquisition are

acquisition planning, solicitation, contract tracking and oversight, evaluation, and

transition to support.

• Information security helps protect the integrity, confidentiality, and availability of the

agency’s data and systems it relies on by reducing the risks of tampering, unauthorized

intrusions and disclosures, and serious disruptions of operations. Information security

activities include conducting risk assessments, promoting awareness, implementing

controls, performing evaluations, and providing centralized coordination and oversight of

all security activities.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 27 GAO/AIMD-00-170 SBA IT Management

14

IT Areas Evaluated (continued)

• IT human capital management helps provide employees with the appropriate

knowledge and skills to effectively execute critical IT functions. Key processes for

human capital management involve assessing IT knowledge and skills requirements,

inventorying existing staff’s knowledge and skills and assessing them against

requirements, developing strategies and plans to fill the gap between requirements and

existing staffing, and evaluating and reporting on progress in filling the gap in knowledge

and skills.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 28 GAO/AIMD-00-170 SBA IT Management

15

IT Policies, Procedures, and Practices

Evaluation Indicators

In evaluating the five key IT areas at SBA, we assessed applicable policies,

procedures, and practices. We use three broad indicators to depict our results:

Blank Circle indicates that policies and procedures do not exist or are

substantially obsolete or incomplete; and practices for planning, monitoring and

evaluation are predominantly ad hoc, or not performed.

Half Circle indicates that policies and procedures are predominantly current

and facilitate key functions; and selected key practices for planning, monitoring,

and evaluation have been implemented.

Solid Circle indicates that policies and procedures are current and

comprehensive for key functions; and practices for planning, monitoring, and

evaluation adhere to policies, procedures, and generally accepted standards.

For each of the areas we reviewed, these indicators provide our judgment on the

current state of SBA policies, procedures, and practices. SBA is the first federal

agency in which we have used these indicators to represent our assessment of

the five key IT areas. Accordingly, there is no basis to judge how SBA is

performing in relation to other agencies.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 29 GAO/AIMD-00-170 SBA IT Management

16

IT Policies, Procedures, and Practices

Evaluation Summary

Investment management

Requirements management

Project planning

Project tracking & oversight

Software development & acquisition

Quality assurance

Configuration management

Acquisition planning

Solicitation

Contract tracking & oversight

Product evaluation

Transition to support

Selection process

Selection data

Selection decisions

Control process

Control data

Control decisions

Evaluation process

Evaluation data

Evaluation decisions

Business processes

Information flows & relationships

Architecture

Applications

Data descriptions & relationships

Risk assessments

Central management

Security

Controls

Awareness

Evaluation

Requirements

Workforce strategies & plans

Progress evaluation

Human capital

Inventory

Architecture

Technical reference model

Standards profiles

Change management

Legacy systems integration

Technical infrastructure

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 30 GAO/AIMD-00-170 SBA IT Management

17

• In the selection phase, the organization determines priorities and makes decisions

about which projects will be funded during the year. The costs, benefits, and risks of all

IT projects are assessed and the projects are compared against each other and ranked.

• In the control phase, all projects are consistently controlled and managed. Progress

reviews, in which progress is compared against projected cost, schedule, and expected

mission benefits, are conducted at key milestones in each project's life cycle.

• The evaluation phase completes the IT investment management process by comparing

actuals against estimates in order to assess performance and identify areas in which

future decision-making can be improved.

IT Policies, Procedures, and Practices

IT Investment Management -- Overview

IT investment management is an integrated approach that provides for the

continual identification, selection, control, life-cycle management, and evaluation

of IT investments. An IT investment management process should have three

essential phases--selection, control, and evaluation.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 31 GAO/AIMD-00-170 SBA IT Management

18

Each phase is supported by

IT Policies, Procedures, and Practices

IT Investment Management -- Overview (continued)

• the processes that the organization is using to select, manage, and evaluate its IT

investments,

• the data (cost, benefit, and risk) that are being used to make IT decisions, and

• the IT decisions that are being made using the defined processes and data.

We evaluated IT investment management using the Clinger-Cohen Act, OMB’s

Capital Programming Guide, and GAO’s guide Assessing Risks and Returns:

A Guide for Evaluating Federal Agencies’ IT Investment Decision-making. We

reviewed IT investment management practices for the current SBA investment

portfolio. This portfolio includes the Loan Monitoring System, Joint Accounting

and Administrative System, and Credit Management Modernization.

Our evaluation covered three phases of investment management: selection,

control, and evaluation. For each phase, we evaluated investment processes,

investment data, and investment decisions.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 32 GAO/AIMD-00-170 SBA IT Management

19

Categorization is ad-hoc and not

necessarily based on risk-adjusted

return on investment. The Investment

Review Board was not involved in

making final selection decisions.

Project data are inconsistent or missing.

Little or no project data analysis

performed. Beginning to capture

investment data in a database.

SBA is beginning to establish a process

for screening, analyzing, and selecting

IT projects. The Investment Review

Board has been established and

performed limited project categorization.

Selection

decisions

Selection

data

Selection

process

Selection

Determine

priorities and

make decisions

about which

projects will be

funded

IT Policies, Procedures, and Practices

IT Investment Management -- Evaluation

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 33 GAO/AIMD-00-170 SBA IT Management

20

No evidence of control actions or

decisions to continue, modify, or

terminate IT projects exists. Similarly,

no evidence of IT investment portfolio

analysis exists.

IT project data are not captured and

analyzed to support control of IT

projects.

No policies and procedures for

controlling investments are in place. In

addition, no process is yet defined for

monitoring IT projects and comparing

actual data on costs, benefits,

schedules, and risks against original

estimates.

Control

decisions

Control

data

Control

process

IT Policies, Procedures, and Practices

IT Investment Management -- Evaluation (continued)

Control

Oversee

investments,

identify underperforming

investments,

implement

corrective action

plans

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 34 GAO/AIMD-00-170 SBA IT Management

21

No evidence of IT investment evaluation

exists to determine if investments are

performing as expected. Similarly,

there are no decisions to continue,

modify, or terminate projects. In

addition, there are no decisions to

improve the investment process.

IT project data are not captured and

post implementation reviews are not

performed to evaluate IT projects.

No policies and procedures for

evaluating investments exist. No post

implementation review process is yet

defined for assessing investment

performance and improving investment

management.

Evaluation

decisions

Evaluation

data

Evaluation

process

Evaluation

Conduct postimplementation

reviews and feed

lessons learned

back to the

selection and

control processes

SBA’s IT Policies, Procedures, and Practices

IT Investment Management -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 35 GAO/AIMD-00-170 SBA IT Management

22

IT Policies, Procedures, and Practices

Impact of IT Investment Management Weaknesses

• Without a selection process that screens, analyzes, and prioritizes IT

investments, SBA lacks assurance that IT selections will result in missionfocused benefits and that risk-adjusted return on investment is maximized.

• Without a control process that compares actual cost, benefit, schedule, and

risk data with original estimates, SBA lacks assurance that selected projects

are being developed on time, within budget, and according to requirements.

• Without an evaluation process that conducts post-implementation reviews,

SBA lacks assurance that completed projects are generating expected

mission-focused benefits.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 36 GAO/AIMD-00-170 SBA IT Management

23

IT Investment Management

Suggested Actions

SBA should adopt policies and procedures and define processes for

• investment selection, to ensure that IT projects result in mission-focused

benefits and that risk-adjusted return on investment is maximized.

• investment control, to determine whether selected projects are being

developed on time, within budget, and according to requirements, and to take

corrective actions as appropriate.

• investment evaluation by conducting post implementation reviews, to

determine whether completed projects are generating expected missionfocused benefits.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 37 GAO/AIMD-00-170 SBA IT Management

24

IT Policies, Procedures, and Practices

Plans to Address IT Investment Management Weaknesses

In March 2000, OCIO officials stated that by the end of fiscal year 2000, SBA

plans to

• develop and implement an investment selection process that includes

screening, scoring, and selecting projects;

• develop and implement an investment control process to oversee and control

projects on a quarterly basis;

• collect additional data from all investment projects; and

• compare actual data with estimates in order to assess project performance.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 38 GAO/AIMD-00-170 SBA IT Management

25

An IT architecture is a blueprint—consisting of logical and technical

components—to guide and constrain the development and evolution of a

collection of related systems. At the logical level, the architecture provides a

high-level description of an organization's mission, the business functions being

performed and the relationships among the functions, the information needed to

perform the functions, and the flow of information among functions. At the

technical level, the architecture provides the rules and standards needed to

ensure that the interrelated systems are built to be interoperable and

maintainable.

The Clinger-Cohen Act assigns the CIO the responsibility for developing,

implementing, and maintaining the architecture.

In developing the architecture, OMB guidelines specify that it include the

following components:

IT Policies, Procedures, and Practices

IT Architecture -- Overview

• Business processes describe the core business processes that support an agency’s

missions
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 39 GAO/AIMD-00-170 SBA IT Management

26

• Information flows and relationships analyze information used in business processes

and describe relationships among information flows; the flows indicate where the

information is needed and how the information is shared to support mission functions

• Applications identify, define, and organize activities that capture, manipulate, and

manage information to support the organization's mission

• Data descriptions and relationships describe how data are maintained, accessed,

and used

• Technology infrastructure describes the IT resources (e.g., hardware, software,

communications networks) and functional capabilities

• Technical reference model identifies the information services (e.g., database,

communications, security services) used throughout the agency

• Standards profiles include (1) the standards that support the information services

identified in the technical reference model, (2) the standards that are essential for

interoperability, and (3) information security profiles for information assurance

IT Policies, Procedures, and Practices

IT Architecture -- Overview (continued)
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 40 GAO/AIMD-00-170 SBA IT Management

27

• Change management -- managing and documenting changes to the architecture that

are needed as business functions evolve

• Legacy systems integration -- developing and implementing a strategy for interfacing

existing and new systems that will permit them to interoperate cost-effectively

IT Policies, Procedures, and Practices

IT Architecture -- Overview (continued)

We evaluated SBA’s IT architecture using the Clinger-Cohen Act, OMB’s

guidance, NIST guidelines, and the CIO Council’s Federal Enterprise

Architecture Framework. We reviewed SBA’s IT architecture practices for the

Loan Monitoring System.

The OMB guidelines specify that the organization should establish two key

processes to implement and maintain the architecture:
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 41 GAO/AIMD-00-170 SBA IT Management

28

Both SBA’s draft technology policy and

draft architecture describe the core

business processes that support the

agency’s mission. These core business

processes are decomposed into

business activities: Capital Access,

Entrepreneurial Development, Federal

Contracting and Minority

Entrepreneurial Development, Disaster

Assistance, and Advocacy.

Business

processes

Architecture

components

Describe

components of

enterprise

architecture

IT Policies, Procedures, and Practices

IT Architecture -- Evaluation

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 42 GAO/AIMD-00-170 SBA IT Management

29

Both SBA’s draft technology policy on

information architecture and the draft

architecture itself identify activities

performed by each business entity.

However, each entity’s responsibilities

are not clearly defined for a shared

activity. This prevents the effective

exchange of information among entities,

including the development of necessary

interfaces for information flows and

interoperability. Currently, each

business entity defines, collects, and

manages its own information. Further,

the structure and organization of

information used by new business

activities is not identified and defined.

Information flows

and relationships

Architecture

components

Describe

components of

enterprise

architecture

IT Policies, Procedures, and Practices

IT Architecture -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 43 GAO/AIMD-00-170 SBA IT Management

30

SBA’s draft technology policy describes

data resources and data management.

The draft architecture describes how

data are maintained, accessed, and

used. However, data quality measures,

data security rules, and data validation

rules have not been defined.

Data descriptions

and relationships

Both SBA’s draft technology policy and

draft architecture identify applications

that support business activities for

capturing and manipulating information.

Applications

Architecture

components

Describe

components of

enterprise

architecture

IT Policies, Procedures, and Practices

IT Architecture -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 44 GAO/AIMD-00-170 SBA IT Management

31

Both SBA’s draft technology policy and

the draft architecture identify technical

infrastructure standards that support

information transfer and processing.

Currently, not all information systems

conform to telecommunications

standards and network management

protocols.

Technology

infrastructure

Architecture

components

Describe

components of

enterprise

architecture

IT Policies, Procedures, and Practices

IT Architecture -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 45 GAO/AIMD-00-170 SBA IT Management

32

Both SBA’s draft technology policy and

the draft architecture identify standards

for information exchange, resource

sharing, and security services.

However, procedures have not been

established to ensure compliance.

SBA’s draft technology policy stated

that SBA adopted the Zachman

framework for its architecture

development. However, SBA has not

clearly defined how the framework will

be applied for the development of its

architecture. The products that specify

the contents of the framework also have

not been identified.

Standards

profiles

Technical

reference model

Architecture

components

Describe

components of

enterprise

architecture

IT Policies, Procedures, and Practices

IT Architecture -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 46 GAO/AIMD-00-170 SBA IT Management

33

SBA has now developed draft

guidelines to migrate current

applications to the planned

environment.

No procedures exist to ensure that

system changes comply with the

architecture. Change controls have not

been implemented to support

maintenance and implementation of the

architecture.

Legacy systems

integration

Change

management

Maintenance

and

implementation

Ensure that

changes,

removals, or

installations of

new software and

hardware are

compatible with

existing systems

and meet

architecture

requirements

IT Policies, Procedures, and Practices

IT Architecture -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 47 GAO/AIMD-00-170 SBA IT Management

34

IT Policies, Procedures, and Practices

Impact of IT Architecture Weaknesses

• Without a systematic process for developing an architecture and addressing

key architecture components, SBA lacks assurance that the architecture will

meet the agency’s current and future information processing needs.

• Without policies and procedures for architecture maintenance, SBA lacks

assurance that new systems and software changes will be compatible with

other systems and SBA’s planned operating environment.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 48 GAO/AIMD-00-170 SBA IT Management

35

IT Architecture

Suggested Actions

• SBA should develop a systematic process for architecture development to

ensure that the architecture will meet the agency’s current and future

information processing needs. It should also set target dates for completion

of each component of the architecture.

• SBA should establish policies and procedures for architecture maintenance

to ensure that new systems and software changes are compatible with other

systems and SBA’s planned operating environment. It should also set

target dates for full implementation of the maintenance processes.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 49 GAO/AIMD-00-170 SBA IT Management

36

IT Policies, Procedures, and Practices

Plans to Address IT Architecture Weaknesses

In March 2000, SBA officials stated that they plan to

• incorporate specific security standards for hardware, software, and

communications in the target architecture;

• use the target architecture to guide IT investments; and

• establish a change management process by the end of this fiscal year.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 50 GAO/AIMD-00-170 SBA IT Management

37

To provide the software needed to support mission operations, an organization

can develop software using its staff or acquire software through a contractor.

To effectively manage software development and acquisition processes, the

organization needs to establish policies and procedures and assign

organizational responsibilities for their implementation. To manage its software

projects, the organization should have well-defined software development and

acquisition processes, including the methodologies and standards that will be

used.

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Overview
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 51 GAO/AIMD-00-170 SBA IT Management

38

• Requirements management establishes and documents common understandings

between the customer and the software project of the customer's requirements to be

addressed.

• Project planning identifies and organizes the work elements for performing the

software engineering and managing the project.

• Project tracking and oversight measures and controls the performance, cost, and

schedule objectives of the project throughout its life. It provides visibility into actual

progress so that management can act effectively when the software project's

performance deviates significantly from plans.

• Software quality assurance determines if the process being used by the project and

the resulting products comply with the organization’s policies and procedures.

• Configuration management establishes and maintains the integrity of the products

throughout the project's software life cycle, through a structured process for

documenting proposed and approved changes in requirements and plans.

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Overview

(continued)

Key processes for software development include the following:
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 52 GAO/AIMD-00-170 SBA IT Management

39

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Overview

(continued)

• Acquisition planning identifies and organizes the work elements for the contractor to

perform the software engineering and the organization’s support and oversight of the

contractor.

• Solicitation details the solicitation and selection of contractors qualified to satisfy the

contract’s requirements for the project’s software-related products and services. The

solicitation package includes the contractual software requirements, proposal evaluation

criteria, and product-acceptance criteria.

• Contract tracking and oversight ensures that the contractor’s software engineering is

managed and complies with contract requirements and adheres to relevant laws,

policies, regulations, and other guidance.

• Product evaluation evaluates contractor products against technical requirements

throughout the total period of the acquisition to provide an integrated approach that

takes advantage of all evaluation results.

• Transition to support ensures that the software support organization has the capacity

and capability to provide the required support upon assumption of responsibility for the

support of the software products.

Additional key process areas needed for software acquisition include the following:
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 53 GAO/AIMD-00-170 SBA IT Management

40

SBA’s IT Policies, Procedures, and Practices

Software Development and Acquisition -- Overview

(continued)

We evaluated SBA’s policies and procedures on software development and

acquisition using GSA’s Guide to Planning, Acquiring, and Managing IT

Systems, and standards issued by the Software Engineering Institute (SEI) and

the Institute of Electrical and Electronics Engineers, Inc. We reviewed selected

projects to confirm the agency’s declaration of software processes in use, but did

not perform an SEI Capability Maturity Model study. We reviewed SBA’s IT

practices by performing case studies on four selected projects: SmartStream,

PRO-Net, Subsidy Rate, and HubZones.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 54 GAO/AIMD-00-170 SBA IT Management

41

SBA has a draft systems development

methodology that provides project

planning guidance. SBA has begun to

formalize practices for a major

modernization project. However,

project plans do not consistently contain

personnel assignments, cost estimates,

and milestones.

There are no policies and procedures

for requirements management, and

processes and guidance are not yet

defined. SBA has begun to formalize

this capability for two recent projects.

Project planning

Requirements

management

Software

development

Apply best

practices to

achieve key

objectives:

(1) within budget,

(2) on schedule,

and (3) according

to requirements

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Evaluation

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 55 GAO/AIMD-00-170 SBA IT Management

42

SBA has a draft systems development

methodology that provides guidance for

quality assurance. However, quality

assurance practices are not being

performed on any software projects.

Quality

assurance

SBA has a draft systems development

methodology that provides guidance for

project tracking and oversight.

However, ad hoc methods (manual and

software tools) are used to record

project data. Project managers also do

not perform periodic comparisons

between projected and actual results.

Project tracking

and oversight

Software

development

Apply best

practices to

achieve key

objectives: (1)

within budget, (2)

on schedule, and

(3) according to

requirements

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Evaluation

(continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 56 GAO/AIMD-00-170 SBA IT Management

43

SBA’s draft information system

enterprise configuration management

plan provides guidance for configuration

identification, change control,

configuration status accounting,

configuration management audits, data

management and library functions,

interface management, and contractor

control. However, configuration

management is not yet performed on

software projects. SBA recently hired

staff with configuration management

expertise.

Configuration

management

Software

development

Apply best

practices to

achieve key

objectives:

(1) within budget,

(2) on schedule,

and (3) according

to requirements

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Evaluation

(continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 57 GAO/AIMD-00-170 SBA IT Management

44

SBA’s 1984 guidance for solicitation

does not conform to generally accepted

standards. SBA maintains a database

of contracting opportunities for small

companies. Contracts do not

consistently include acceptance criteria.

SBA’s 1984 guidance for acquisition

planning does not conform to generally

accepted standards. Acquisition

planning is not consistent among

software projects.

Solicitation

Acquisition

planning

Software

acquisition

Apply best

practices to

achieve key

objectives:

(1) within budget,

(2) on schedule,

and (3) according

to requirements

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Evaluation

(continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 58 GAO/AIMD-00-170 SBA IT Management

45

SBA’s 1984 guidance for product

evaluation does not conform to

generally accepted standards. SBA

does not consistently establish criteria

prior to evaluating software products

against requirements.

Product

evaluation

SBA’s 1984 guidance for contract

tracking and oversight does not conform

to generally accepted standards.

Contract tracking

and oversight

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Evaluation

(continued)

Software

acquisition

Apply best

practices to

achieve key

objectives:

(1) within budget,

(2) on schedule,

and (3) according

to requirements.

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 59 GAO/AIMD-00-170 SBA IT Management

46

There are no policies for transition to

support, and no process has yet been

defined. Transitioned systems did not

always include a completed inventory of

software programs and related

products. Practices are inconsistent for

Web-based software.

Transition to

support

IT Policies, Procedures, and Practices

Software Development and Acquisition -- Evaluation

(continued)

SOFTWARE

ACQUISITION

Apply best

practices to

achieve key

objectives:

(1) within budget,

(2) on schedule,

and (3) according

to requirements.

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 60 GAO/AIMD-00-170 SBA IT Management

47

IT Policies, Procedures, and Practices

Impact of Software Development and Acquisition

Weaknesses

• Without adequate processes for software development, SBA lacks assurance

that project plans, including documentation, configuration management, and

quality assurance, will be developed and followed; and that software will meet

user needs.

• Without adequate processes for software acquisition, SBA lacks assurance

that acquisition plans will be developed, contractual requirements specified,

and that acquired products will meet user needs.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 61 GAO/AIMD-00-170 SBA IT Management

48

Software Development and Acquisition

Suggested Actions

• SBA should complete the systems development methodology and develop a

plan to institutionalize and enforce its use agencywide.

• SBA should establish policies, procedures, and processes for software

development and software acquisition, and develop a mechanism to enforce

them. These policies, procedures and processes need to address areas such

as

• requirements management,

• project planning,

• project tracking and oversight,

• software quality assurance,

• configuration management,

• acquisition planning,

• solicitation,

• contract tracking and oversight,

• product evaluation, and

• transition to support.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 62 GAO/AIMD-00-170 SBA IT Management

49

IT Policies, Procedures, and Practices

Plans to Address Software Development and Acquisition

Weaknesses

• OCIO is tailoring a systems development methodology (developed by another

agency) for its use.

• OCIO officials stated that requirements management practices will be initially

implemented on a project basis before establishing these practices

agencywide.

• OCIO officials stated that SBA plans to establish a group for product

evaluation for the LMS activities.

• SBA is working on a configuration management plan. This plan discusses the

need for uniform policies and guidance for the configuration management

discipline on large software projects such as the LMS.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 63 GAO/AIMD-00-170 SBA IT Management

50

IT Policies, Procedures, and Practices

Information Security -- Overview

Information security protects an organization’s computer-supported resources

and assets. Such protection ensures the integrity, appropriate confidentiality,

and availability of the data and systems of an organization. Integrity ensures that

data have not been altered or destroyed in an unauthorized manner.

Confidentiality ensures that information is not made available or disclosed to

unauthorized individuals, entities, or processes. Availability ensures that data

will be accessible or usable upon demand by an authorized entity.

Key activities for managing information security risks include:

• Risk assessment -- identifying security threats and vulnerabilities to information assets

and operational capabilities, ranking risk exposures, and identifying cost-effective

controls.

• Awareness -- promoting awareness concerning security risks and educating users

about security policies and procedures.

• Controls -- implementing controls necessary to deal with identified risks.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 64 GAO/AIMD-00-170 SBA IT Management

51

We evaluated SBA’s policies and procedures on information security using the

Clinger-Cohen Act, Computer Security Act, and guidelines issued by OMB, GAO,

and NIST. We reviewed SBA’s IT practices on two selected systems:

Automated Loan Control System and Wide Area Network. We also reviewed

Office of Inspector General reports on information security.

IT Policies, Procedures, and Practices

Information Security -- Overview (continued)

• Evaluation -- monitoring effectiveness of controls and awareness activities through

periodic evaluations.

• Central management -- coordinating security activities through a centralized group.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 65 GAO/AIMD-00-170 SBA IT Management

52

There are no policies and procedures to

deal with risk assessments. Also, no

process is yet defined for conducting

risk assessments and detecting

systems security deficiencies. Security

risks have not been periodically

assessed. SBA recently conducted an

analysis to determine annual workload

requirements for security reviews such

as risk analyses, vulnerability

assessments, and certifications of key

applications, its wide area network,

mainframe facility, and local area

networks.

Risk

assessments

Risk

assessment

Assess security

threats and

vulnerabilities

and determine

security needs

IT Policies, Procedures, and Practices

Information Security -- Evaluation

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 66 GAO/AIMD-00-170 SBA IT Management

53

SBA’s draft security policies indicate

that an awareness program must be

implemented to ensure that all

individuals involved with the operations,

maintenance, or use of computer

systems and sensitive software

applications are aware of information

security policies and procedures. A

notice for display on computer screens

has recently been implemented to

remind employees and contractors of

their basic information security

responsibilities. However, security

awareness training and education have

not been provided.

Awareness

Risk

management

Educate users

and others on

risks and related

policies

IT Policies, Procedures, and Practices

Information Security -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 67 GAO/AIMD-00-170 SBA IT Management

54

SBA has draft policies for security

controls and security procedures for

user password assignment, password

deactivation, and user access query.

SBA recently drafted a guide for testing

security features of a system. SBA also

recently developed a template for

security plans to document controls of

each system. SBA has implemented

controls including antivirus software,

firewalls, user identification and

authentication, user-authority screens,

and audit trails. However, some

controls are not fully effective. Further,

a comprehensive disaster recovery and

business continuity plan initiated in

1998 has not yet been completed.

Controls

Risk

management

Establish policies

and implement

controls to

reduce and/or

mitigate risks

IT Policies, Procedures, and Practices

Information Security -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 68 GAO/AIMD-00-170 SBA IT Management

55

SBA’s draft security policies and

procedures define responsibilities for

central management. However,

security responsibilities are currently

fragmented among all of SBA’s field and

program offices without centralized

oversight.

Central

management

SBA recently developed a process for

performing system security certification

and accreditation. In addition, SBA has

performed informal network testing,

recorded loan transactions, and

monitored internet activities. However,

analysis and documentation of these

activities is lacking.

Evaluation

Risk

management

Test and evaluate

effectiveness of

policies and

controls

IT Policies, Procedures, and Practices

Information Security -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 69 GAO/AIMD-00-170 SBA IT Management

56

IT Policies, Procedures, and Practices

Impact of Information Security Weaknesses

• Without conducting periodic risk assessments, SBA will not adequately

identify vulnerabilities to implement needed controls.

• Without a complete awareness program, SBA lacks assurance that staff will

adhere to established policies.

• Without policies and procedures, SBA lacks assurance that security controls

are being consistently applied to address identified vulnerabilities.

• Without periodic security evaluations, SBA lacks assurance that established

policies, procedures, and controls are effective against identified

vulnerabilities.

• Without institutionalized central oversight and coordination, SBA lacks

assurance that identified weaknesses are being addressed on an ongoing

basis.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 70 GAO/AIMD-00-170 SBA IT Management

57

Information Security

Suggested Actions

• SBA should conduct periodic security risk assessments to identify and rank

threats and vulnerabilities.

• SBA should implement a complete, effective security awareness program.

• SBA should periodically update policies and procedures on information

security and implement security controls to address identified vulnerabilities.

This should include completing the development and testing of its

comprehensive disaster recovery and business continuity plan. The plan

should then be updated and tested periodically.

• SBA should conduct periodic security evaluations to determine whether

policies, procedures, and controls are effective against identified

vulnerabilities, and take remedial action, as needed.

• SBA should develop and implement a centralized mechanism to monitor and

enforce compliance on information security by employees, contractors, and

program offices.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 71 GAO/AIMD-00-170 SBA IT Management

58

IT Policies, Procedures, and Practices

Plans to Address Information Security Weaknesses

• OCIO officials stated that SBA plans to finish development and testing of a

comprehensive disaster recovery and business continuity plan.

• OCIO has hired additional staff to address security weaknesses identified by

SBA’s Inspector General. These staff are responsible for

• performing security certification reviews of new and existing IT systems,

• administering user identification and passwords,

• developing and maintaining security policies, procedures, guidance, and

training.

• SBA recently established a committee charged with developing solutions to

resolving security weaknesses identified by the agency’s Office of Inspector

General as part of the financial statement audit.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 72 GAO/AIMD-00-170 SBA IT Management

59

Human capital centers on viewing people as assets whose value to an

organization can be enhanced through investment. As the value of people

increases, so does the performance capacity of the organization, and therefore

its value to clients and other stakeholders.

To maintain and enhance the capabilities of IT staff, the organization should

conduct four basic activities:

IT Policies, Procedures, and Practices

IT Human Capital -- Overview

• assess the knowledge and skills needed to effectively perform IT operations to support

agency mission and goals;

• inventory knowledge and skills of current IT staff to identify gaps in needed capabilities;

• develop strategies and implement plans for hiring, training, and professional

development to fill the gap between requirements and current staffing; and

• evaluate progress made in improving IT human capital capability, and use the results of

these evaluations to continuously improve the organization’s human capital strategies.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 73 GAO/AIMD-00-170 SBA IT Management

60

We evaluated SBA’s policies and procedures on IT human capital using the

Clinger-Cohen Act and our guide Human Capital: A Self-Assessment Checklist

for Agency Leaders. We reviewed IT human capital practices on two selected

projects: Loan Monitoring System and SmartStream.

IT Policies, Procedures, and Practices

IT Human Capital -- Overview
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 74 GAO/AIMD-00-170 SBA IT Management

61

OCIO is planning to develop procedures

for conducting an assessment. This

office maintains a skills inventory of its

headquarters staff.

There are no policies and procedures to

identify requirements of IT knowledge

and skills. In addition, SBA has not

conducted an assessment to determine

short- and long-term requirements of IT

knowledge and skills.

Inventory

knowledge and

skills of current

IT staff

Assessment of

needed IT

knowledge and

Needs skills

assessment

Assess needed

and current IT

knowledge and

skills

IT Policies, Procedures, and Practices

IT Human Capital -- Evaluation

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 75 GAO/AIMD-00-170 SBA IT Management

62

There are no policies and procedures

for evaluating progress in addressing

gaps in knowledge and skills. SBA has

not evaluated progress in improving IT

human capital capabilities or used

evaluation data to continuously improve

human capital strategies.

Progress

evaluation

There are no policies and procedures

for addressing gaps in IT knowledge

and skills. And, the gaps have not been

identified to support the development of

workforce strategies and plans for

recruiting, hiring, compensating, and

training.

Workforce

strategies and

plans WORKFORCE

ENHANCEMENT

Develop

workforce

strategies and

plans and

evaluate progress

IT Policies, Procedures, and Practices

IT Human Capital -- Evaluation (continued)

Issue Activity Assessment Comments

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 76 GAO/AIMD-00-170 SBA IT Management

63

IT Policies, Procedures, and Practices

Impact of IT Human Capital Weaknesses

Without periodic IT staff needs assessment and workforce enhancement

activities, SBA lacks assurance that it will effectively

• identify needed IT knowledge and skills,

• understand its current skill level,

• develop workforce strategies and implement plans to maintain the necessary

IT knowledge and skills to support the agency mission, and

• evaluate and report on progress in addressing knowledge and skill gaps.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 77 GAO/AIMD-00-170 SBA IT Management

64

IT Human Capital

Suggested Actions

• SBA should identify its IT knowledge and skills requirements.

• SBA should perform periodic IT staff assessments to identify current levels of

IT knowledge and skills.

• Based on the results of these assessments, SBA should develop workforce

strategies and implement plans to acquire and maintain the necessary IT

knowledge and skills to support the agency mission.

• SBA should periodically evaluate its progress in improving its IT human

capital capability and use the results to continuously improve its human

capital strategies.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 78 GAO/AIMD-00-170 SBA IT Management

65

IT Policies, Procedures, and Practices

Plans to Address IT Human Capital Weaknesses

In March 2000, OCIO officials stated that SBA plans to

• assess the skills and knowledge levels of IT staff;

• conduct a survey of IT staff to identify tools and training needed to effectively

perform their assigned duties; and

• take a leadership role in developing policies and procedures for recruiting,

hiring, and compensating IT staff.
Appendix I

Briefing on Small Business Administration’s

Management of Information Technology

Page 79 GAO/AIMD-00-170 SBA IT Management

66

Agency Comments

• In commenting on a draft of this briefing, SBA said that it agrees with the

recommendations and actions are underway to address them.

• SBA questioned the accuracy of statements made concerning 12 IT management

activities. In cases where SBA provided supporting documentation, we made

appropriate revisions.

• SBA also questioned the fairness of the assessment of its operations against industry

standards because many of these standards have emerged in the last few years. SBA

contended that other small federal agencies would not fair much better in meeting

industry standards.

• It should be noted that the industry standards have sufficient flexibility that key IT

processes can be developed that are appropriate for the size and complexity of the

IT environment of each organization. Although SBA’s operations may or may not

compare favorably with other small federal agencies, comparison with industry

standards or best practices is a sound approach for identifying activities that can be

improved to enhance the capability of supporting the agency’s mission and

obtaining a positive return on the IT investment.
Page 80 GAO/AIMD-00-170 SBA IT Management

Appendix II

Comments From the Small Business

Administration ApendxiI

Note: GAO comments

supplementing those in the

report text appear at the end

of this appendix.
Appendix II

Comments From the Small Business

Administration

Page 81 GAO/AIMD-00-170 SBA IT Management
Appendix II

Comments From the Small Business

Administration

Page 82 GAO/AIMD-00-170 SBA IT Management

See comment 1.

See comments 2 and 4.
Appendix II

Comments From the Small Business

Administration

Page 83 GAO/AIMD-00-170 SBA IT Management

See comment 4.

See comment 3.

See comment 5.
Appendix II

Comments From the Small Business

Administration

Page 84 GAO/AIMD-00-170 SBA IT Management

See comments 4 and 6.

See comment 9.

See comment 7.

See comment 8.
Appendix II

Comments From the Small Business

Administration

Page 85 GAO/AIMD-00-170 SBA IT Management

See comment 10.

See comment 11.

See comment 12.
Appendix II

Comments From the Small Business

Administration

Page 86 GAO/AIMD-00-170 SBA IT Management
Appendix II

Comments From the Small Business

Administration

Page 87 GAO/AIMD-00-170 SBA IT Management
Appendix II

Comments From the Small Business

Administration

Page 88 GAO/AIMD-00-170 SBA IT Management
Appendix II

Comments From the Small Business

Administration

Page 89 GAO/AIMD-00-170 SBA IT Management
Appendix II

Comments From the Small Business

Administration

Page 90 GAO/AIMD-00-170 SBA IT Management
Appendix II

Comments From the Small Business

Administration

Page 91 GAO/AIMD-00-170 SBA IT Management

IT Policies, Procedures, and Practices

Evaluation Summary

Investment management

Requirements management

Project planning

Project tracking & oversight

Software development & acquisition

Quality assurance

Configuration management

Acquisition planning

Solicitation

Contract tracking & oversight

Product evaluation

Transition to support

Selection process

Selection data

Selection decisions

Control process

Control data

Control decisions

Evaluation process

Evaluation data

Evaluation decisions

Business processes

Information flows & relationships

Architecture

Applications

Data descriptions & relationships

Risk assessments

Central management

Security

Controls

Awareness

Evaluation

Requirements

Workforce strategies & plans

Progress evaluation

Human capital

Inventory

Architecture

Technical reference model

Standards profiles

Change management

Legacy systems integration

Technical infrastructure

Policies and procedures for key

functions; selected key practices for

planning, monitoring, and evaluation

Incomplete or obsolete

policies and procedures;

ad-hoc practices

Comprehensive, current policies and procedures;

practices for planning, monitoring, and evaluation adhere

to policies, procedures, and generally accepted standards
Appendix II

Comments From the Small Business

Administration

Page 92 GAO/AIMD-00-170 SBA IT Management

The following are GAO’s additional responses to SBA’s letter dated April 4,

2000.

GAO Comments 1. Business processes—because SBA has now established a completion

date, the statement “SBA has not yet provided a completion date for the

architecture” has been removed from the briefing slide.

2. Information flows and relationships—the Chief Operating Officer

states that SBA developed an information architecture in 1995 that lists

the entities and individual data elements used and collected by each of

the SBA business activities. However, the 1995 architecture is obsolete

and is being replaced by a new draft IT architecture. The draft IT

architecture still does not include the flows and relationships of

information needed by different business entities and does not identify

who is responsible for maintaining and updating the information. This

information is needed for other components of the architecture to

develop proper information and communications services.

3. Applications—because SBA has now provided a consolidated list of

applications in its latest draft version of the IT architecture, the

statement “the draft architecture does not provide a consolidated

inventory of applications” has been removed from the briefing slide.

4. Data descriptions and relationships—GAO assessed SBA’s current

effort to develop its IT architecture and did not compare its effort with

other agencies.

5. Technology architecture—GAO did not compare SBA’s technology

architecture with other federal agency architectures.

6. Technical reference model—the Zachman framework for enterprise

architecture calls for populating various “cells” of the framework with

models and defining the generic contents of each of the cells of the

framework. We noted “SBA has not clearly defined how the framework

will be applied for the development of its architecture” because SBA

does not identify cells of the framework to be populated with models

and if a cell of the framework is not populated with a model, SBA does

not explain why that part of SBA IT architecture is not relevant. Also,

we noted “SBA IT architecture does not identify the products

specifying the contents of the framework” because the contents of

SBA’s architecture components for applications and technical
Appendix II

Comments From the Small Business

Administration

Page 93 GAO/AIMD-00-170 SBA IT Management

infrastructure do not adequately address plans and controls for

defining the roles, responsibilities, and skills required within the

architecture process.

7. Legacy systems integration—because SBA has now provided a list of

legacy systems for migration to the target architecture, the statement

“SBA is also working to develop a list of legacy systems for migration to

the target architecture” was removed from the briefing slide.

8. Requirements management—the Chief Operating Officer states that

SBA uses IEEE guidance for requirements documentation and has

developed procedures for requirements management. We acknowledge

that SBA recently said that it will adhere to the format recommended

by the IEEE standard for specifying system requirements. However,

SBA’s use of this particular industry standard on the LMS project,

though commendable, is an exception to the general practices

employed by SBA on its other system development projects. SBA lacks

organizational policy and procedures for implementing generally

recognized best practices in this area, including allocating

requirements, implementing requirements traceability, assessing the

impact of proposed changes to requirements, and measuring

requirements variability for use as a management indicator of project

risk.

9. Project planning—the Chief Operating Officer states that SBA is using

the LMS project to develop and establish guidance for project planning.

We commend the intention of SBA to define guidance for this area and

formalize its adoption throughout the agency. Our review focused,

however, on reporting what is currently in place and how the current

state of affairs compares with generally accepted industry practices. In

this regard, SBA’s stated intention is not yet matched by a plan to attain

specific improvements in this area. For example, there is no identifiable

task or scheduled date for defining, issuing, and implementing

agencywide policies on standards and accountability for project

planning, the use of the systems development methodology, the

application of documented procedures, and the performance of

standard organizational practices defined for this area.

10. Project tracking and oversight—the Chief Operating Officer states that

SBA managers formally tracked progress on projects that we did not

review and informally tracked progress on projects that we did review.

Tracking, as applied by best practices in this area, is used to measure,
Appendix II

Comments From the Small Business

Administration

Page 94 GAO/AIMD-00-170 SBA IT Management

identify, and report on the health of a project’s schedule and cost, as

these relate to work products, critical events, and other project

commitments. However, we found that at SBA, project management

reports were not always available and, when available, lacked

comparative data for analysis. In addition, recording and reporting of

project information either did not occur, or were inconsistently

performed.

11. Configuration management—the Chief Operating Officer states that

SBA is performing configuration management for the LMS system and

that configuration management guidance is included in the LMS

statement of work. Our review of the LMS project revealed that

configuration management practices were not performed—we did not

find any items placed under configuration management.

12. Needs assessment—the Chief Operating Officer states that SBA

conducted a training needs survey in late fiscal year 1998. Our review of

IT human capital activities revealed that this survey did not focus on

the training needs of SBA’s IT staff, nor was it reflective of an analysis

of the short- and long-term knowledge and skills requirements of SBA’s

IT staff. Several times during our review, the CIO stated that SBA had

not yet done an assessment of its IT staff’s knowledge and skills

requirements, nor had it developed strategies for addressing gaps in its

current knowledge and skills level.

(511851) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)


United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00