United States General Accounting Office

GAO Report to the Chairman, Subcommittee

on Oversight, Committee on Ways and

Means, House of Representatives

January 1997 TAX SYSTEMS

MODERNIZATION

Imaging System’s

Performance

Improving but Still

Falls Short of

Expectations

GAO/GGD-97-29

GAO United States

General Accounting Office

Washington, D.C. 20548

General Government Division

B-270263 

January 16, 1997

The Honorable Nancy L. Johnson

Chairman, Subcommittee on Oversight

Committee on Ways and Means

House of Representatives

Dear Chairman Johnson:

One of the major objectives of the Internal Revenue Service’s (IRS)

modernization effort is to move away from a labor-intensive tax return

processing system that relies on thousands of employees transcribing data

from paper tax returns and move to an electronic system that reduces

processing costs and eliminates transcription errors. One strategy for

achieving that objective is to reduce the number of paper returns by

increasing the number of returns filed electronically. For returns that will

continue to be filed on paper, IRS plans to achieve its objective through

document imaging and optical character recognition (OCR). The Service

Center Recognition/Image Processing System (SCRIPS) is the first of what

was to have been two document imaging and OCR systems.1

SCRIPS became operational in the latter part of 1994, and we reported that it

experienced numerous performance problems during the 1995 filing

season.2

 This report responds to your request that we follow up on those

problems and assess the performance of SCRIPS in 1996. Specifically, this

report (1) identifies the primary causes for performance problems that

occurred in 1995; (2) assesses whether SCRIPS performance improved in

1996 as of September 30, 1996; and (3) provides a status report on IRS’

future plans for SCRIPS.

Background In July 1995, we reported on IRS’ progress in implementing some of the

business and technological components of its modernization effort, known

as Tax Systems Modernization (TSM).3

 Although we said that IRS had made

some progress, we also said that pervasive management and technical

weaknesses existed that placed the modernization effort at risk. Among

1

The second system, the Document Processing System (DPS), was intended to replace SCRIPS and

expand the imaging capability to more complex tax forms. However, on October 8, 1996, IRS

announced that it was terminating that project.

2

The 1995 Tax Filing Season: IRS Performance Indicators Provide Incomplete Information About Some

Problems (GAO/GGD-96-48, Dec. 29, 1995).

3

Tax Systems Modernization: Management and Technical Weaknesses Must Be Corrected If

Modernization Is To Succeed (GAO/AIMD-95-156, July 26, 1995).

Page 1 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

other things, (1) IRS did not have a business strategy to maximize

electronic filing, the result of which could slow the planned decrease in

the workload of paper processing systems; and (2) IRS lacked the full range

of managerial and technical foundations to realize its modernization

objectives. Some of these key foundation components were a complete

cost/benefit analysis of the overall modernization effort and thorough

testing of individual systems before they were implemented. In September

1996, we reported on IRS’ progress in addressing the managerial and

technical weaknesses we identified in July 1995.4

 We concluded that

although IRS is working to resolve these weaknesses, it had not fully

satisfied any of our recommendations.

SCRIPS was one of the systems that was designed under the conditions cited

in our July 1995 report. It was intended to replace the aging OCR equipment

that IRS had been using to process all of the paper Federal Tax Deposit

(FTD) coupons; almost all of the paper information returns (e.g, Forms

1099); some of the individual income tax returns filed on Form 1040EZ;

and some employment tax returns (Form 941). In addition, SCRIPS was

expected to process Form 1040PC, a paper form that taxpayers can

generate when they use computer software to prepare a tax return.

Figure 1 shows the various SCRIPS components. Under the character

recognition and image capture component of SCRIPS, scanners (1) read

information from the document and convert the information to

machine-readable format for later computer processing and (2) create an

image of the document. In the event of recognition errors during

document scanning, IRS staff can access an image of the tax return in lieu

of having to locate the original paper tax return to make corrections. IRS

expected that having an image of the tax return would improve the

productivity of staff doing data validation. In addition, the images of FTD

coupons and certain information return documents are stored on optical

disk for later use. The older OCR systems used microfilming as the storage

medium—essentially a manual process that required more physical

storage space than optical disks. IRS continues to retain a paper copy of the

Form 1040EZ rather than store the image because IRS has certain legal

concerns.5

4

Internal Revenue Service: Business Operations Need Continued Improvement

(GAO/AIMD/GGD-96-152, Sept. 9, 1996).

5

One concern is the validity of an imaged signature. Another concern is that SCRIPS images only one

side of the two-sided Form 1040EZ. The second page includes a worksheet that taxpayers use for tax

calculations, and IRS considers it part of the tax return.

Page 2 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Page 3 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Figure 1: SCRIPS Components

Character Recognition


& Image Capture

Image Processing

The image controller helps 

ensure that each scanned 

document image moves from 

the scanner to the server. It 

also provides data backup 

until images reach the server.

The multiprocess server 

maintains control over the tax 

data. All changes and updates 

are processed through the 

server.

Data Management

3 Scanners with Monitor, Keyboards and Printer 3 Image Controllers

1 Multiprocessor


 Server

Updates

Updates

Updates

Final


Output

A scanner operator feeds a block 

of documents into the scanner. 

The scanner numbers each 

document, takes an image, and 

makes the first OCR attempt.

a

 


 

Page 4 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Cont.

In those instances where the secondary recognition and name and 

address block reader cannot identify the correct characters, the file 

is sent to a workstation operator. That operator reviews the image 

(with the capability to magnify it) to determine what corrections are 

needed. Updated files are sent back to the file server.

Updates

Updates

Updates

Final Output

Secondary Character


Recognition and Identification

5 Recognition Engines


 for Tax Data

Outputs

66 Workstations

2 Name and Address


 Block Readers

9-Track Tape Drive

Forwarded to the next 


stage with tax data for 


further processing.

Optical Disks 


Store FTDs 


and information


return images.

8 mm Tape Drive

Stores tax data.

Secondary character recognition is used for those fields (1) where 

characters are not completely read by the scanner or (2) IRS 

requires secondary recognition. Tax data is read by the recognition 

engine, and the block reader reads name and address information.

Source: GAO graphic based on IRS data.

Page 5 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

IRS expected that SCRIPS would provide faster, more accurate document

processing. Specifically, IRS expected that SCRIPS would result in a

20 percent productivity increase over manual data entry and a 10 percent

productivity increase over older OCR equipment. Other expected benefits

included lower costs for system maintenance and storage of tax return

data.

Originally, IRS planned to implement SCRIPS in all 10 service centers where

it currently processes paper returns. However, after the contract was

awarded in February 1993, IRS decided to consolidate paper tax return

processing in five centers. Accordingly, SCRIPS was tested in Cincinnati in

the summer of 1994, and the other four SCRIPS centers began using SCRIPS

between September and November 1994.

Results in Brief SCRIPS experienced significant performance problems in 1995. Two

problems were system downtime and slow processing rates. For example,

although SCRIPS was expected to increase productivity by 20 percent over

the manual data entry system, SCRIPS actually processed Forms 1040EZ in

1995 at a rate that was about 7 percent slower than manual data entry.

Because of performance problems, two of the five centers stopped

processing Forms 1040EZ on SCRIPS and reverted to using manual data

entry.

SCRIPS performance deficiencies in 1995 stemmed primarily from both

hardware and software problems. Hardware problems occurred with the

scanner that captures the image of the document. Software problems

occurred with the operating system for the image controller that

reconciles the number of documents scanned to the number of documents

in the SCRIPS database. The hardware and software problems were not

detected before the rollout of SCRIPS because testing in Cincinnati was

incomplete. For example, SCRIPS was never tested at the peak volumes that

IRS experiences during a filing season.

Although IRS had expected that SCRIPS would be processing five document

types—forms 1040EZ, 941, and 1040PC; FTD coupons; and information

returns—IRS postponed plans to process forms 941 and 1040PC on SCRIPS.

Of the three remaining document types, the Cincinnati test certified the

software application only for FTD coupons. Therefore, the software

applications for information returns and Form 1040EZ were not

thoroughly tested before they were put into production.

Page 6 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

To improve the performance of SCRIPS for 1996, IRS made hardware and

software modifications, some of which were made before the start of the

1996 filing season. As a result of some of these enhancements and more

staff familiarity with SCRIPS, according to IRS officials in all five SCRIPS

service centers, SCRIPS performed significantly better during the 1996 filing

season than it did in 1995. For example, between April and June,

unscheduled system downtime decreased from about 791 hours in 1995 to

about 43 hours in 1996. Also, performance on another indicator—the

number of documents processed per hour—improved for Forms 1040EZ

and FTD coupons.6

 Specifically, the number of Forms 1040EZ processed

per hour in 1996 increased by four documents per hour (about 6 percent)

and the number of FTD coupons increased by 3 documents per hour (less

than 1 percent). However, the number of information returns processed

per hour decreased by five documents per hour (about 3 percent).

In addition to a slower processing rate for information returns, SCRIPS (1) is

not processing all the forms that it was expected to process in 1996, (2) is

expected to cost more than originally estimated, and (3) is expected to

provide lower labor cost savings than IRS originally anticipated. IRS’

October 1994 business case for SCRIPS stated that in 1996 SCRIPS would be

processing all FTD coupons and information returns, all Forms 1040EZ,

50 percent of the Forms 1040PC, and 93 percent of the Forms 941.

However, during the first 9 months of 1996, SCRIPS processed only FTD

coupons, 60 percent of the information returns, and about 50 percent of

the Forms 1040EZ. This reduced level of performance stems in large part

from (1) IRS’ decision, after the contract was awarded, to consolidate all

paper processing in 5 service centers instead of 10; and (2) IRS’ decision to

purchase only 5 systems when the contractor was planning to provide 10

systems—1 in each of the 10 service centers. Furthermore, IRS officials did

not know the extent to which the hardware and software modifications

that had already been made for 1996 and those that were planned for later

in the year would increase SCRIPS’ capability for processing more 1040EZs

and information returns. Therefore, IRS officials did not significantly

increase the expectations for SCRIPS in 1996 over those that they had in

1995.

The latest cost estimate for SCRIPS is $288 million—considerably more than

previous cost estimates, which, according to IRS’ post-implementation

6

The number of documents processed per hour, commonly referred to as the composite rate, includes

not only the time to scan the documents but also the time spent by work station operators validating

and correcting tax return data. Validation and correction are necessary in those instances when the

scanner and secondary recognition systems cannot read certain tax return data or the system detects

an error made by a taxpayer.

Page 7 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

review report on SCRIPS, ranged from $133 million to $209 million. At least

$20 million of that increase is attributed to maintenance costs that were

not included in either the $133 million or $209 million estimates.7

 Also, IRS

estimated in October 1994 that SCRIPS would provide about $17 million in

labor savings from fiscal years 1994 through 2000. In September 1995 IRS

lowered that estimate to about $5 million. Furthermore, a September 1996

IRS investment evaluation of SCRIPS estimated that SCRIPS will yield a

negative return on investment (i.e., SCRIPS’ costs will exceed its benefits)

from 1991 to 2001. However, the evaluation noted that the estimate is

based on performance in 1995 and does not reflect the software and

hardware modifications that were made in 1996.

In July 1995, IRS decided to terminate all software programming for the

1040PC because of SCRIPS’ instability, a lack of system capacity, and a

number of software application problems. Also, IRS has decided not to use

SCRIPS to process Forms 941 in fiscal year 1997. Decisions on using SCRIPS

to process Forms 941 beyond fiscal year 1997 have not been finalized. In

addition to funding constraints, according to IRS officials, a decision on

using SCRIPS for Forms 941 will depend on (1) the results of a

September 1996 capacity test of SCRIPS, which were not available when we

completed our audit work; and (2) decisions by IRS’ Investment Review

Board8

 on recommendations made by a paper processing task team.

Objectives, Scope,

and Methodology

Our objectives were to (1) determine the primary causes for SCRIPS

performance problems in 1995; (2) assess whether those problems were

corrected as of September 30, 1996; and (3) provide a status report on IRS’

future plans for SCRIPS.

To accomplish these objectives, we did the following:

• We interviewed National Office officials in the SCRIPS project office and the

taxpayer service function, which has responsibility for those tax forms

processed on SCRIPS, to obtain their views on the extent to which SCRIPS’

performance improved during fiscal year 1996.

7

The supporting documentation for IRS’ $288 million cost estimate showed a separate breakout for

maintenance costs for only fiscal years 1995 and 1996. We were unable to determine what, if any,

portion of the costs for fiscal years 1997 to 2000 was attributed to maintenance. According to IRS

officials, maintenance costs may have been combined with other costs for those fiscal years.

8

In October 1995, IRS created an executive-level Investment Review Board, currently co-chaired by the

Deputy Commissioner and the Chief Financial Officer, for selecting, controlling, and evaluating all of

IRS’ information technology investments.

Page 8 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

• We interviewed IRS contracting office officials to determine what, if any,

performance requirements the contractor was being held to in 1995 and

1996.

• We interviewed officials at all five SCRIPS service centers to determine

whether performance had improved in 1996 and to identify the

performance indicators officials were using to evaluate SCRIPS

performance.

• We observed SCRIPS in operation at the service centers in Memphis and

Cincinnati and at the program development site in Washington, D.C.

• We interviewed contractor officials about the workload requirements that

IRS had specified for SCRIPS.

• We reviewed IRS business cases9 for SCRIPS that documented the objectives

of the system, its expected benefits, and its estimated cost. IRS prepared

business cases for SCRIPS in February 1992, April 1992, December 1993,

October 1994, and December 1995. To assess SCRIPS, we used the

performance expectations that were included in the October 1994 business

case because it was the one that was in effect at the beginning of the 1995

filing season.

• We reviewed IRS evaluations and reports on the implementation of SCRIPS,

including an August 1995 performance evaluation report; a December 1995

post-implementation review report that assessed whether business goals

were met; a May 1995 Internal Audit report on SCRIPS testing in Cincinnati;

a February 1996 Internal Audit report on the rollout of SCRIPS; and a

September 1996 report on the results of an investment evaluation of SCRIPS.

• We compared available SCRIPS performance data for January through

September 1995 and 1996.

• We computed composite rates—the number of documents processed per

hour—for Form 1040EZ, information returns, and FTD coupons. To

compute those rates, we used IRS data on the number of hours spent for

various aspects of tax return processing, including scanning, data

correction, and data validation for January through September 1995 and

1996. Our composite rate calculations, similar to those done by IRS shortly

after SCRIPS was implemented, did not include time for document

preparation or another function that is referred to as code and edit.10

9

IRS prepares business cases to justify an information technology project and to demonstrate that it is

cost beneficial.

10For SCRIPS, the IRS data we received for computing composite rates showed the combined hours for

both the code and edit function and the document preparation function. During code and edit, staff

prepare returns for computer entry by, among other things, ensuring that all data are present and

legible. For the manual entry system, the code and edit function was not combined with the document

preparation function, and the hours associated with document preparation were not shown. Therefore,

we did not include these aspects of processing in our composite rate comparison.

Page 9 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

We did our audit work between October 1995 and September 1996 and in

accordance with generally accepted government auditing standards. Other

than identifying obvious reporting errors, we did not verify the accuracy of

IRS’ data on the number of hours spent for various aspects of tax return

processing. We did attempt to detemine what elements were included in

IRS’ cost estimates for SCRIPS, but we did not attempt to verify the accuracy

of the costs for those elements. We requested comments on a draft of this

report from the Commissioner of Internal Revenue or her designated

representative. Responsible IRS officials, including the National Director

for Submission Processing, the Assistant Commissioner for Forms and

Submission Processing, and the SCRIPS project manager, provided IRS’

comments in a November 13, 1996, meeting. Their comments on our

recommendation were reiterated in a November 19, 1996, memorandum

from the Acting Chief of Taxpayer Service. IRS’ comments are summarized

and evaluated on pages 22 and 23.

SCRIPS Performed

Substantially Below

Expectations in 1995

SCRIPS performed well below expectations in 1995. Extensive, unscheduled

system downtime and slower than expected processing rates affected

SCRIPS’ ability to meet the expectations that IRS had established before the

start of the 1995 filing season. Hardware problems with the scanner

contributed to significant amounts of system downtime; various software

problems contributed to slow processing rates.

SCRIPS Processed Less

Volume and at Slower

Rates Than Expected

In 1995, SCRIPS processed 19 percent more FTD coupons than IRS expected.

However, as shown in table 1, SCRIPS did not meet IRS’ volume expectations

for the three other document types scheduled for processing in

1995—information returns, Form 1040EZ, and Form 941 (IRS did not

expect to start processing Form 1040PC on SCRIPS until 1996).

Page 10 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Table 1: Number of Documents

SCRIPS Expected to Process and

Number Actually Processed in 1995 Document type

Number IRS expected to

process in 1995a

Number actually

processed in 1995

FTD coupons 76,422,540 90,630,944

Information returns

57,369,000 24,897,860

Form 1040EZ 8,594,500 4,873,419

Form 941 4,826,140 0b

aIRS expectations as of October 1994.

bSometime between October and December 1994 IRS decided not to use SCRIPS to process

Forms 941.

Source: IRS data.

SCRIPS processed 19 percent more FTD coupons than IRS had expected

because more FTD coupons were filed than expected, and service center

officials placed the highest priority on processing those forms. That

priority stemmed from (1) IRS procedures that require that 90 percent of all

FTD coupons be processed within 24 hours of receipt; and (2) the absence

of a backup processing system, because IRS had cancelled its maintenance

contract for the older OCR equipment that had been processing FTD

coupons. IRS had backup systems for the other documents that SCRIPS was

expected to process in 1995. The manual data entry system could be used

for forms 1040EZ and 941, and IRS extended the maintenance contract for

the older OCR equipment that was processing information returns.

Not only did SCRIPS not process the number of forms expected, but the

speed with which it did its processing was slower than expected. As

shown in table 2, SCRIPS did not meet the processing rate expectations for

any of the three document types it processed in 1995. Moreover, the actual

processing rate for Forms 1040EZ in 1995 was about 7 percent less than

the rate achieved in 1995 by manual data entry.

Table 2: Expected and Actual

Processing Rates for SCRIPS in 1995

Compared to Rates for Systems

SCRIPS Was to Replace

Processing rate (documents per hour) in 1995

Document type

Expected for

SCRIPS

Actual for

SCRIPS

Actual for older

systems

1040EZ 186 63 68

Information returns 178 147 131

FTD coupons 1,170 758 Not applicablea

aIRS cancelled the maintenance contract for the OCR system that had been processing FTD

coupons. SCRIPS was the only system used to process FTD coupons in 1995.

Source: GAO computations using IRS data for January through December 1995.

Page 11 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Because of performance problems, two of the five service centers stopped

using SCRIPS to process Forms 1040EZ in 1995.11 Instead, they reverted to

manual data entry, which required using more staff resources than

planned, thus increasing processing costs. IRS had planned to use 25.6 staff

years to process other-than-full-paid12 Forms 1040EZ during the 1995 filing

season in the five SCRIPS centers but used 66.5 staff years instead.13

Hardware problems contributed to SCRIPS’ performance deficiencies in

1995. According to IRS’ August 1995 performance evaluation report, the

SCRIPS scanner experienced the most hardware failures, which contributed

to a “substantial amount of downtime” at the centers.14 According to IRS

data, between April and June 1995, the five service centers experienced

about 791 hours of unscheduled downtime. In addition, the scanner

jammed when paper was extremely thin, which was sometimes the case

with information returns.

Software problems also occurred. According to IRS officials and

evaluations of SCRIPS, the image controller did not operate fast enough to

keep up with the scanner because the operating system software was

inefficient. The image controller is to track each scanned document image

file to ensure that it moves from the scanner and ultimately to the server

where it is stored. This component helps reconcile the number of

documents scanned to the number of documents in the database. IRS

officials and evaluations of SCRIPS also indicated that SCRIPS did not provide

accurate reports on this reconciliation—referred to as “run-to-run”

balancing. The FTD coupon run-to-run balancing report, for example, did

not provide total counts to help IRS staff confirm that all of the FTD

coupons in a block that was scanned had in fact been processed. This

balancing is particularly important for FTD coupons because IRS needs to

accurately classify tax deposits to the appropriate Treasury account. At

one center we visited, officials told us that they had to revert to manual

counts for doing run-to-run balancing.

11According to IRS officials, service center directors had discretion in managing the processing

workload. Some chose to revert to the manual system; others did not.

12Other-than-full-paid tax returns are those returns filed by taxpayers who either were due a refund or

did not pay the full amount of tax owed at the time of filing. These returns were about 92 percent of

the Forms 1040EZ filed at the five SCRIPS service centers during the 1995 filing season.

13The number of staff years for Form 1040EZ includes those used on SCRIPS and manual data entry.

14IRS’ evaluation of SCRIPS performance during the 1995 filing season noted that wide variations in

scanner operator skills also affected the scanner’s performance. IRS officials also noted that even

though the scanner may have been “down,” other SCRIPS components could have been operating.

Page 12 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Performance Problems Not

Thoroughly Identified

Before Rollout Due to

Incomplete Testing

Many of the problems experienced with SCRIPS in 1995 might have been

anticipated if IRS had thoroughly tested SCRIPS before installing the system

in the other four service centers. The pilot test of SCRIPS was incomplete

because it (1) did not certify all software applications that were to be used

during 1995; and (2) did not test SCRIPS’ ability to handle peak processing

volumes, such as those experienced in the tax return filing season.

An organization tests a new or modified system to detect system design

and development errors and to correct them before putting a system into

operation. In our July 1995 report on TSM, we said that although IRS

recognized the importance of testing, it had not yet developed a complete

and comprehensive testing plan for TSM.

15 We said that individual TSM

systems were developing their own test plans, which IRS described as

rudimentary and inadequate. This testing environment was in effect when

IRS did a pilot test of SCRIPS at the Cincinnati Service Center in the summer

of 1994.

The purpose of a pilot test is to evaluate the performance of a system in

one location before deciding whether to implement the system at other

locations. IRS uses the pilot test to certify that the system is meeting its

program or business objectives. During the pilot test, IRS is to collect data

on the performance of the system and compare the data against

established performance goals to certify that the system is performing as

expected.

Due to delays in receiving and testing software, only the FTD coupon

application was certified as a result of the pilot test. The certifications for

Forms 1040EZ and information returns were not done due to management

concerns that conducting these certifications would delay the rollout of

SCRIPS to five centers, which was scheduled for January 1995. As a result of

incomplete testing, for example, IRS did not identify problems with the

information returns software until SCRIPS was fully operational. Due to

these software problems, a higher percentage of returns were sent to data

validation than originally anticipated, thus decreasing system productivity.

According to IRS officials, those problems were corrected before the 1996

filing season.

The pilot test was also incomplete because it did not (1) provide IRS a clear

indication of SCRIPS’ ability to perform under peak workload conditions

and (2) show the impact of running multiple software applications. If the

system had been tested at larger volumes, running multiple software

15GAO/AIMD-95-156.

Page 13 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

applications, problems may have been identified earlier and resources

could have been diverted to address performance problems before the

system’s rollout. According to an Internal Audit report on the pilot test, the

contract required that SCRIPS process 200,000 documents a day during peak

periods. During the 1-day volume test in the pilot, only 45,000 documents

were scanned into the database. The remaining components of the system

processed an additional 142,000 documents that had been scanned before

the test. Thus, the scanner, one of the primary sources of downtime during

1995, was not tested under levels IRS experiences in a production

environment. Also, only 10,000 documents were processed

completely—only 5 percent of the daily production workload specified in

the contract.

According to IRS officials, a full production test plan had been developed in

December 1993 that would simulate peak filing season volumes. However,

the plan was never implemented. One reason IRS officials provided for not

implementing the test plan was that IRS never had full access to a SCRIPS

system to do the test. The system that IRS was to use for the test was the

same system that the contractor was using for systems development work.

Another factor that IRS officials cited was a sense of urgency to roll out

SCRIPS for FTD coupons because IRS had no backup processing system.16

IRS’ December 1995 report on the post-implementation review of SCRIPS

cited several problems with the testing of SCRIPS in addition to those that

occurred in the pilot test. For example, although an equipment acceptance

test was done, the test was not adequate to validate the system’s readiness

for production. The report also stated that IRS waived an “operational

capabilities demonstration” that would have included (1) tests for the

readability of forms and (2) measures of the number of documents

scanned per hour. The report said that such a test, done as part of the

contract award process, may have shown that the system could not meet

the minimum performance requirements.

The post-implementation review team proposed that tests be done to

verify that the system could handle the production workload. It also stated

that if such a test could not be done, a simulated test should be done

focusing on the performance of key areas of the system, particularly those

that could lead to processing bottlenecks. It also proposed a 1-year pilot

test for forms that may be added to SCRIPS in the future and proposed that

the pilot test include the peak volumes of a filing season.

16As previously stated, IRS had cancelled the maintenance contract for the OCR system that IRS was

using to process FTD coupons. In addition, IRS had redistributed the workload of paper FTD coupons

in 1995—consolidating the work of 10 service centers into 4 service centers.

Page 14 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Performance

Improved for 1996,

but SCRIPS

Performed Below

Original Expectations

and Costs May Be

Higher Than Expected

Officials in the five service centers that used SCRIPS in 1996 said that it is

performing significantly better than it did in 1995. IRS officials told us that

the primary performance expectation for SCRIPS in 1996 was system

stabilization. One of the primary indicators these officials used in

evaluating stabilization was the amount of downtime. According to IRS

service center officials and available IRS data, downtime decreased

substantially in 1996. This reduction enabled SCRIPS to process more Forms

1040EZ and information returns in 1996 while continuing to process all FTD

coupons. In addition, the processing rates for SCRIPS (i.e., the number of

documents processed per hour) improved for two document types—Form

1040EZ and FTD coupons.

Despite these improvements, the system, as of September 30, 1996, (1) was

not processing all the forms that the October 1994 business case said it

would process, (2) was expected to cost more than original estimates, and

(3) was not expected to provide the estimated labor savings that were

cited in the October 1994 business case.

SCRIPS Performance

Improved in 1996

According to service center officials, SCRIPS performed “significantly

better” in 1996 than it did in 1995 because the system experienced

significantly less downtime than in 1995. IRS did not begin tracking

downtime in 1995 until April. Comparable data for April through June 1995

and 1996 show that unscheduled downtime did decrease

significantly—from about 791 hours to 43 hours.

As shown in table 3, mostly because of less unscheduled system

downtime, SCRIPS processed many more documents during the first 9

months of 1996 than it did during the first 9 months of 1995.

Table 3: Number of Documents

SCRIPS Processed in 1995 and 1996

as of September 30 Document type

Number processed as of

September 30, 1995

Number processed as of

September 30, 1996

Form 1040EZ 4,870,493 8,309,384

Information returns 24,631,809 22,618,452

FTD coupons 67,247,198 70,857,389

Total 96,749,500 101,785,225

Source: IRS data.

Also, according to IRS officials, new software was installed in

November 1995 to correct the run-to-run balancing problems that we

Page 15 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

discussed earlier. According to these officials, since that software was

installed, SCRIPS centers have not reported problems with tracking the

number of documents scanned and processed.

Also in 1996, as shown in table 4, processing rates increased for Forms

1040EZ and FTD coupons.

Table 4: Actual Processing Rates by

Document Type as of September 30,

1995, and 1996

Processing rate (documents per hour)

Form type

SCRIPS actual rate

as of Sept. 30, 1995

SCRIPS actual rate

as of Sept. 30, 1996

Actual for older

systems as of

Sept. 30, 1996

Form 1040EZ 63 67 60

Information returns 150 145 133

FTD coupons 719 722 Not applicablea

aIRS cancelled the maintenance contract for the OCR system that had been processing FTD

coupons. SCRIPS was the only system used to process paper FTD coupons in 1996.

Source: GAO computations based on IRS data.

SCRIPS Processed Many

Fewer Documents Than

Originally Expected in

1996 While Its Estimated

Costs Have Increased

Despite the improved performance in 1996, SCRIPS is still doing much less

than expected. In addition, estimated costs have increased and estimated

labor cost savings have decreased.

SCRIPS Processed Fewer

Documents Than Expected in

1996

The October 1994 business case stated that in 1996 SCRIPS would be

processing (1) all Forms 1040EZ, (2) all FTD coupons, (3) all information

returns, (4) 93 percent of the Forms 941, and (5) 50 percent of the Forms

1040PC. During the first 9 months of 1996, as in 1995, SCRIPS processed all

the FTD coupons IRS had received but no forms 1040PC or 941. Also,

although SCRIPS processed more Forms 1040EZ and information returns

during the first 9 months of 1996 compared with the first 9 months of 1995,

it still processed only about 50 percent of the Forms 1040EZ and about

60 percent of paper information returns. As discussed in the next few

paragraphs, this reduced level of performance, compared to the

October 1994 business case, stems in large part from a major change in IRS’

plans after the SCRIPS contract was awarded. Furthermore, IRS officials did

not know the extent to which hardware and software modifications that

had been made for 1996 and those that were planned for later in the year

Page 16 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

would affect SCRIPS’ ability to process more Forms 1040EZ and information

returns. Therefore, IRS officials did not significantly increase the

expectations for SCRIPS in 1996 over those that they had in 1995.

As specified in the contract, SCRIPS was designed on the assumption that it

would be installed in each of the 10 service centers that were then

processing paper returns. However, in December 1993—10 months after

the contract was awarded—IRS announced plans to consolidate the

processing of paper tax returns in five centers. As a result, although the

total workload for SCRIPS remained the same, the volume to be processed

by any one of the five SCRIPS service centers on the average doubled. Thus,

the system that the contractor had designed would not meet the workload

requirements without further systems development work.

In May 1994, IRS attempted to revise the original contract to meet the

volume requirements at five service centers. According to IRS’

post-implementation review report, a statement of work was written to

revise the contract requirements to accommodate the five service center

scenario. In December 1994, at IRS’ request, the contractor proposed

changes totaling about $21 million. According to contractor officials, that

proposal represented an interim attempt to meet the new workload

requirements under a five service center scenario.

According to IRS officials, IRS could not afford all of the proposed

modifications. Also, IRS officials did not believe that all the proposed

changes were needed. As a result, negotiations on these proposals were

never completed. Thus, IRS decided to purchase five systems, one for each

of the five SCRIPS service centers. To compensate for having fewer systems

than intended, IRS decided to (1) continue paper processing of Forms

1040EZ in the other five service centers and (2) extend the maintenance

contract for the OCR equipment that was being used to process information

returns.

After the 1995 filing season, IRS issued a statement of work for system

enhancements that would help stabilize SCRIPS performance in 1996.

According to IRS officials, they purchased those enhancements that they

believed offered the greatest potential for improving SCRIPS’ performance

in fiscal year 1996. For example, IRS purchased a third scanner for each

service center that could (1) be used as a backup if one of the two primary

scanners failed and (2) provide the ability to scan documents ahead and

have them wait in queue for further processing. Many of these

Page 17 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

enhancements were implemented late in the 1996 filing season, after most

of the Forms 1040EZ and information returns had been processed.

According to IRS’ Office of Assistant Chief Counsel, IRS cannot hold the

contractor to any specific performance requirements for the number of

documents that SCRIPS must process within a specific time period, (e.g., per

week, per hour). When IRS modified the contract to reflect that SCRIPS

would be put in 5 centers instead of 10, according to the Assistant Chief

Counsel’s Office, it did not clearly establish throughput requirements—the

number of documents to be scanned per hour. Thus, despite paying for

enhancements for 1996, IRS has determined that it cannot currently hold

the contractor to any specific performance requirements. In

February 1996, IRS Internal Audit recommended that IRS finalize throughput

requirements for SCRIPS and do a test to determine whether the contractor

is meeting the requirements. IRS officials told us that they are examining

options for incorporating throughput requirements into the contract.

IRS officials said that they would have a better foundation for establishing

throughput requirements once IRS evaluates the impact of the

enhancements on SCRIPS’ capacity. According to contractor officials, given

the enhancements made in 1996, they believe SCRIPS is capable of

processing more documents than it has. They pointed out that SCRIPS’

capability to process documents depends not only on the system’s design

but also on IRS’ human resource decisions, such as the number of work

stations that are staffed and employee incentives (or lack thereof).

IRS officials tested SCRIPS in late September 1996. According to the test

plan, the purpose of the test was to determine (1) the maximum number of

FTD coupons, Forms 1040EZ, and information returns that SCRIPS can

process; (2) the amount of free time, if any, that will be available to

process Forms 941; (3) any bottlenecks in the SCRIPS system that can be

eliminated to increase system throughput; and (4) the performance

thresholds that could be put into the SCRIPS contract. However, because

the software application for Form 941 was not complete, it would have

been difficult to fully assess SCRIPS’ processing capability. The results of

the test were not available when we completed our audit work.

Expected Costs Have Increased

and Anticipated Labor Cost

Savings Have Decreased

Although the workload being processed on SCRIPS continues to be less than

expected, SCRIPS’ estimated costs have risen. Also, anticipated labor cost

savings have decreased.

Page 18 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

According to IRS’ post-implementation review report, previous cost

estimates for SCRIPS have ranged from $133 million to $209 million. The

estimate of $133 million was made in February 1992 and again in

April 1992. That cost estimate, however, assumed that SCRIPS would be

implemented in 1994 and did not include the cost of maintaining SCRIPS.

The current life-cycle cost estimate for SCRIPS is $288 million, which

includes at least $20 million for maintenance.17 That estimate was included

in the Department of the Treasury’s May 6, 1996, report to Congress on IRS’

progress in responding to our recommendations on the managerial and

technical weaknesses of TSM.

18

We could not determine how much IRS has already spent on SCRIPS since its

inception because IRS does not have an accurate cost accounting system.

Using the latest life-cycle cost estimate, SCRIPS is estimated to have cost

about $145 million from fiscal year 1989 through fiscal year 1996.

In October 1994, IRS estimated that SCRIPS would provide about $17 million

in labor cost savings from fiscal years 1994 through fiscal year 2000. In

September 1995, IRS lowered that estimate to about $5 million. Also, IRS’

September 1996 investment evaluation report on SCRIPS concluded that the

system will yield a negative return on investment (i.e., costs will exceed

benefits) from 1991 to 2001. However, the evaluation report stated that its

return on investment estimate does not fully capture the operational

benefits of the hardware and software enhancements that were made

since the end of fiscal year 1995. Therefore, the report concludes that the

final judgment on SCRIPS’ performance cannot be made until after the 1997

filing season.

Use of SCRIPS for

Form 941 Is Uncertain

As discussed previously, SCRIPS was not used to process forms 1040PC and

941 in 1995 or 1996. In July 1995, IRS decided to terminate all software

programming for the 1040PC because of SCRIPS’ instability, a lack of system

capacity, and a number of software application problems. It is uncertain

when, if at all, SCRIPS will be used to process Forms 941.

The President’s fiscal year 1997 budget request included $850 million for

TSM, about $38 million of which was for SCRIPS. In a June 6, 1996, letter,

Treasury submitted a revised TSM funding request of $664 million to the

House Appropriations Committee, of which about $30 million was for

17See footnote 7.

18Report to House and Senate Appropriations Committees: Progress Report on IRS’ Management and

Implementation of Tax Systems Modernization, May 6, 1996.

Page 19 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

SCRIPS. According to the letter, at this funding level, SCRIPS would not be

used to process Forms 941 for fiscal year 1997. Congress subsequently

appropriated $336 million for TSM for fiscal year 1997. About 60 percent of

that appropriation is earmarked for operational TSM projects, such as

SCRIPS, but it was unclear when we prepared this report how much IRS

would allocate to SCRIPS. However, we would expect that IRS will continue

funding SCRIPS because the project is operational and IRS has no backup

system for FTD coupons.

In addition to funding constraints, decisions on the use of SCRIPS beyond

fiscal year 1997 for Forms 941, according to IRS officials, will hinge on:

(1) the results of the September 1996 test of SCRIPS, which we discussed

earlier; and (2) Investment Review Board actions on recommendations

made by a paper processing task team.

In March 1996, IRS convened a task team to develop a paper processing

strategy due to delays in implementing various aspects of IRS’ tax return

processing vision. Specifically, IRS had originally expected that (1) a

significant number of returns would be received electronically, thereby

reducing the need for some manual data entry; and (2) DPS would be

positioned to begin processing the remaining paper tax returns. As we

reported in October 1995, IRS’ electronic filing program is falling short of

expectations.19 Current estimates indicate that IRS may receive only

33 million electronic returns in 2001 rather than the 80 million that IRS had

set as its goal. The shortfall in meeting IRS’ 80-million goal stems from the

lack of a comprehensive business strategy to attract taxpayers to

electronic filing. IRS is currently trying to develop such a strategy. Also, IRS

announced on October 8, 1996, that it was terminating the DPS project

because of “revised priorities and budget realities for the next several

years.”

Among other things, the task team developed a paper processing strategy

based on the assumption that the maximum number of electronic returns

that can be expected in 2001 is about 33 million. In addition, the team

evaluated the need for a replacement system for IRS’ current manual data

entry system. According to IRS officials, one part of the paper processing

strategy may be to contract out the processing of some documents. These

documents could include information returns because the processing of

those documents is not as time-sensitive as the processing of income tax

returns. In addition, in the future, more FTD coupons are to be submitted

electronically as called for in the North American Free Trade Agreement.

19Tax Administration: Electronic Filing Falling Short of Expectations (GAO/GGD-96-12, Oct. 31, 1995).

Page 20 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

For example, IRS estimates that about 44 percent of the FTD coupons will

be received electronically in fiscal year 1998, compared with only

1 percent in fiscal year 1996. Thus, given that some of SCRIPS’ existing

workload could decrease or be contracted out, IRS could possibly increase

the number of Forms 1040EZ that SCRIPS could process or add additional

forms, such as Forms 941, without requiring any additional system

capacity beyond what is currently available.

Conclusions Because SCRIPS was developed before IRS started taking actions to address

the managerial and technical weaknesses of TSM that we identified in

July 1995, SCRIPS suffered from some of those weaknesses that we said

would contribute to such things as cost overruns and failure to meet

mission goals. SCRIPS is expected to cost more than originally expected

and, according to IRS’ September 1995 estimates, could provide less than

one-third of the originally expected labor cost savings. We recognize that

IRS is now completely reliant on SCRIPS for processing paper FTD coupons.

However, the decrease in expected labor cost savings and the increase in

estimated costs raise questions about the cost-effectiveness of SCRIPS.

In addition, one of the most critical weaknesses for SCRIPS was a lack of

thorough and complete system testing before the system was rolled out to

five service centers. Although IRS tested SCRIPS in September 1996 to

determine the maximum number of Forms 1040EZ, information returns,

and FTD coupons SCRIPS can process, that test does not substitute for a test

of SCRIPS’ ability to process any new document types along with the

existing ones under a production environment that replicates peak volume

conditions.

Recommendation to

the Commissioner of

Internal Revenue

We recommend that before deciding to increase the percentage of Forms

1040EZ or information returns that SCRIPS processes or using SCRIPS to

process other tax forms, such as Forms 941, the Commissioner (1) do a

cost-benefit analysis that includes examining the costs and benefits of

alternative ways for processing those forms, such as those developed by

the paper processing task team; and (2) if the analysis shows that it is

cost-effective to have SCRIPS process Forms 941, ensure that IRS tests

SCRIPS’ ability to process the existing software applications along with any

new software applications that may be added using peak volumes or

volumes simulating peak workload conditions to ensure that SCRIPS can

meet performance expectations.

Page 21 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

Agency Comments

and Our Evaluation

We requested comments on a draft of this report from the Commissioner

of Internal Revenue or her designated representative. Responsible IRS

officials, including the National Director for Submission Processing, the

Assistant Commissioner for Forms and Submission Processing, and the

SCRIPS project manager, provided IRS’ comments in a November 14, 1996,

meeting. Their comments on our recommendation were reiterated in a

November 19, 1996, memorandum from the Acting Chief of Taxpayer

Service. Besides commenting on our recommendation, the Deputy Chief

Information Officer, Systems Development, provided several factual

clarifications that we incorporated in the report where appropriate. The

officials also asked that we update our report to reflect processing rate

information through September 30, 1996, and they gave us the relevant

data. We made that update.

IRS officials generally agreed with our recommendation. With respect to

the first part of our recommendation, the officials generally agreed that IRS

should do a cost-benefit analysis before deciding to increase the

percentage of Forms 1040EZ or information returns that SCRIPS processes

or to add additional forms, such as Forms 941. They said that this analysis

would be done by the end of fiscal year 1997 and would take into account

approved results from the paper processing task team. IRS officials said

that in the event of workload imbalances during the 1997 tax return filing

season, if necessary, they may decide to increase the workload for SCRIPS

before the cost/benefit analysis is completed. We recognize that IRS’

priority must be to process tax returns during the filing season in a timely

manner, and it needs to reserve the right to do so.

With respect to the second part of our recommendation, IRS officials said

that they plan to work with the contractor to identify and implement any

necessary contract modifications that may be needed to ensure complete

testing. They said the testing will ensure that SCRIPS can meet performance

expectations in a peak production environment.

IRS officials said that a few changes are planned for the 1997 filing season

that could help improve SCRIPS’ future performance. Specifically, they

mentioned that IRS would be testing different incentive systems for SCRIPS

operators to determine the extent to which incentives affect operator

performance, which could also affect overall SCRIPS’ performance. They

also mentioned that IRS is negotiating with the contractor to provide a new

scanner feeder that should resolve some of the problems experienced

when information returns are filed on extremely thin paper.

Page 22 GAO/GGD-97-29 Performance of SCRIPS
B-270263 

IRS officials expressed concern about using $133 million as the baseline

cost estimate for SCRIPS. That estimate was included in a February 1992

business case and again in a revision dated April 23, 1992. IRS officials said

that we should have used an earlier cost estimate of $209 million that was

included in IRS’ Information Systems Initiative Summary Database in

August 1991. IRS’ post-implementation review report on SCRIPS also stated

that the SCRIPS project office considered the $209 million as the baseline

life-cycle cost estimate. However, the report also noted that between fiscal

years 1993 and 1996, IRS cited four other different life-cycle cost estimates

for budget purposes, including an estimate of $132 million. We revised the

report to acknowledge that $209 million was one of the cost estimates for

SCRIPS, but we believe that the business case estimate is an appropriate

baseline because the business case was a major basis for decisions to go

forward with SCRIPS. IRS officials said that IRS developed the SCRIPS business

case before it began taking steps to manage information technology

projects as investments. Since that time, for example, IRS has developed an

investment justification handbook to help ensure that project cost and

benefit analyses that are included in business cases are standardized and

complete.

We are sending copies of this report to the Subcommittee’s Ranking

Minority Member, the Chairman and Ranking Minority Member of the

House Committee on Ways and Means, the Chairman and Ranking

Minority Member of the Senate Committee on Finance, various other

congressional committees, the Secretary of the Treasury, the

Commissioner of Internal Revenue, the Director of the Office of

Management and Budget, and other interested parties.

Major contributors to this report are listed in the appendix. Please contact

me on (202) 512-9110 if you have any questions.

Sincerely yours,

Lynda D. Willis

Director, Tax Policy and

 Administration Issues

Page 23 GAO/GGD-97-29 Performance of SCRIPS
Appendix I 

Major Contributors to This Report

General Government

Division, Washington,

D.C.

David Attianese, Assistant Director, Tax Policy and Administration Issues

Sherrie Russ, Evaluator-in-Charge

Christopher Hess, Evaluator

Monika Gomez, Evaluator

Kansas City Field

Office

Cecelia Ball, Senior Evaluator

Marvin McGill, Evaluator

(268708) Page 24 GAO/GGD-97-29 Performance of SCRIPS
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100